1. After Integrated Practices: Complementary Approaches to Improving Child Services and Outcomes Bruce F. Chorpita, Ph.D. Eric L. Daleiden, Ph.D. March 18, 2008 Burbank, CA

2. Agenda  Intros and acknowledgements  Background  The “Evidence Based System”  The EBT Literature  PW EBS Database  Common elements  The Clinical Dashboard  Clinical Process Modeling  Modularity  Results so far…  Discussion

3. Traditional Approach to Evidence Based Practice

8. “ Dissemination as Usual ”  Assumes “replacement” approach  Institutionalizes service programs  Assumes an evidence base for all problems  Assumes unlimited resources and learning capacity

9. More Concerns  Fixed content  Fixed intensity  Fixed length  Single target approach  Too few choices  Too many choices  Expiration problem Aarons (2004); Addis & Krasnow (2000); Addis, Wade, & Hatgis (2004); Chorpita, Daleiden, & Weisz (2005); Kimhan & Chorpita (2006); Persons (1995)

10. Throw out the Bath Water: Keep the Baby  Move away from the idea that “Problem A gets Treatment B”  We decided to map the decisions made at the service system level – not just the IF- THEN of treatment selection  Goal: To build an Evidence Based System

11. Alternative: The Evidence Based System

12. Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

13. Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Multisystemic Therapy (MST) Example Home & Community Based TAMS & SAMS Instrumental & Ultimate Outcomes MST 4 – 5 Members Team Supervisor Cross-Team Supervisor Family Therapy, etc.

14. How should we select a program? Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

15. How should we select a program? General Services Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Evidence-Based Services Model

16. How should we select a program? Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Individualized Case Conceptualization Model

17. How should we select a program? Local Aggregate Evidence Case-Specific Historical Information Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Practice-Based Evidence Model

18. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

19. Traditional EBS Model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

20. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

21. Tx Program Selection Valid alternatives to deciding by program? General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

22. Common Elements Approach Stay tuned…

23. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

24. A Brief Aside on Evidence Bases

25. The Phases of Evidence 1.Data: Discretely identifiable units 2.Information: Data in a context that provides it meaning 3.Knowledge: Information helpful to decision- making 4.Wisdom: Knowing when to apply our knowledge Speigler, I. (2000). Knowledge management: A new idea or a recycled concept? Communications of the Association for Information Systems, 3, 1 – 23.

26. The Phases of Evidence: Example 1.Data: 65 2.Information: 65° F 3.Knowledge: It warm enough for me to wear a T-shirt. 4.Wisdom: I will be giving a professional talk, so the temperature outside is irrelevant to my attire.

27. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

28. General Services Research: Turning Data into Knowledge I  Meta Analysis of Children’s Treatment Research  322 studies  41 years  832 study groups  25,435 youth participants  $407 million in today’s dollars  Largest meta-analysis to date  Automated for practitioners

29. ANXIETY 145 protocols, 18 treatment families

30. Results: Anxiety Best Support  CBT (26)  Exposure (24)  Modeling (7)  Education (2) Good Support  Cognitive Behavior Therapy with Parents (2)  Relaxation (2)  Cognitive Behavior Therapy and Meds (1)  Hypnosis (1)  Assertiveness Training (1)

31. Results: Anxiety  Attention  Client Centered Therapy  EMDR  Relationship Counseling  Teacher Psychoeducation  No Support

32. ATTENTION & HYPERACTIVITY 67 protocols, 18 treatment families

33. Results: Attention & Hyperactivity Best Support  Self verbalization (4)  Behavior Therapy and Medication (2) Good Support  Parent Management Training (5)  Contingency Management (4)  Physical Exercise (3)  Biofeedback (2)  Social Skills and Meds (2)  Relaxation and Exercise (1)  Parent Management Training and Problem Solving (1)  Education (1)

34. Results: Attention & Hyperactivity  Client Centered Therapy  Self-Control Training  Skill Development  Parent Management Training and Self-Verbalization*  No Support

35. AUTISM SPECTRUM 21 protocols, 3 treatment families

36. Results: Autism Spectrum  Intensive Behavioral Treatment (4)  Intensive Communication Training (3)  Best Support

37. Results: Autism Spectrum  Auditory Integration Training  No Support

38. DEPRESSION 39 protocols, 15 treatment families

39. Results: Depression Best Support  Cognitive Behavior Therapy (12)  Cognitive Behavior Therapy and Medication (2) Good Support  Interpersonal Therapy (2)  Relaxation (2)  Cognitive Behavior Therapy with Parents (2)  Client Centered Therapy (1)  Family Therapy (1)

40. Results: Depression  Attention  Counselors Care  Counselors Care plus Anger Management  Life Skills  Problem Solving  Social Skills  No Support

41. DISRUPTIVE BEHAVIOR 173 protocols, 34 treatment families

42. Results: Disruptive Behavior Best Support  Parent Management Training (26)  Contingency Management (9)  Multisystemic Therapy (9)  Social Skills (6)  Cognitive Behavior Therapy (6)  Assertiveness Training (2) Good Support  Problem Solving (7)  Communication Skills (3)  Relaxation (2)  Parent Management Training and Problem Solving (2)  Client Centered Therapy (2)  Anger Control (2)  Rational Emotive Therapy (1)  Multidimensional Treatment Foster Care (1)  Functional Family Therapy (1)  Transactional Analysis (1)

43. Results: Disruptive Behavior  Assertiveness Training  Attention  Catharsis  Collaborative Problem Solving  Education  Exposure  Family Empowerment  Family Systems Therapy  Group Therapy (!!)  Life Skills  Project CARE (!!)  Psychodynamic  Self Verbalization  Skill Development  No Support

44. TRAUMATIC STRESS 16 protocols, 7 treatment families

45. Results: Traumatic Stress Best Support  Cognitive Behavior Therapy with Parents (5) Good Support  Cognitive Behavior Therapy (4)

46. Results: Traumatic Stress  Client Centered Therapy  Cognitive Behavior Therapy with Parents Only  EMDR  No Support

47. BREAK

48. THE PW DATABASE How do I keep track of all this information?

49. Live Demo Possible?

50. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

51. Treatment Team The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Supervision Client Progress Treatment Integrity

52. General Services Research: Turning Data into Knowledge II  Common Elements approach  Identified components of evidence based practices  Complements integrated program approach

53. Is there a different level of analysis? Protocol Family

54. Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training

55. Is there a different level of analysis? Protocol Family Practice Element Practice Element Practice Element Practice Element Practice Element Practice Element

56. Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training Commands Attending Time Out Rewards Time Out These are “practice elements.”

57. Example Attending

58. Anxiety

59. ADHD

60. Autism

61. Depression

62. Disruptive Behavior

64. Traumatic Stress

65. BREAK

66. The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

67. Case Specific History: Turning Data into Knowledge III  Clinical Dashboard  Progress  Practices

69. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

71. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

72. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

73. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

75. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

76. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

78. A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

79. This tells you the treatment types that work for this problem.

80. This tells you the practice elements associated with those treatment types.

81. Do the practices fit the problem?

82. A Bit of Practice Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, PW DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

83. Select new practices?

84. Seek Consultation?

85. Handling Caseloads

91. BREAK

92. MODULARITY IN PRACTICE

93. Modular CBT for Anxiety

94. Fear Ladder Social Skills Rewards Learning About Anxiety Practice Cognitive Restructuring Maintenance Finish

95. Fear Ladder Practice: In Vivo Learning About Anxiety Practice: Imaginal in vivo possible? more items to practice? Maintenance yes no Finish

96. no Fear Ladder In Vivo Exposure Learning About Anxiety Imaginal Exposure child ready to practice? in vivo possible? more items to practice? Maintenance yes no Finish Interference

97. no yes Fear Ladder Social Skills: Meeting People Time Out Active Ignoring Rewards In Vivo Exposure Learning About Anxiety Imaginal Exposure Cognitive Restructuring: Probability Cognitive Restructuring: STOP Social Skills: Nonverbals child ready to practice? mild disruptive behavior? low motivation? parents rewarding avoidance? negative beliefs or depression? moderate disruptive behavior? in vivo possible? more items to practice? Maintenance social skills deficits? other Cognitive Restructuring: Catastrophic bright, verbal, or older? troubleshoot yes no Finish

98. Modular CBT Outcomes  Small N experimental design  Better than monitoring control  7 of 7 diagnosis free at post  6 of 7 diagnosis free at follow-up

99. Modular CBT for Anxiety

101. MATCH-ADC: Modular Approach to Therapy for Children

102. Three Steps Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN

103. Three Steps Complete next in sequence Interference Evaluate & Triage Yes No Able to proceed Yes No Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN Gains Complete

104. Three Steps Complete next in sequence Exposure Fear Related Behavior Related Interference Other Evaluate & Triage Yes No Able to proceed Yes No Commands Active Ignoring Time Out Rewards Anticipating Problems Non compliance Specific Triggers Attention Seeking Low Motivation Serious Behavior Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN Gains Complete

105. Child STEPs Treatment Project  Youth Network on Child Mental Health  5-Year, multisite randomized trial  Boston, Honolulu  Anxiety, Depression, Conduct Problems  Community therapists  SMT, MMT, Usual Care  180 children enrolled so far

106. Training Results p <.01

107. Treatment Results  Preliminary results look promising…

108. Other Efforts  Minnesota Department of Human Services  Hawaii Child and Adolescent Mental Health Division

109. Sample Practice Element from Practitioner Guide

111. One Size Does Not Fit All DashboardsProtocolsEvidence Review Hawaii CAMHDCAMHMISPW Practitioner’s Guide; Integral protocols (e.g., MST) EBS Biennial Report; Blue Menu Minnesota DHSMS Office VersionPW Practitioner’s Guide PW EBS Website Child STEPsSTEPs MCTP MISMATCH-ADCMacArthur Phase I University of Hawaii MS Office VersionMATCH-ADC Modular CBT EBS Biennial Report; Blue Menu Each also has unique decision model…

112. Results in Hawaii From Daleiden, Chorpita, Arensdorf, Donkervoet, & Brogan (2006)

113. Avoid Hospital Residential Services for Conduct Disorders: MST Initiative MST Began EBS Task Force Began

114. Quality Reviews: System Performance Quality Dimension Examples: Functional AssessmentService Coordination & Transition Long-term viewCaregiver Supports Service Plan & ImplementationEffective Results Service Array & Integration Monitoring & Modification

115. Quality Reviews: Child Status Quality Dimension Examples: Learning ProgressCommunity Home Personal ResponsibilityCaregiver Functioning Safety/Personal Well-beingChild/Family Satisfaction Emotional/Behavioral Well-being

116. Rate of Improvement? Final Effect Size for Change =.07/mo,.84/yr

117. Getting Better at Getting Them Better End of 2004 Start of 2002

118. Greater Impairment Treated at Less Restrictive Levels of Care

119. Cost per Outcome

120. On the Horizon…  Distillation tree  Relevance mapping  Better understanding of known risks

121. Summary  Modeling the clinical decision process  Multiple evidence bases  Dashboards  Common elements  Modularity  Emphasis on the system, not just the Treatment

122. Review and Discussion

123. What seems promising? What doesn’t? What are the primary current and future needs of agencies in CA? Are there ways to structure practice development efforts across multiple agencies or counties? Are there economies of scale for new initiatives? Is there interest in research collaboratives or partnerships?

124. Thank you!

125. Contact Information  chorpita@hawaii.edu  chorpita@psych.ucla.edu (after July 1)  eric@daleiden.com http://www.practicewise.com