After Integrated Practices: Complementary Approaches to Improving Child Services and Outcomes Bruce F. Chorpita, Ph.D. Eric L. Daleiden, Ph.D. March 18, 2008 Burbank, CA

Agenda  Intros and acknowledgements  Background  The “Evidence Based System”  The EBT Literature  PW EBS Database  Common elements  The Clinical Dashboard  Clinical Process Modeling  Modularity  Results so far…  Discussion

Traditional Approach to Evidence Based Practice

“ Dissemination as Usual ”  Assumes “replacement” approach  Institutionalizes service programs  Assumes an evidence base for all problems  Assumes unlimited resources and learning capacity

More Concerns  Fixed content  Fixed intensity  Fixed length  Single target approach  Too few choices  Too many choices  Expiration problem Aarons (2004); Addis & Krasnow (2000); Addis, Wade, & Hatgis (2004); Chorpita, Daleiden, & Weisz (2005); Kimhan & Chorpita (2006); Persons (1995)

Throw out the Bath Water: Keep the Baby  Move away from the idea that “Problem A gets Treatment B”  We decided to map the decisions made at the service system level – not just the IF- THEN of treatment selection  Goal: To build an Evidence Based System

Alternative: The Evidence Based System

Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Multisystemic Therapy (MST) Example Home & Community Based TAMS & SAMS Instrumental & Ultimate Outcomes MST 4 – 5 Members Team Supervisor Cross-Team Supervisor Family Therapy, etc.

How should we select a program? Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

How should we select a program? General Services Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Evidence-Based Services Model

How should we select a program? Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Individualized Case Conceptualization Model

How should we select a program? Local Aggregate Evidence Case-Specific Historical Information Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Practice-Based Evidence Model

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Traditional EBS Model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Tx Program Selection Valid alternatives to deciding by program? General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Common Elements Approach Stay tuned…

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

A Brief Aside on Evidence Bases

The Phases of Evidence 1.Data: Discretely identifiable units 2.Information: Data in a context that provides it meaning 3.Knowledge: Information helpful to decision- making 4.Wisdom: Knowing when to apply our knowledge Speigler, I. (2000). Knowledge management: A new idea or a recycled concept? Communications of the Association for Information Systems, 3, 1 – 23.

The Phases of Evidence: Example 1.Data: 65 2.Information: 65° F 3.Knowledge: It warm enough for me to wear a T-shirt. 4.Wisdom: I will be giving a professional talk, so the temperature outside is irrelevant to my attire.

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

General Services Research: Turning Data into Knowledge I  Meta Analysis of Children’s Treatment Research  322 studies  41 years  832 study groups  25,435 youth participants  $407 million in today’s dollars  Largest meta-analysis to date  Automated for practitioners

ANXIETY 145 protocols, 18 treatment families

Results: Anxiety Best Support  CBT (26)  Exposure (24)  Modeling (7)  Education (2) Good Support  Cognitive Behavior Therapy with Parents (2)  Relaxation (2)  Cognitive Behavior Therapy and Meds (1)  Hypnosis (1)  Assertiveness Training (1)

Results: Anxiety  Attention  Client Centered Therapy  EMDR  Relationship Counseling  Teacher Psychoeducation  No Support

ATTENTION & HYPERACTIVITY 67 protocols, 18 treatment families

Results: Attention & Hyperactivity Best Support  Self verbalization (4)  Behavior Therapy and Medication (2) Good Support  Parent Management Training (5)  Contingency Management (4)  Physical Exercise (3)  Biofeedback (2)  Social Skills and Meds (2)  Relaxation and Exercise (1)  Parent Management Training and Problem Solving (1)  Education (1)

Results: Attention & Hyperactivity  Client Centered Therapy  Self-Control Training  Skill Development  Parent Management Training and Self-Verbalization*  No Support

AUTISM SPECTRUM 21 protocols, 3 treatment families

Results: Autism Spectrum  Intensive Behavioral Treatment (4)  Intensive Communication Training (3)  Best Support

Results: Autism Spectrum  Auditory Integration Training  No Support

DEPRESSION 39 protocols, 15 treatment families

Results: Depression Best Support  Cognitive Behavior Therapy (12)  Cognitive Behavior Therapy and Medication (2) Good Support  Interpersonal Therapy (2)  Relaxation (2)  Cognitive Behavior Therapy with Parents (2)  Client Centered Therapy (1)  Family Therapy (1)

Results: Depression  Attention  Counselors Care  Counselors Care plus Anger Management  Life Skills  Problem Solving  Social Skills  No Support

DISRUPTIVE BEHAVIOR 173 protocols, 34 treatment families

Results: Disruptive Behavior Best Support  Parent Management Training (26)  Contingency Management (9)  Multisystemic Therapy (9)  Social Skills (6)  Cognitive Behavior Therapy (6)  Assertiveness Training (2) Good Support  Problem Solving (7)  Communication Skills (3)  Relaxation (2)  Parent Management Training and Problem Solving (2)  Client Centered Therapy (2)  Anger Control (2)  Rational Emotive Therapy (1)  Multidimensional Treatment Foster Care (1)  Functional Family Therapy (1)  Transactional Analysis (1)

Results: Disruptive Behavior  Assertiveness Training  Attention  Catharsis  Collaborative Problem Solving  Education  Exposure  Family Empowerment  Family Systems Therapy  Group Therapy (!!)  Life Skills  Project CARE (!!)  Psychodynamic  Self Verbalization  Skill Development  No Support

TRAUMATIC STRESS 16 protocols, 7 treatment families

Results: Traumatic Stress Best Support  Cognitive Behavior Therapy with Parents (5) Good Support  Cognitive Behavior Therapy (4)

Results: Traumatic Stress  Client Centered Therapy  Cognitive Behavior Therapy with Parents Only  EMDR  No Support

BREAK

THE PW DATABASE How do I keep track of all this information?

Live Demo Possible?

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Treatment Team The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Supervision Client Progress Treatment Integrity

General Services Research: Turning Data into Knowledge II  Common Elements approach  Identified components of evidence based practices  Complements integrated program approach

Is there a different level of analysis? Protocol Family

Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training

Is there a different level of analysis? Protocol Family Practice Element Practice Element Practice Element Practice Element Practice Element Practice Element

Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training Commands Attending Time Out Rewards Time Out These are “practice elements.”

Example Attending

Anxiety

ADHD

Autism

Depression

Disruptive Behavior

Traumatic Stress

BREAK

The full system model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Case Specific History: Turning Data into Knowledge III  Clinical Dashboard  Progress  Practices

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

This tells you the treatment types that work for this problem.

This tells you the practice elements associated with those treatment types.

Do the practices fit the problem?

A Bit of Practice Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, PW DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

Select new practices?

Seek Consultation?

Handling Caseloads

BREAK

MODULARITY IN PRACTICE

Modular CBT for Anxiety

Fear Ladder Social Skills Rewards Learning About Anxiety Practice Cognitive Restructuring Maintenance Finish

Fear Ladder Practice: In Vivo Learning About Anxiety Practice: Imaginal in vivo possible? more items to practice? Maintenance yes no Finish

no Fear Ladder In Vivo Exposure Learning About Anxiety Imaginal Exposure child ready to practice? in vivo possible? more items to practice? Maintenance yes no Finish Interference

no yes Fear Ladder Social Skills: Meeting People Time Out Active Ignoring Rewards In Vivo Exposure Learning About Anxiety Imaginal Exposure Cognitive Restructuring: Probability Cognitive Restructuring: STOP Social Skills: Nonverbals child ready to practice? mild disruptive behavior? low motivation? parents rewarding avoidance? negative beliefs or depression? moderate disruptive behavior? in vivo possible? more items to practice? Maintenance social skills deficits? other Cognitive Restructuring: Catastrophic bright, verbal, or older? troubleshoot yes no Finish

Modular CBT Outcomes  Small N experimental design  Better than monitoring control  7 of 7 diagnosis free at post  6 of 7 diagnosis free at follow-up

Modular CBT for Anxiety

MATCH-ADC: Modular Approach to Therapy for Children

Three Steps Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN

Three Steps Complete next in sequence Interference Evaluate & Triage Yes No Able to proceed Yes No Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN Gains Complete

Three Steps Complete next in sequence Exposure Fear Related Behavior Related Interference Other Evaluate & Triage Yes No Able to proceed Yes No Commands Active Ignoring Time Out Rewards Anticipating Problems Non compliance Specific Triggers Attention Seeking Low Motivation Serious Behavior Social Skills Cognitive: BLUE Skill Building Relaxation Activity Selection Problem Solving Psychoed Child Psychoed - Parent Covert Relaxation Wrap up Cognitive: FUN Gains Complete

Child STEPs Treatment Project  Youth Network on Child Mental Health  5-Year, multisite randomized trial  Boston, Honolulu  Anxiety, Depression, Conduct Problems  Community therapists  SMT, MMT, Usual Care  180 children enrolled so far

Training Results p <.01

Treatment Results  Preliminary results look promising…

Other Efforts  Minnesota Department of Human Services  Hawaii Child and Adolescent Mental Health Division

Sample Practice Element from Practitioner Guide

One Size Does Not Fit All DashboardsProtocolsEvidence Review Hawaii CAMHDCAMHMISPW Practitioner’s Guide; Integral protocols (e.g., MST) EBS Biennial Report; Blue Menu Minnesota DHSMS Office VersionPW Practitioner’s Guide PW EBS Website Child STEPsSTEPs MCTP MISMATCH-ADCMacArthur Phase I University of Hawaii MS Office VersionMATCH-ADC Modular CBT EBS Biennial Report; Blue Menu Each also has unique decision model…

Results in Hawaii From Daleiden, Chorpita, Arensdorf, Donkervoet, & Brogan (2006)

Avoid Hospital Residential Services for Conduct Disorders: MST Initiative MST Began EBS Task Force Began

Quality Reviews: System Performance Quality Dimension Examples: Functional AssessmentService Coordination & Transition Long-term viewCaregiver Supports Service Plan & ImplementationEffective Results Service Array & Integration Monitoring & Modification

Quality Reviews: Child Status Quality Dimension Examples: Learning ProgressCommunity Home Personal ResponsibilityCaregiver Functioning Safety/Personal Well-beingChild/Family Satisfaction Emotional/Behavioral Well-being

Rate of Improvement? Final Effect Size for Change =.07/mo,.84/yr

Getting Better at Getting Them Better End of 2004 Start of 2002

Greater Impairment Treated at Less Restrictive Levels of Care

Cost per Outcome

On the Horizon…  Distillation tree  Relevance mapping  Better understanding of known risks

Summary  Modeling the clinical decision process  Multiple evidence bases  Dashboards  Common elements  Modularity  Emphasis on the system, not just the Treatment

Review and Discussion

What seems promising? What doesn’t? What are the primary current and future needs of agencies in CA? Are there ways to structure practice development efforts across multiple agencies or counties? Are there economies of scale for new initiatives? Is there interest in research collaboratives or partnerships?

Thank you!

Contact Information   (after July 1) 