Evidence-Based Clinical Reasoning in Children's Mental Health Services Bruce F. Chorpita Eric L. Daleiden CHARPP Conference: Real Data in Real Time September 10, 2007

Acknowledgments

The Story of John A Tale of Two Cultures

This is John.

Premise n As researchers attempt to implement multiple EBTs, they come across the practitioner’s reality n This was my experience with the CAMHD Practice Development Section

Practitioner Concerns n Fixed content n Fixed intensity n Fixed length n Single target approach n Replacement n Empty cell problem n Crowded cell problem n Expiration problem Aarons (2004); Addis & Krasnow (2000); Addis, Wade, & Hatgis (2004); Chorpita, Daleiden, & Weisz (2005); Kimhan & Chorpita (2006); Persons (1995)

Throw out the Bath Water: Keep the Baby n Move away from the idea that “Problem A gets Treatment B” n We decided to map the decisions made at the system level – not just the IF- THEN of treatment selection n Goal: To build an Evidence Based System

Where should we treat the client? Service Setting

How should we treat the client? Therapeutic Practices Service Setting

Are we providing quality service to the client? Therapeutic Practices Service Setting Treatment Integrity

Is the client getting better? Therapeutic Practices Service Setting Client Progress Treatment Integrity

Who should treat the client? Therapeutic Practices Service Setting Treatment Team Client Progress Treatment Integrity

How should we manage the treatment? Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Treatment programs formalize these elements Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Selecting a program structures other decisions Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Multisystemic Therapy (MST) Example Home & Community Based TAMS & SAMS Instrumental & Ultimate Outcomes MST 4 – 5 Members Team Supervisor Cross-Team Supervisor Family Therapy, etc.

Tx Program Selection Valid alternatives to deciding by program? Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

How should we make decisions? Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

How should we make decisions? General Services Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Evidence-Based Services Model

How should we make decisions? Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Individualized Case Conceptualization Model

How should we make decisions? Local Aggregate Evidence Case-Specific Historical Information Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity Practice-Based Evidence Model

The Full System Model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

The Phases of Evidence 1. Data: Discretely identifiable units 2. Information: Data in a context that provides it meaning 3. Knowledge: Information helpful to decision- making 4. Wisdom: Knowing when to apply our knowledge c.f., Speigler, I. (2000). Knowledge management: A new idea or a recycled concept? Communications of the Association for Information Systems, 3, 1 – 23.

The Phases of Evidence: Example 1. Data: Information: 70º F 3. Knowledge: It is warm enough to wear shorts. 4. Wisdom: I am giving a professional talk today, so my knowledge that it is warm enough to wear shorts is irrelevant to my attire.

The Phases of Evidence: Managing Evidence Bases 1. Data: Define and capture relevant data 2. Information: Organize and analyze into meaningful units for consumers 3. Knowledge: Deliver the information to the decision-making situation 4. Wisdom: Prioritize use of the knowledge bases

General Services Research: Turning Data into Knowledge I n Meta Analysis of Children’s Treatment Research –Over 300 studies, 60 years –Largest meta-analysis to date –Made dynamic for providers

Results: Anxiety Best Support n CBT n Education n Exposure n Modeling Good Support n Assertiveness Training n Cognitive Behavior Therapy and Medication n Cognitive Behavior Therapy with Parents n Hypnosis n Play Therapy n Relaxation

Results: Trauma Best Support n CBT Good Support n Cognitive Behavior Therapy with Parents n Play Therapy

Results: Attention Best Support n Contingency Management n Parent Management Training n Self Verbalization n Behavior Therapy and Medication Good Support n Biofeedback n Contingency Management n Education n Management Training and Problem Solving n Physical Exercise n Relaxation and Physical Exercise n Social Skills and Medication

Results: Depression Best Support n CBT n CBT and Medication Good Support n Client Centered Therapy n Cognitive Behavior Therapy with Parents n Interpersonal Therapy n Play Therapy n Relaxation

Results: Conduct Best Support n Assertiveness Training n Cognitive Behavior Therapy n Contingency Management n Exposure n Modeling n Parent Management Training n Social Skills Good Support n Anger Control n Client Centered Therapy n Communication Skills n Functional Family Therapy n Multidimensional Treatment Foster Care n Multisystemic Therapy n Parent Management Training and Problem Solving n Physical Exercise n Problem Solving n Rational Emotive Therapy n Relaxation n Transactional Analysis

How do we get this knowledge to the treatment team? General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

General Services Research: Turning Data into Knowledge II n Common Elements approach –Identified components of evidence based practices –Generated profiles using a decision tree approach

Is there a different level of analysis? Protocol Family

Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training

Is there a different level of analysis? Protocol Family Practice Element Practice Element Practice Element Practice Element Practice Element Practice Element

Is there a different level of analysis? Incredible Years PCIT Defiant Children Parent Training Commands Attending Time Out Rewards Time Out These are “practice elements.”

Example Attending

Exposure Relaxation Cognitive Psychoeducational-Child Modeling Self-Monitoring Therapist Praise/Rewards Psychoeducational-Parent Self-Reward/Self-Praise Maintenance/Relapse Prevention Problem Solving Tangible Rewards Relationship/Rapport Building Assertiveness Training Praise Social Skills Training Guided Imagery Communication Skills Educational Support Response Prevention Differential Reinforcement Goal Setting Play Therapy Emotional Processing Natural and Logical Consequences Parent Coping Monitoring Peer Pairing Stimulus Control or Antecedent Man. Time Out Behavioral Contracting Activity Scheduling Catharsis Family Therapy Hypnosis Supportive Listening Individual Therapy for Caretaker PracticeWise Evidence Based Services Practice Element Profile Anxiety Levels I and II – September 05, 2007 Proportion of Studies (n = 91)

PracticeWise Evidence Based Services Practice Element Profile Trauma Levels I and II – September 05, 2007 Proportion of Studies (n = 10) Cognitive Exposure Psychoeducational-Child Relaxation Maintenance/Relapse Prevention Psychoeducational-Parent Assertiveness Training Communication Skills Modeling Problem Solving Emotional Processing Goal Setting Relationship/Rapport Building Self-Monitoring Tangible Rewards Differential Reinforcement Natural and Logical Consequences Parent Coping Praise Monitoring Play Therapy Stimulus Control or Antecedent Man. Time Out

PracticeWise Evidence Based Services Practice Element Profile Attention Levels I and II – September 05, 2007 Proportion of Studies (n = 25) Praise Problem Solving Tangible Rewards Psychoeducational-Parent Stimulus Control or Antecedent Man. Time Out Commands Modeling Monitoring Relaxation Therapist Praise/Rewards Differential Reinforcement Response Cost Social Skills Training Educational Support Self-Reward/Self-Praise Biofeedback/Neurofeedback Communication Skills Parent Coping Guided Imagery Insight Building Goal Setting Natural and Logical Consequences Peer Pairing Relationship/Rapport Building Self-Monitoring Talent or Skill Building Attending Behavioral Contracting

PracticeWise Evidence Based Services Practice Element Profile Depression Levels I and II – September 05, 2007 Proportion of Studies (n = 24) Cognitive Psychoeducational-Child Maintenance/Relapse Prevention Activity Scheduling Problem Solving Self-Monitoring Goal Setting Social Skills Training Relaxation Communication Skills Psychoeducational-Parent Self-Reward/Self-Praise Behavioral Contracting Talent or Skill Building Guided Imagery Modeling Therapist Praise/Rewards Stimulus Control or Antecedent Man. Assertiveness Training Insight Building Parent Coping Relationship/Rapport Building Tangible Rewards Crisis Management Exposure Play Therapy Supportive Listening

PracticeWise Evidence Based Services Practice Element Profile Conduct Levels I and II – September 05, 2007 Proportion of Studies (n = 89) Praise Tangible Rewards Problem Solving Time Out Commands Psychoeducational-Parent Modeling Monitoring Cognitive Differential Reinforcement Communication Skills Goal Setting Response Cost Therapist Praise/Rewards Social Skills Training Maintenance/Relapse Prevention Behavioral Contracting Natural and Logical Consequences Attending Self-Monitoring Relationship/Rapport Building Stimulus Control or Antecedent Man. Parent Coping Family Therapy Psychoeducational-Child Educational Support Peer Pairing Relaxation Talent or Skill Building Functional Analysis Marital Therapy Assertiveness Training Family Engagement Insight Building Crisis Management Individual Therapy for Caretaker Self-Reward/Self-Praise Exposure Play Therapy Guided Imagery Interpretation Activity Scheduling Supportive Listening Line of Sight Supervision Case Management

The Full System Model General Services Research Local Aggregate Evidence Case-Specific Historical Information Causal Mechanism Research Tx Program Selection Therapeutic Practices Service Setting Treatment TeamSupervision Client Progress Treatment Integrity

Case Specific History: Turning Data into Knowledge III n Clinical Dashboard –Progress –Practices

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

A Foray into Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, EBS DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, EBS DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention EBS DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) EBS Database Start

This tells you the treatment types that work for this problem.

This tells you the practice elements associated with those treatment types.

Do the practices fit the problem?

A Bit of Practice Wisdom Clinical Progress? Clinical Dashboard: Progress Pane Continue plan until goals met Significant concerns? Critical Incidents & Complaints Reports, etc. Consult with specialists as needed yes no yes no yes no yes Clinical Dashboard: Practice Pane, PW DB Prob. w/ Tx Selection? Treatment Integrity? Therapy Protocols, Dashboards, PW DB, Consultation Consider adding consultation or training supports Identify barriers and revise plan Options 1. Increase supports 2. Change Intervention 3. Further Consultation 4. Add intervention PW DB, Local Best Practices, Tx Team New Case? no yes Select Evidence- Based Service (EBS) PW Database Start

Select new practices?

Seek Consultation?

Results in Hawaii From Daleiden, Chorpita, Arensdorf, Donkervoet, & Brogan (2006)

Avoid Hospital Residential Services for Conduct Disorders: MST Initiative MST Began EBS Task Force Began

Quality Reviews: System Performance Quality Dimension Examples: Functional AssessmentService Coordination & Transition Long-term viewCaregiver Supports Service Plan & ImplementationEffective Results Service Array & Integration Monitoring & Modification

Quality Reviews: Child Status Quality Dimension Examples: Learning ProgressCommunity Home Personal ResponsibilityCaregiver Functioning Safety/Personal Well-beingChild/Family Satisfaction Emotional/Behavioral Well-being

Rate of Improvement? Final Effect Size for Change =.07/mo,.84/yr

Getting Better at Getting Them Better End of 2004 Start of 2002

Greater Impairment Treated at Less Restrictive Levels of Care

Cost per Outcome

This is John now.

Thank You! Questions?