Evaluating the Impact of an Interconnected Systems Framework Kelly L. Perales, LCSW © 2014 Community Care Behavioral Health Organization

About Community Care Behavioral health managed care company founded in 1996; part of UPMC and headquartered in Pittsburgh Federally tax exempt non-profit 501(c)(3) Major focus is publicly-funded behavioral health care services; currently doing business in PA and NY Licensed as a Risk-Assuming PPO in PA; NCQA- Accredited Quality and Disease Management Programs Serving over 750,000 individuals receiving Medical Assistance in 39 counties through a statewide network of over 1,800 providers 2© 2014 Community Care Behavioral Health Organization

HealthChoices Regions Served 3© 2014 Community Care Behavioral Health Organization Southwest Region Lehigh-Capital Region Southeast Region Northeast Region North Central Region: County North Central Region: State North Central Region: County Community Care Office Pike Erie Crawford Mercer Venango Butler Armstrong Indiana Westmoreland Allegheny Greene Fayette Somerset Cambria Blair Delaware Clarion Forest Warren McKean Potter Cameron Elk Jefferson Clearfield Bedford Centre Clinton Fulton Franklin Adams Cumberland Perry Mifflin Snyder Union Lycoming TiogaBradford Columbia Montour Northumberland Dauphin York Lancaster Chester Berks Lebanon Schuylkill Montgomery Philadelphia Juniata Sullivan Huntingdon Bucks Lehigh Northampton Carbon Monroe Pike Luzerne Wyoming Lackawanna Susquehanna Wayne Lawrence Beaver Washington

CSBBH Community and School Based Behavioral Health (CSBBH) – Families – Advocates – Providers – Schools – Other Child Serving Systems – Counties – Office of Mental Health and Substance Abuse Services (OMHSAS) 4© 2014 Community Care Behavioral Health Organization

CSBBH 35 teams from 12 provider organizations 58 school buildings in 21 school districts 12 counties youth and families 5© 2014 Community Care Behavioral Health Organization

Stakeholder-Reported Outcomes Gathering and reporting can improve care – Evaluation activities integrated into care Desire for connection among families, schools, and community-based services & resources to aid youth Routine tracking of progress can improve outcomes – Stakeholder-reported outcomes to improve care – Clinicians can make more informed adjustments to treatment plans – When discussed with stakeholders, engages & empowers 6© 2014 Community Care Behavioral Health Organization

Consumer-Reported Outcomes Gathering and reporting can improve care Challenges exist: – Burden to collection, scoring, and having information available to discuss in sessions Community Care as a resource to support providers and schools in developing brief process to gather and use outcomes 7© 2014 Community Care Behavioral Health Organization

Collaboration with Multiple Sources 8 School administrators, staff, teachers Providers Parents/caregivers Youth © 2014 Community Care Behavioral Health Organization

Functional and Behavioral Outcomes Collaborative process with provider, teacher, and family input Goal to measure progress in treatment and enhance therapy – Useful across children with multiple diagnoses and different ages – Brief enough to be completed and scored by busy clinicians and families – Sensitive to change, allowing scores to document improvement as child improves during treatment course – Strength based where possible – Can be used to facilitate conversations between families and clinicians 9© 2014 Community Care Behavioral Health Organization

Collection of Outcomes Purpose to gather feedback from caregivers about how they feel treatment is going Graph of results are available immediately via the secure Web portal: – Results of the survey are discussed with caregivers Clinicians use feedback to help establish strength- based, appropriate treatment goals 10© 2014 Community Care Behavioral Health Organization

Children Served by CSBBH 2009-present – 1,801 students – Majority are boys (71.4%) – 15.7% are Hispanic – 69.6% white, 14.2% black or African American, 0.2% Asian, and 15.6% other – Ages range from 4 to 19 years old, with a mean age of 9.5 years 11© 2014 Community Care Behavioral Health Organization

SDQ The Strengths and Difficulties Questionnaire (SDQ) measures caregiver, teacher, and youth report of child behavior (Goodman 1997) The SDQ contains four sub-scales for difficulties: 1) emotional symptoms, 2) hyperactivity, 3) peer problems, and 4) conduct problems A summation of the four difficulties scales is made to compute Total Difficulties Score One strength-based sub-scale – pro-social behaviors The SDQ is completed every three months; the SDQY is completed by youth ages 14 and older 12© 2014 Community Care Behavioral Health Organization

SDQ-Parent 13© 2014 Community Care Behavioral Health Organization For SDQ information, 4 subscales are totaled for a Total Difficulties score. This score is graphed against an indicator for the 10 th Percentile score (green line). Children with Total Difficulties scores 25 or higher are reporting substantially higher problem behaviors than other children

SDQ 14 There was significant improvement in parent (p<.0001) and teacher (p<.0001) reported total difficulties scores over time Parents have significantly higher (p<0.05) average ratings of difficulties and pro-social behaviors compared to teachers’ ratings © 2014 Community Care Behavioral Health Organization

Child Outcomes Survey Family functioning – Shared decision making, supporting each other Child functioning – Success in getting along with family, friends, doing well at school, completing household tasks – Overall wellness Caregiver perception of therapeutic relationship – Feeling respected, working on important goals, well-suited approach, caregiver confidence 15© 2014 Community Care Behavioral Health Organization

Is the COS being used? How much is the Child Outcomes Survey (COS) being discussed in sessions? – Asked parents on a 1-10 scale how much the Child Outcomes Survey results were discussed with clinician? – Examined what factors are associated with use of the COS results in sessions 16© 2014 Community Care Behavioral Health Organization

Discussion of COS 17 Most discussion with families of children who are doing better in treatment Least discussion with families of children who are doing least well in treatment © 2014 Community Care Behavioral Health Organization

COS: Child and Family Functioning There was a significant increase in family functioning over time (p<.0001) There was a significant increase in child functioning over time (p<.0001) 18© 2014 Community Care Behavioral Health Organization

COS: Therapeutic Relationship There was an overall significant improvement in therapeutic relationship over time (p=0.002) 19© 2014 Community Care Behavioral Health Organization

Monitoring Care: BH Service Utilization 20© 2014 Community Care Behavioral Health Organization This chart represents the service utilization for 726 children who used mental health services Pre-CSBBH (red bars) and During (blue bars) and the 475 children who did not use any services Pre-CSBBH (green bars)

Monitoring Care: Pharmacy Data 21© 2014 Community Care Behavioral Health Organization Diagnoses Medications

Feedback from School Personnel SubscaleMeanSDMin.Max. Competency Impact Satisfaction Collaboration © 2014 Community Care Behavioral Health Organization Average ratings on scale of 1-10

Standardized Assessment of Academic Performance 23 Math # Scored in Math % Advanced in Math % Proficient in Math % Basic in Math % Below Basic in Math CCH Students All Students Reading # Scored in Reading % Advanced Reading % Proficient Reading % Basic Reading % Below Basic Reading CCH Students All Students © 2014 Community Care Behavioral Health Organization

Use of Outcomes is Promising Clinicians appear to be able to administer outcome measures and use information in therapy sessions Discussion of information from COS associated with better therapeutic relationships and improves child and family functioning Standardized assessments aid in understanding functioning and performance Continued efforts to facilitate sustainable inclusion of outcomes and quality improvement 24© 2014 Community Care Behavioral Health Organization

Scranton, PA History and Time Line District and Community Leadership Team – established in school year Transformation of mental health services for children/youth and families Utilized PBIS Implementer’s Blueprint and Stages of Implementation (Fixen) Began in two schools, after reviewing data indicating positive outcomes, then expansion

Scranton, PA “Current Conditions” Eleven Elementary Schools (K-5) – 2 implementing ISF at all three tiers – 4 implementing PBIS at tier one and have SMH – 4 implementing PBIS at tier one – 1 will be trained/kick off PBIS this year Three Intermediate Schools (grades 6-8) – 3 implementing PBIS at tier one and has SMH – 3 beginning tier two Two High Schools (grades 9-12) – 1 with SMH and previously implementing PBIS – 1 implementing PBIS at tier one and has SMH

School Year 22 licensed mental health professionals 44 bachelor’s level behavioral health workers Closure of center-based partial hospitalization program Closure of five school-based partial hospitalization programs ROI – less restrictive educational placements, return to home schools, less restrictive mental health placements, cost savings Increased collaboration, communication – improved outcomes

Child Outcomes Survey (COS) Family Functioning:

Child Outcomes Survey (COS) Child Functioning:

Child Outcomes Survey (COS) Therapeutic Alliance:

Strengths and Difficulties Questionnaire Parent (SDQ-P) and Teacher (SDQ-T) Total Difficulties:

ISF Comparison Significant improvement over time for child and family functioning For PBIS schools, higher report of family functioning than non PBIS schools Therapeutic Alliance ratings were higher in the PBIS schools. 32© 2014 Community Care Behavioral Health Organization

Family Functioning 33© 2014 Community Care Behavioral Health Organization

Child Functioning 34© 2014 Community Care Behavioral Health Organization

Therapeutic Alliance 35© 2014 Community Care Behavioral Health Organization

Contact Kelly Perales | 36© 2014 Community Care Behavioral Health Organization