1. Evaluation of HCHD community behavior health program 2005 - 2006 Evaluators: - Charles Begley - Scott Hickey - Britta Ostermeyer - Ann Teske - Thien Vu - Julia Wolf - Mark Kunik

2. Outline Background Objectives Evaluation framework and methods Results Discussion and recommendation

3. Background HCHD: - serves around 300,000 individuals/year, most are low income and/or underinsured - Operates BTGH, Lyndon Johnson GH, Quentin Mease Community Hospital, 11 CHCs, 7 school-based clinics, a healthcare program for the homeless, a center for HIV/AIDS (Thomas st.) and a dental center CBHP: - First launched by HCHD in 3 CHCs - Officially created in July 2005, - CBHP team includes psychiatrists, psychotherapists, counselors, residents and med students in 11 HCHD CHCs, 5 partner centers, 2 school-based clinics

4. Background Why CBHP? - Estimates: 20,000 children and 84,000 adults in Harris County needed mental health services in 2004 - There was only one District’s outpatient clinic at BTGH at the time - Only 8,800 adults and 1,700 children were served - Average appointment time: 6 months CBHP objectives: 1. to redirect BH patients to community clinics (integrated care) 2. to provide specific BH services 3. to develop & provide educational services for PCPs 4. to consult and coordinate with primary care providers

5. Evaluation framework

6. Evaluation objectives Process evaluation to describe major resources and features of the program to describe services provided and patients served Preliminary impact evaluation to evaluate initial impact on access, BH outcomes, provider satisfaction and costs

7. Methods VariablesMeasurement method Data sources Program resources and features: - Number and type of CBHP staff - Resources used - simple calculation - aggregated - Project documents - Monitoring report Amount and type of services provided & characteristics of patient served: - # of patient seen by provider & patient types - demographic characteristics - types of counseling sessions by individual/group/family/total - aggregated (using patient-coded medical records) - CBHP integrated database Initial impacts of the program: - Provider satisfaction - Accessibility - Health outcomes - Provider survey - Pre-post analyses - BASIS-24 analysis - 100 questionnaires PCPs, psychiatrists and BH therapists - HCHD database - patient’s BASIS-24 assessments

8. Program resources and features Results Program resources and features Originally proposed: - hire 1 Prog. Director, 4 licensed Social workers & 1 psychiatrist - Involve 4 HCHD’s CHC and 3 private community clinics Actually implemented - As proposed - Additional: 1 project coordinator, 6 social workers and 10 part time psychiatrists - Involve all 11 HCHD’s CHC, 4 private community clinics - Additional education sessions: teleconference lectures, psychotherapy referrals, DVD and audio tape lectures

9. 2,895 patients seen - 2075 females and 820 males - 801 by psychiatrists, 1,824 by BH therapists - 2,363 MH patients, 6 substance abuse, 336 both - 34 referred by project Insight, 12 referred by Council on Alcohol & Drug Houston - 75 65 - 835 African American, 58 Asian, 754 Caucasian, 1225 Hispanic; 3 American Indian; 17 others - 55 Katrina and 10 Rita victims 7,392 counseling sessions: - 1,696 psychiatry; 3,342 individual counseling, 562 group counseling, 830 families and 95 phone sessions Services provided & patient characteristics Results Services provided & patient characteristics Timeframe: July 2005 – May 2006

10. Results Initial impact on patient’s health BASIS-24: - 24 questions - domains are psychiatric and substance abuse functioning: depression, relationship, self- harm, emotional ability, psychosis and substance abuse - administered at least twice in the study period with the follow-up assessment at least 30 days after the first - responses scored using weighted average algorithm that give overall score for each assessment Results: - Significant improvement detected in overall score (p <.000) and 4 out of 6 domain scores: depression, self-harm; emotional lability and substance abuse (p<.000) - Average improvement percent change: 26% in overall score, 30% in depression, 75% in self- harm, 37% in emotional lability and 72% in substance abuse scores

11. Results Provider’s satisfaction BHS + PsychiatristPCPs VariablesObsMeanObsMean Accessibility184.58274.18 General quality improvement of PHC184.68274.53 Common understanding about CBHP184.69274.10 Time flexibility183.25273.78 Interaction between PCPs and BH184.00243.92 PCP education173.34 General satisfaction scores by area

12. Provider satisfaction – service accessibility Results Provider satisfaction – service accessibility Variable Combined scoreBH + PsyPCPs Poor accessibility prior to CBHP4.424.564.33 Improved accessibility by CBHP4.474.784.26 CBHP reduced ER visit4.404.614.26 CBHP reduced length of time for accessing BH services4.164.443.96 CBHP helps improve access care in appropriate time frame4.244.504.07 Satisfaction scores on accessibility by provider type

13. Provider satisfaction - quality of care Results Provider satisfaction - quality of care VariableBHS + PsyPCPs BHC necessary for PHC4.944.70 BHC improve adherence to treatment4.894.67 BHC offered by CBHP improve general quality of PHC4.724.44 CBHP enhances PCPs’ ability to provide BHC4.444.52 BHC enhances quality of the clinics4.394.30 Mean scores on quality of care variables by provider type

14. Provider satisfaction – time flexibility and staffing Results Provider satisfaction – time flexibility and staffing Mean scores on time flexibility reported by providers VariableMeanMinMax Time flexibility of BH physicians3.2515 Time flexibility of PCPs3.7825

15. Provider satisfaction – interaction between PCP & BH physicians Results Provider satisfaction – interaction between PCP & BH physicians Variable BH physiciansPCP Effective referral between PCPs and BH Therapists3.944.26 Comfortable with referring patients to PCPs/BH Therapists4.004.16 Awareness of CBHP protocol, roles and functions of members4.063.54 Mean scores on interaction between PCPs and BH physicians

16. Provider satisfaction – Educational activities and materials for PCPs Results Provider satisfaction – Educational activities and materials for PCPs Purpose: to improve PCP’s capacity to provide BH at CHC through on-going trainings and on-site consultation Materials: teleconference lectures, DVD + audio tapes Result: negative 52% of PCPs have not received any educational materials mean score on effectiveness of PCP education session was 3.34, lower than cut point

17. Provider satisfaction – Common vision, understanding and overall satisfaction Results Provider satisfaction – Common vision, understanding and overall satisfaction VariableCombinedBH physiciansPCP Share sense of responsibility4.404.784.15 Common treatment goals4.404.674.22 Understanding of roles and responsibility4.334.674.11 Formal and informal interaction4.204.723.85 Share knowledge4.384.784.11 Common vision/philosophy of CBHP4.19 Total (combined score)4.314.504.10 Mean scores on common vision, understanding by provider type

18. Service patterns Results Service patterns

21. Conclusions & recommendations CBHP has achieved many of its implementation objectives Impacts of CBHP on patients’ health, accessibility to BH services were positive Patient flow for BH was initially re-directed to lower cost and more convenient settings BH providers’ working schedules should be more flexible Interaction between PCPs and BH providers should be further facilitated for smoother operation of CBHP More BH staff is needed More rigorous evaluation plan should be developed Educational efforts should be improved