Evaluation of HCHD community behavior health program Evaluators: - Charles Begley - Scott Hickey - Britta Ostermeyer - Ann Teske - Thien Vu - Julia Wolf - Mark Kunik

Outline Background Objectives Evaluation framework and methods Results Discussion and recommendation

Background HCHD: - serves around 300,000 individuals/year, most are low income and/or underinsured - Operates BTGH, Lyndon Johnson GH, Quentin Mease Community Hospital, 11 CHCs, 7 school-based clinics, a healthcare program for the homeless, a center for HIV/AIDS (Thomas st.) and a dental center CBHP: - First launched by HCHD in 3 CHCs - Officially created in July 2005, - CBHP team includes psychiatrists, psychotherapists, counselors, residents and med students in 11 HCHD CHCs, 5 partner centers, 2 school-based clinics

Background Why CBHP? - Estimates: 20,000 children and 84,000 adults in Harris County needed mental health services in There was only one District’s outpatient clinic at BTGH at the time - Only 8,800 adults and 1,700 children were served - Average appointment time: 6 months CBHP objectives: 1. to redirect BH patients to community clinics (integrated care) 2. to provide specific BH services 3. to develop & provide educational services for PCPs 4. to consult and coordinate with primary care providers

Evaluation framework

Evaluation objectives Process evaluation to describe major resources and features of the program to describe services provided and patients served Preliminary impact evaluation to evaluate initial impact on access, BH outcomes, provider satisfaction and costs

Methods VariablesMeasurement method Data sources Program resources and features: - Number and type of CBHP staff - Resources used - simple calculation - aggregated - Project documents - Monitoring report Amount and type of services provided & characteristics of patient served: - # of patient seen by provider & patient types - demographic characteristics - types of counseling sessions by individual/group/family/total - aggregated (using patient-coded medical records) - CBHP integrated database Initial impacts of the program: - Provider satisfaction - Accessibility - Health outcomes - Provider survey - Pre-post analyses - BASIS-24 analysis questionnaires PCPs, psychiatrists and BH therapists - HCHD database - patient’s BASIS-24 assessments

Program resources and features Results Program resources and features Originally proposed: - hire 1 Prog. Director, 4 licensed Social workers & 1 psychiatrist - Involve 4 HCHD’s CHC and 3 private community clinics Actually implemented - As proposed - Additional: 1 project coordinator, 6 social workers and 10 part time psychiatrists - Involve all 11 HCHD’s CHC, 4 private community clinics - Additional education sessions: teleconference lectures, psychotherapy referrals, DVD and audio tape lectures

2,895 patients seen females and 820 males by psychiatrists, 1,824 by BH therapists - 2,363 MH patients, 6 substance abuse, 336 both - 34 referred by project Insight, 12 referred by Council on Alcohol & Drug Houston African American, 58 Asian, 754 Caucasian, 1225 Hispanic; 3 American Indian; 17 others - 55 Katrina and 10 Rita victims 7,392 counseling sessions: - 1,696 psychiatry; 3,342 individual counseling, 562 group counseling, 830 families and 95 phone sessions Services provided & patient characteristics Results Services provided & patient characteristics Timeframe: July 2005 – May 2006

Results Initial impact on patient’s health BASIS-24: - 24 questions - domains are psychiatric and substance abuse functioning: depression, relationship, self- harm, emotional ability, psychosis and substance abuse - administered at least twice in the study period with the follow-up assessment at least 30 days after the first - responses scored using weighted average algorithm that give overall score for each assessment Results: - Significant improvement detected in overall score (p <.000) and 4 out of 6 domain scores: depression, self-harm; emotional lability and substance abuse (p<.000) - Average improvement percent change: 26% in overall score, 30% in depression, 75% in self- harm, 37% in emotional lability and 72% in substance abuse scores

Results Provider’s satisfaction BHS + PsychiatristPCPs VariablesObsMeanObsMean Accessibility General quality improvement of PHC Common understanding about CBHP Time flexibility Interaction between PCPs and BH PCP education General satisfaction scores by area

Provider satisfaction – service accessibility Results Provider satisfaction – service accessibility Variable Combined scoreBH + PsyPCPs Poor accessibility prior to CBHP Improved accessibility by CBHP CBHP reduced ER visit CBHP reduced length of time for accessing BH services CBHP helps improve access care in appropriate time frame Satisfaction scores on accessibility by provider type

Provider satisfaction - quality of care Results Provider satisfaction - quality of care VariableBHS + PsyPCPs BHC necessary for PHC BHC improve adherence to treatment BHC offered by CBHP improve general quality of PHC CBHP enhances PCPs’ ability to provide BHC BHC enhances quality of the clinics Mean scores on quality of care variables by provider type

Provider satisfaction – time flexibility and staffing Results Provider satisfaction – time flexibility and staffing Mean scores on time flexibility reported by providers VariableMeanMinMax Time flexibility of BH physicians Time flexibility of PCPs3.7825

Provider satisfaction – interaction between PCP & BH physicians Results Provider satisfaction – interaction between PCP & BH physicians Variable BH physiciansPCP Effective referral between PCPs and BH Therapists Comfortable with referring patients to PCPs/BH Therapists Awareness of CBHP protocol, roles and functions of members Mean scores on interaction between PCPs and BH physicians

Provider satisfaction – Educational activities and materials for PCPs Results Provider satisfaction – Educational activities and materials for PCPs Purpose: to improve PCP’s capacity to provide BH at CHC through on-going trainings and on-site consultation Materials: teleconference lectures, DVD + audio tapes Result: negative 52% of PCPs have not received any educational materials mean score on effectiveness of PCP education session was 3.34, lower than cut point

Provider satisfaction – Common vision, understanding and overall satisfaction Results Provider satisfaction – Common vision, understanding and overall satisfaction VariableCombinedBH physiciansPCP Share sense of responsibility Common treatment goals Understanding of roles and responsibility Formal and informal interaction Share knowledge Common vision/philosophy of CBHP4.19 Total (combined score) Mean scores on common vision, understanding by provider type

Service patterns Results Service patterns

Conclusions & recommendations CBHP has achieved many of its implementation objectives Impacts of CBHP on patients’ health, accessibility to BH services were positive Patient flow for BH was initially re-directed to lower cost and more convenient settings BH providers’ working schedules should be more flexible Interaction between PCPs and BH providers should be further facilitated for smoother operation of CBHP More BH staff is needed More rigorous evaluation plan should be developed Educational efforts should be improved