Performance Outcomes in EAP National Behavioral Consortium September 6, 2007 Loews Coronado, San Diego Russ Hagen, Dave Sharar, Managing Director

Agenda Overview of Issues with Performance Outcomes in EAP (slides 3-13) Some example Chestnut Outcome Projects (15-28) Where do we go as a field? (29-30)

The Conceptual Dilemma – What is EAP? A “theory" of workplace intervention A "specialized field" or profession A "place" where other professions ply their trade A set of "common components" or a "practice model" A type of "funding mechanism" for counseling or mental health services “EAPs have enlarged their scope: Can we align metrics with relevant attributes of EAP definition?”

The Research & Empirical Base of EAP Citations of effectiveness are often obsolete/flawed (about 39 studies in past years) Existing studies are often unpublished or proprietary No agreed upon markers of success, or true comparisons across vendors, models, programs No identified academic discipline or plan to build researchers Little transparency or collaboration among vendors

Performance is in the Eye of the Beholder Purchaser: Are my EAP dollars being spent wisely? Provider: (primary vendor or affiliate): Does good service or results produce rewards? Client (employee or family member): Is the EAP responsive to my personal needs?

State of performance measurement in EAP Substandard performance is largely invisible Many measures are blunt, incomplete, distorting, exaggerated (e.g. Utilization rates) We lack common definitions & standard markers of success Under capitation, “marginal” performance receives the same rate as “optimal”

Why measure performance? Describe the effects or results of our interventions (outcome measures) Improve an aspect of the process of care (process measure), which in theory leads to better outcomes Make comparisons across vendors or program models Counterforce to deflation of EAP rates

Issues with Employers Many employers don’t know a good measure from a bad one Lack of senior management engagement Over-reliance on consultants and brokers who do not understand EAP Competing goals between finance, HR, benefits, occupational medicine, etc. Good performance measurement is difficult-and this difficulty is not always appreciated

Why are outcomes so difficult? Natural Problems Confounding factors beyond your control Sample size too small to produce effect Long delays when measuring over time Low frequency of interesting outcomes

Why are outcomes so difficult? Human Problems Inadequate information systems No extra funding (vendor bears cost) Accessing employer data Point of measurement complexity Insufficient level of clinical detail How does one address poor outcomes?

Criteria for selecting measures: Does the measure serve to enhance the productivity or well-being of employees? (e.g. is it “mission critical”? Is the measure based on science or opinion? (and if opinion, is there consensus)? Is it feasible? (resource availability, automated data collection, and statistically meaningful comparisons)

Practical Suggestions Implement new P4P pricing Align better outcomes with higher payments Educate employers to buy based on results, not price

Why P4P in EAP? Financial incentives in EAP (under capitation) are perverse, flat, almost non-existent Many contracts use “penalties” (withhold rather than bonus) So, delivering high quality EAP does not usually pay

"The larger issue is not whether EAPs are effective, but which EAPs are." Ken Collins. "EAP Cost/Benefit Analysis: The Last Word." in EAPA Exchange (2000, Nov/Dec, p. 31).

Started in 1985 and grew to 90 full/part time staff grossing $9 Million a year in external funds (NIH, SAMHSA, Foundations) LI-Research: Several major experiments, quasi-experiments and major surveys LI-Training and Publications: 100s of training days and largest collection of evidence-based treatment manuals EBTx Coordinating Center---Supports training, certification, and coaching of clinicians and clinical supervisors learning A-CRA and ACC GAIN Coordinating Center – supports training, certification and use of the GAIN to support diagnosis, placement, treatment planning, and research Chestnut strategy - link research capability with funded EAP outcome studies

Types of EAP Outcomes General Clinical Outcomes Work Productivity Ratings Relationships between clinical outcomes and workplace variables

Global Appraisal of Individual Needs (GAIN) Short Screener (SS) embedded into WebMD's Health Risk Assessment for ADM A scientifically valid, 3 minute behavioral health screener for use in general populations Identifies who has a disorder and who does not with 95% accuracy Approximates the type of problem and severity Guides further assessment & can be used as a measure of change Website:

ADM Pilot (n=1469) 69% 77% 83% 52% 25% 21% 16% 32% 7% 17% 1% 2% 0%20%40%60%80%100% Internal Disorder External Disorder Substance Use Disorder Total Disorder Screener LowModerateHigh Source: Collected as part of ADM health risk assessments from 11/05 to 8/06; Total Disorder Screener is based on 14 of 20 GSS items (one item in the internal not asked, and the violence and crime screener were not asked).

ADM Pilot (n=1469) Internal and Disorder Screener Items by Total Screener Score

76% 90% 35% 82% 31% 15% 3% 8% 14% 17% 9% 53% 4% 7% 56% 0%20%40%60%80%100% Internal Disorder External Disorder Substance Use Disorder Violence and Crime Total Disorder Screener LowModerateHigh Clinical Sample of Workers 12 Months After SA Treatment (n=115) Source: ERI (Dennis & Scott) Interviews conducted between 1/05 and 5/05

Costs of Service Utilization in the NEXT 12 Months by Total Disorder Screener (n=115) Source: ERI (Dennis & Scott) Interviews conducted between 1/05 and 5/05; p<.05 Each has a sharp right skew Higher Median Costs

Median Hours Absent by ADM Employees by Total Disorder Screener Score Based on those with any absenteeism

Next Steps with the GAIN SS 1.Link ADM GAIN SS scores to ADM medical & pharmacy claims, workers' compensation, and other measures to determine if we can predict future claims or expenses 2.Implement a process to proactively "reach out" to ADM members with moderate or high risk scores 3.Use the GAIN SS as a longitudinal follow-up tool to measure reductions in symptoms

ADM Formal Management Referrals in 2003

Workers' Compensation Study 2005  Examined financial effect of ADM’s EAP on workers’ compensation claim dollars  Pre and Post EAP  N= 217  3 years pre- and post-intervention  55% reduction in claims Alcohol, drugs & depression cases in 2002

Survey Data Percent reporting dissatisfaction with...

Survey Data Percent who have considered early repatriation

Survey Data “How has your personal life been affected while living abroad?” Percent reporting a “negative” impact. 39% decrease 49% decrease

Tension between price and performance cannot be resolved without measurement Obtain agreement on core set of performance measures All vendors/program should report on same measures Compete on who is best at addressing employee problems Where do we go from here?

GOOD TO GREAT CONCEPT BUSINESS SECTORSOCIAL SECTORS Defining and Measuring “Great” Widely agreed-upon financial metrics of performance. Money is both an input (a means to success) and an output (a measure of success). Fewer widely agreed-upon metrics of performance. Money is only an input, not an output. Performance relative to mission, not financial returns, is the primary definition to success. Summary difference between business and social sectors thought the good-to-great framework (Jim Collins)