1. RANDRAND CAHPS® Relevance of CAHPS® for Workers’ Compensation Medical Care Donna Farley Senior Health Policy Analyst, RAND Workers’ Compensation Colloquium May 1, 2003

2. 2 RANDRAND CAHPS® Goal of Presentation To explore how customer satisfaction for worker’s compensation medical care can draw upon experiences with the Consumer Assessment of Health Plans (CAHPS)®.

3. 3 RANDRAND CAHPS® CAHPS Is a Consumer Assessment Survey and Reporting System Purpose: to provide consumers information on health plan and provider performance as judged by other consumers who used them. Complete survey and reporting package »Thoroughly tested surveys »Sampling and survey administration protocols »Guidance for comparative reports of survey results Used across the country for various populations and health care settings

4. 4 RANDRAND CAHPS® The CAHPS Reporting Dimensions Global ratings (on a 0 to 10 scale) »Rating of primary doctor or nurse »Rating of health plan Reports of consumers’ experiences (composites) »Getting needed care (4 items) »Getting care without long waits (4 items) »How well doctors communicate (4 items) »Courtesy and respect by office staff (2 items) »Health plan customer service, information (3 items)

5. 5 RANDRAND CAHPS® Formats to Report CAHPS Scores Stars Plan 1** Plan 2** Plan 3** Plan 4*** Plan 5ns Dimension: Courtesy, respect, and helpfulness of medical office staff Bars 0% 100% Sometimes, never UsuallyAlways

6. 6 RANDRAND CAHPS® The First CAHPS Cycle: The Basic Surveys and Reports Designed and tested surveys and reports Tested implementation of the survey protocols and reporting formats in field demonstrations Modified the survey and report template based on lessons from the demonstrations Assessed the effects of CAHPS information on actual health plan choices made by consumers

7. 7 RANDRAND CAHPS® How We Tested CAHPS Products Laboratory experiments – simulated health plan choices by consumers »Commercial: CAHPS scores versus costs »Medicaid: CAHPS scores versus benefits Field demonstrations – CAHPS survey and report implemented by sponsors »Large private employers »Medicaid programs »States as employers

8. 8 RANDRAND CAHPS® Lessons About CAHPS Products from the Demonstrations The CAHPS survey »Covers the most important topics »The expertise behind survey was far more than sponsors could do on their own »Off-the-shelf questionnaires and protocols for sampling and data collection were valued »Concerns about high costs to field the survey Reporting of CAHPS scores »Opinions differed on whether and how to report »Consensus that report should be short and clear »Distribute CAHPS reports with other information

9. 9 RANDRAND CAHPS® What Effects Does CAHPS Have? Efficacy versus Effectiveness The effectiveness of an intervention depends on — »its efficacy in the treated population and »the proportion of the target population reached for treatment We tested CAHPS effects on consumer plan choices »Laboratory studies used to estimate efficacy »Field demonstrations used to estimate effectiveness under “real world” conditions.

10. 10 RANDRAND CAHPS® Efficacy Findings from the Laboratory Studies Participants more likely to choose health plans that performed better on CAHPS. Medicaid beneficiaries traded off transportation or dental benefits for plans with higher ratings. Commercial plan enrollees considered CAHPS ratings and costs separately in choices. Provided evidence of the efficacy of CAHPS information under controlled conditions.

11. 11 RANDRAND CAHPS® Effectiveness Findings from the Demonstrations CAHPS information influenced plan choices only under certain circumstances; effects were small. Effects limited to the subset of consumers who actively study the information they are provided. Even for these consumers, the information may make a difference only when »differences in ratings among plans are large, »ratings are discordant with previously held beliefs about plan quality, and »the reports are easy to read and understand.

12. 12 RANDRAND CAHPS® The Second CAHPS Cycle: Building Upon CAHPS 1 Products Strengthening evidence base for effective reports Developing tools for quality improvement activities to improve performance on CAHPS Modifying survey to cover three levels – health plan, provider group, individual provider (with results rolled up or down) Expanding survey and report to other settings and populations

13. 13 RANDRAND CAHPS® Consumer Satisfaction for Workers’ Compensation Medical Care How will information on consumer satisfaction (consumer assessment) be used for workers’ compensation medical care? »Feedback to workers’ compensation plans and providers on their performance »Information for survey sponsors to select and monitor plans and providers »Information to workers for choice of a workers’ compensation plan or provider »Comparisons of plans’ and providers’ performance to each other and other benchmarks

14. 14 RANDRAND CAHPS® Drawing Upon CAHPS for Collecting and Reporting Workers’ Assessments Determine what information is to be collected »Aspects of care important to consumers »Information needed by the survey sponsors »Information needed for quality improvement Develop and test items and survey as a whole Design sampling and data collection protocol Determine strategy for reporting survey results – what is reported, to whom, in what format Many aspects of the CAHPS contents and lessons on implementation are relevant to this setting