1. Agency for Healthcare Research and Quality Advancing Excellence in Health Care www.ahrq.gov Developing CAHPS® for People with Mobility Impairments June 8, 2005 Marybeth Farquhar Sue Palsbo

2. Advancing Excellence in Health Care Presentation Overview Purpose and History Purpose and History – Structure (core questions and supplements) – How it is used – Why we decided to create a CAHPS® for people with mobility impairments Steps to create a CAHPS® survey Steps to create a CAHPS® survey Progress to date on CAHPS-PWMI Progress to date on CAHPS-PWMI Example: How CAHPS® helped a physician group practice improve quality Example: How CAHPS® helped a physician group practice improve quality Questions and Answers Questions and Answers

3. Advancing Excellence in Health Care CAHPS® Overall Goal To provide carefully tested, standardized survey questionnaires and reports To provide carefully tested, standardized survey questionnaires and reports Used to report meaningful, reliable information from patients rating their health care experiences. Used to report meaningful, reliable information from patients rating their health care experiences.

4. Advancing Excellence in Health Care CAHPS® Principles Reports and ratings of experiences -- not just satisfaction Reports and ratings of experiences -- not just satisfaction Standardization to enable valid comparisons Standardization to enable valid comparisons Evidence basis for design, protocols, and language Evidence basis for design, protocols, and language All CAHPS® products and services in public domain All CAHPS® products and services in public domain

5. Advancing Excellence in Health Care How Survey Reports are Used Consumers – to choose a health plan Consumers – to choose a health plan Purchasers – to choose health plans to offer to employees or beneficiaries Purchasers – to choose health plans to offer to employees or beneficiaries Health insurers – to improve administrative operations Health insurers – to improve administrative operations Facilities and HMOs – to improve health care operations Facilities and HMOs – to improve health care operations

6. Advancing Excellence in Health Care Example: Maryland Medicaid Report Card (2004)

7. Advancing Excellence in Health Care How Maryland State Collected the Information

8. Advancing Excellence in Health Care A Growing Family of Surveys Ambulatory Level Health Plans Health Plans Group Practices Group Practices Behavioral Health Organizations (ECHO) Behavioral Health Organizations (ECHO) Individual Clinicians* Individual Clinicians* Facility Level Hospitals Hospitals Dialysis Facilities* Dialysis Facilities* Nursing Homes* Nursing Homes* CAHPS® Family * In development

9. Advancing Excellence in Health Care Many Inputs Into CAHPS® Development Process Cognitive interviews Literature and existing instruments Call for measures Public and stakeholder input Focus groups Field testing CAHPS® Surveys

10. Advancing Excellence in Health Care CAHPS® Development Needs Assessment Needs Assessment Conduct Literature Review Conduct Literature Review Gather Stakeholder Input Gather Stakeholder Input Develop & test questionnaire Develop & test questionnaire Develop & test sampling protocol Develop & test sampling protocol Develop & test data collection protocol Develop & test data collection protocol Develop & test reports Develop & test reports Develop analysis program Develop analysis program Develop QI program Develop QI program Translate tools Translate tools

11. Advancing Excellence in Health Care CAHPS® Reporting & Maintenance 1. Promote use with potential sponsors 2. Create database 3. Maintain database 4. Evaluate tool use 5. Provide technical assistance 6. Gather user input 7. Continue research to refine tools 8. Maintain, update & refine tools 9. Disseminate research

12. Advancing Excellence in Health Care Rationale for CAHPS® PWMI Development Persons with physical disabilities have a “thinner margin of health.” Persons with physical disabilities have a “thinner margin of health.” Stakeholders need to reduce disparities in quality and access, so care is equal. Stakeholders need to reduce disparities in quality and access, so care is equal. More persons with mobility impairments are being offered a choice of health plans. More persons with mobility impairments are being offered a choice of health plans.

13. Advancing Excellence in Health Care CAHPS®- PWMI Consortium: A Public-Private Research Team CAHPS II Consortium Grantee Teams AHRQ Westat AIR Harvard RAND CDC NIDRR Federal Agencies National Rehabilitation Hospital; GMU Center for Health Policy, Research & Ethics

14. Advancing Excellence in Health Care Purpose & Goals To develop screening questions that identify adults with mobility impairments To develop screening questions that identify adults with mobility impairments To develop, pilot test, and evaluate the psychometric properties of questions that ask about the quality of their health care experiences. To develop, pilot test, and evaluate the psychometric properties of questions that ask about the quality of their health care experiences.

15. Advancing Excellence in Health Care Progress to Date Developed an 11-item screener to identify respondents with mobility impairments Developed an 11-item screener to identify respondents with mobility impairments Cognitive testing and revisions Cognitive testing and revisions Field tested as part of the Massachusetts Health Medicaid Study and DRRP on Medicaid Quality Indicators for People with Disabilities Field tested as part of the Massachusetts Health Medicaid Study and DRRP on Medicaid Quality Indicators for People with Disabilities Analysis of the field test results (in progress) Analysis of the field test results (in progress) Literature review (in progress) Literature review (in progress) Call for Measures (FRN and this Web cast!) Call for Measures (FRN and this Web cast!) TEP TEP

16. Advancing Excellence in Health Care Planned Tasks for Next 6 Months Work with the TEP Work with the TEP Identify content domains Identify content domains Develop a draft survey Develop a draft survey Cognitive testing of questions and sequence Cognitive testing of questions and sequence Finalize tool for field testing in 2006 Finalize tool for field testing in 2006

17. Advancing Excellence in Health Care Ambulatory Surveys Cover Seven Domains Doctor Communication Coordination/ Integration Access Office Staff Courtesy, Helpfulness and Respect Shared Decision- Making Health Promotion and Education Customer Service

18. Advancing Excellence in Health Care Criteria for Selecting Domains Important for reaching, maintaining a high level of quality Important for reaching, maintaining a high level of quality Important to consumers in selecting health care Important to consumers in selecting health care Aspects of care for which consumers are best or only judge Aspects of care for which consumers are best or only judge

19. Advancing Excellence in Health Care From Domains to Topics to Items: An Example Domain: Access Topics Getting needed care After-hours care Utilization Getting care quickly Wait time Special services Prescriptions Items Problems seeing a specialist Problems getting care, tests, or treatment Etc.

20. Advancing Excellence in Health Care How Physicians Might Use a CAHPS®-PWMI Survey

21. Advancing Excellence in Health Care Limitations of the Health Plan Survey Plan-level data collection too imprecise for practice level improvements: members vs. patients, large samples Plan-level data collection too imprecise for practice level improvements: members vs. patients, large samples Surveys need to focus on the processes that are most relevant to consumers/patients at the plan level and the practice level, e.g. customer service for a plan, office staff interactions and the clinical experience for a practice. Surveys need to focus on the processes that are most relevant to consumers/patients at the plan level and the practice level, e.g. customer service for a plan, office staff interactions and the clinical experience for a practice.

22. Advancing Excellence in Health Care Why a Survey for Clinicians & Groups? It is the right unit of analysis for QI: It is the right unit of analysis for QI: – Consumers have more choice of practices than they do of plans – Data can be actionable for an individual practice or site – Plans/payers can identify benchmark performers more precisely.

23. Advancing Excellence in Health Care Why a Survey for Clinicians & Groups? Data collection at the practice/ provider level survey helps support multiple uses of the same data: Data collection at the practice/ provider level survey helps support multiple uses of the same data: – Maintenance of certification for doctors, ABMS activities – External use by plans/ payers – CMS: Doctor Office Quality project

24. Advancing Excellence in Health Care CAHPS® Clinician & Group Survey: Content Areas  Doctor Communication  Shared Decision- Making  Health Promotion and Education  Coordination/ Integration  Other Clinicians in the Practice (“Care Team”)  Access  Office Staff + Visit Module

25. Advancing Excellence in Health Care Sample Size Requirements for Varying Physician-Level Reliability Thresholds

26. Advancing Excellence in Health Care Stillwater Medical Group (SMG): Baseline Survey Results (n=164) Correlation to Measure of Willingness to Recommend Percentile Rank Adjusted Priority Improvements Communication & Interpersonal Care Access: Getting Needed Care Office Functioning: Scheduling & Visit Flow Preventive Care Integration 1/9/04

27. Advancing Excellence in Health Care Stillwater Medical Group: Results of the Intervention Priority aim: Priority aim: – Improve doctor communication scores Intervention: Intervention: – 3-day doctor training course facilitated by American Academy on Physician and Patient (AAPP) – 56 of 58 doctors attended in May 04 Results: Results: – Notable short-term improvement in question scores related to doctor communication – Mixed results in question scores over the longer term – Verbal support for “Patient-centered” care and need for improvement to help sustain and increase our business

28. Advancing Excellence in Health Care Stillwater Medical Group: What They Learned Everyone wants to provide good service, but they need to constantly work on improving. Everyone wants to provide good service, but they need to constantly work on improving. Their patients really like them and once they start to come here they keep coming back. Their patients really like them and once they start to come here they keep coming back. Data is a powerful motivator, but it needs to be individualized to really impact providers Data is a powerful motivator, but it needs to be individualized to really impact providers They raised awareness about this issue. They raised awareness about this issue.

29. Advancing Excellence in Health Care Harvard Vanguard’s Interventions to Improve Patients’ Experiences Improve visit-based continuity by increasing the percentage of the time that patients see their own primary care physician when they come to the office for care. Improve visit-based continuity by increasing the percentage of the time that patients see their own primary care physician when they come to the office for care. Improve visibility of clinical team by carefully structuring practices into teams with staff known to the patients. Improve visibility of clinical team by carefully structuring practices into teams with staff known to the patients.

30. Advancing Excellence in Health Care HVMA: Cost of Improvement Offset by Benefits Costs Costs – Survey costs – Analysis costs – Investment in practice change Benefits Benefits – Better retention of patients – Better retention of more satisfied staff – Practice growth

31. Advancing Excellence in Health Care Harvard Vanguard Medical Associates: What They Learned They learned a lot about what their patients thought of their experience with them. They learned a lot about what their patients thought of their experience with them. They learned that their staff was also worried about the practice, and frustrated that their patients weren’t more satisfied. They learned that their staff was also worried about the practice, and frustrated that their patients weren’t more satisfied. They learned how hard it is to change practice infrastructure and culture. They learned how hard it is to change practice infrastructure and culture.

32. Advancing Excellence in Health Care Questions & Answers!