1. Quality of care from the patient’s perspective Background and introduction of the CAHPS® questionnaires into the Dutch health and social insurance system Diana Delnoij (NIVEL) Herman Sixma (NIVEL)

2. Structure of the presentation Introduction + developments in society Developments in QoC research; perspectives + ‘state-of-the-art’ Past, present and future of the CAHPS- approach Concluding remarks + discussion

3. Developments macro level technological developments rising expectations + expenditures in the health care sector health care reforms tendency toward democratization

4. Developments meso level professionalization of health care services rise of organizational structures burocratization of services development of category-specific patient organizations

5. Developments micro level changes in doctor-patient relationship consequences ‘ageing society’ changes the family structures the ‘professional’ patient

6. Developments in society: a summary Macro level : - from ‘leadership’ to ‘stewardship’ Meso level : - from ‘supply’ to ‘demand centered’ Micro level : - from ‘patient’ to ‘active participant’

7. insurer consumer provider insurance market purchasing market provider market The new health care ‘market’

8. Part 2 : Quality of care (QoC) from the patient’s perspective; history and developments

9. “ The key to successful doctor-patient partnerships is recognize that patients are experts too ….. The new emphasis is on shared information, shared evaluation, shared decision making, and shared responsibilities..…” (Angela Coulter, BMJ 1999; 319: 719-720)

10. Different QoC perspectives health insurers > cost efficiency providers > guidelines + protocols patients > satisfaction + QoC ratings government > legal framework

11. Four generations QoC research P atient satisfaction studies (see Linder- Pelz, 1982 a.o.) Service Quality models (Parasuraman et al, 1985, 1988) Extended SERVQUAL models (CAHPS, QUOTE, Picker Instruments a.o.) Future: Performance indicator models

12. Main characteristics ‘state-of- the-art’ QoC questionnaires patients involvement qualitative + quantitative methods shift from ‘satisfaction’ to ‘reports’ QoC = multi-dimensional concept applicable in QA and QI studies

13. “ Questions asking for reports tend to reflect better the quality of care and ar more interpretable and actionable for quality improvement purposes than ratings of satisfaction or excellence..…” (Paul Cleary & Susan Edgman-Levitan, JAMA 1997, 278 (19): 1608-12)

14. Examples of some ‘state-of- the-art’ QoC questionnaires Responsiveness measures WHO Instruments Picker-Europe (UK) Family of QUOTE instruments (NL) CAHPS Instruments (USA)

15. What is CAHPS? Consumer Assessment of Health Plan Survey Questionnaires containing items on patients’: experiences with health care providers experiences with their insurance company general rating of health care and health plan Commissioned by: Agency for Healthcare Research and Quality (AHQR). Developed by: Harvard Medical School, the RAND corporation, and the American Institutes for Research

16. Introduction of CAHPS in the Netherlands Initiative: Agis (regional public health insurer) Translation and validation of three questionnaires: 1.CAHPS Adult Commercial Questionnaire 2.Hospital CAHPS 3.Diabetes questionnaire

17. CAHPS instruments (example 1) Please answer the questions in this survey about this stay at [FACILITYNAME]. Do not include any other hospital stay in your answers. YOUR CARE FROM NURSES 1.During this hospital stay, how often did nurses treat you with courtesy and respect? 1  Never 2  Sometimes 3  Usually 4  Always 19.Before giving you the medicine, did hospital staff describe possible side effects in a way you could understand? 1  Yes 2  No

18. CAHPS instruments (example 2) 5. Using any number from 0 to 10 where 0 is the worst possible care and 10 is the best possible care, what number would you give the care you got from all the nurses who treated you? 0  0 Worst possible nursing care 1  1 2  2 3  3 4  4 5  5 6  6 7  7 8  8 9  9 10  10 Best possible nursing care

19. Translating the CAHPS Adult Commercial Questionnaire Translation: forward - backward by 2 x 2 translators Tailored to the Dutch health care system: Absence of nurse practitioners or physician assistants distinction between ER in hospital and out-of- hours services of GPs Adaptation to social-cultural values in NL: Question about educational system Use of ethnicity instead of race

20. Fielding the CAHPS Adult Commercial Questionnaire Representative sample of 1,000 Agis clients who were insured 6 months prior to sample selection In November 2003, 977 insured received a mailed questionnaire; 545 responded (56%) Respondents are significantly older than non-respondents Respondents are also older than the general Dutch population and in worse health

21. Validating the CAHPS Adult Commercial Questionnaire 1.Check consistency of the data  o.k. 2.Check frequencies and missing values  o.k. 3.Check factor structure and reliability  o.k. 4.Compare results with other Dutch studies and with the American National CAHPS Benchmarking Database 2003:  This presentation

22. Comparison of Dutch data (NL) with American benchmark data (USA): access to care

23. Comparison of Dutch data (NL) with American benchmark data (USA): patient-centered care

24. Comparison of Dutch data (NL) with American benchmark data (USA): general rating

25. Translation into consumer information: % consumers who always experience that:

26. Conclusion Dutch respondents are able to fill out the questionnaire Results are internally consistent and reliable Patients’ evaluation of the Dutch and American primary process is strikingly comparable More research is needed on the external validity in the Dutch context

27. Major uncertainties Will insurers really engage in strategic purchasing? If so, is CAHPS-information useful for them? Will consumers choose rationally between different insurers? If so, is CAHPS-information useful for them?

28. Looking at the future: results of a national study on quality of home care from the clients perspective;

29. MDB – model (home care NL)

30. Conceptual model for client survey Communication Artificial aids Professional competence Continuity Accessibility Courtesy Process Structure Quality of home care Autonomy

31. Data collection QuestionnairesNumber% Gross random survey56,094100.0 - could not be delivered2,6754.8 - do not receive home care5000.9 Net random survey52,919100.0 Gross nr. of respondents25,97349.1 - declined to participate9201.7 - nothing filled in4940.9 Net nr. of respondents24,57946.5

32. Process quality on 1 -10 scale

33. Structure quality, 1 – 10 scale

34. QoC ratings 106 home care organizations

35. Overall QoC scores, broken down by size of organization

36. Main conclusions (1) Patient views on QoC should be an important topic for policy makers + health care providers + health insurers + patients themselves Standardization and internationalization of QoC measuring instruments, based on the CAHPS approach offers new possibilities for QA, QI and to derive information to facilitate patient choice

37. Main conclusions (2) Follow-up research on validity, reliability, ‘sensitivity to change’ and discriminating power of new measuring instruments (such as the CAHPS-NL instruments) is still necessary Mesuring QoC from the patients’ perspective should be a part of an integrated approach (TQM, MDB, BSC)

38. Contact details Herman J. Sixma, PhD NIVEL P.O. Box 1568 3500 BN Utrecht The Netherlands Tel.: ++ 31 30 2 729 710 Fax.: ++ 31 30 2 729 729 E-mail: h.sixma@nivel.nl www.nivel.nl