The measurement and comparison of health system responsiveness   The measurement and comparison of health system responsiveness Nigel Rice, Silvana Robone, Peter Smith (Centre for Health Economics, University of York) XX Riunione Scientifica Siep 25 -26 settembre 2008

Introduction (1)   Patients’ views and opinions are an essential means for assessing the provision of health services Traditionally, patients’ views were sought on the quality of care provided Recently the concept of responsiveness has been promoted as a more desirable measure to judge health systems. Responsiveness = the way in which individuals are treated and the environment in which they are treated, encompassing the notion of patient experience with the health care system (Valentine et al., 2003)

Introduction (2) ISSUE: data on Responsiveness are self-reported.   ISSUE: data on Responsiveness are self-reported. Ex: “For your [child’s] last visit, how would you rate the experience of being involved in making decisions about your health care or treatment?” Response categories: “Very good”, “Good”, “Moderate”, “Bad”, and “Very bad”. Individuals may systematically interpret the meaning of the available response categories differentially across population sub-groups (Sadana et al., 2002). REPORTING HETEROGENEITY: Responses are influenced by individuals' preferences and expectations. EX

Introduction (3)   Use of anchoring vignettes to address the issue of reporting heterogeneity. Vignettes = descriptions of fixed levels of a latent construct, such as respons. EX: “When the clinic is not busy, [Mamadou] can choose which doctor he sees. But most often it is busy and then he gets sent to whoever is free”. How would you rate [Mamadou’s] freedom to choose his health care provider? 1. Very good 2. Good 3. Moderate 4. Bad 5. Very bad Any systematic variation across individuals in the rating of the vignettes can be attributed to reporting heterogeneity OUR AIM: Evaluate the presence of reporting bias across socio-economic groups within countries, and how it is related to the characteristics of the individuals.

DATA (1) The World Health Survey   The World Health Survey Launched by the World Health Organisation (WHO) in 2001, 70 countries Survey modes: face to face interview and computer assisted telephone interviews Samples: randomly selected (+ 18 years), sizes between 600 and 10,000 The Responsiveness Module Sections: Needing Health Care and General Evaluation of the Health System, Seeing Health Care Providers, Outpatients and Care at Home, Inpatient Hospital, Vignettes Responsiveness Domains: Autonomy, Choice, Clarity of communication, Confidentiality, Dignity, Prompt attention, Quality of basic amenities, Social support Our sample: Mexico, India and Philippines, inpatient care

DATA (2) Dependent Variables Independent Variables   Dependent Variables 2 out of the 8 domains, Dignity and Clear Communication (considered as most important by the respondents in the countries selected) For each domain, 2 questions about respondents` experiences of contact with health systems. Response categories: “very good”, “good”, “moderate”, “bad” and “very bad”. Independent Variables Education: categorical variable (7 categories) or continuous variable (number of years in education). Gender: dummy variable, 1 if woman, 0 if man. Income: categorical variable (quintiles of the distribution of household permanent income, 1 if in the lowest income quintile, 5 if in the highest one). Permanent income measured with the HOPIT model (Ferguson et al., 2003). Age: continuous variable (years)

Econometric model The Hierarchical Ordered Probit Model (HOPIT)   The Hierarchical Ordered Probit Model (HOPIT) Terza (1985), Tandon et al. (2003) Two parts model: Reporting behaviour equation + Responsiveness equation Assumptions: a) Response consistency Individuals classify the vignettes in a way that is consistent with the rating of their own experiences of the service provided. b) Vignette equivalence The level of the variable represented by any one vignette is perceived by all respondents in the same way and on the same unidimensional scale (King et al., 2004, p.194).

Descrip. Stat.   Summary frequencies for the reporting of vignettes Mexico. .1 .2 .3 .4 .5 .6 % own vig1 vig2 vig3 vig4 vig5 i_clarity_communic mean of verygood mean of good mean of moderate mean of bad mean of verybad

Results (1) Tests of homogenous reporting and parallel cut-point shift   Tests of homogenous reporting and parallel cut-point shift (p-value for Wald Test under H0 ) Mexico Homo geneity Parallel cut - point shift All Inc. Women Age Educ. Dignity Greeted and talked to respectfully Privacy respected Communication Provides explained thing s Enough time for questions .000 . 000 .210 .012 .988 .851 .010 .083 .119 .005 .978 .738 .053 India Educ Provides explained things .002 .00 .209 .007 .314 .557 .024 .001 .009 .160 .030 .003 .183 .044 5 3 .362 .199 392 .389 .094 .036 189 .862 .710 101 Philippines .092 .619 .0 50 .716 .505 521 232 .011 .016 027 08 .250 .1 38 .004 .205 .714 .042 .560 .565 140 017 .069 .017 03 .159 .073

Results (2)   Estimated coefficients of permanent income in the reporting bias equation MEXICO: positive for first and second cut-point, negative fourth cut-point INDIA: mainly negative for all the cut points

Results (3)   Estimated coefficients of permanent income in the responsiveness equation OPROBIT For most items, positive and significant effect of income on responsiveness HOPIT Positive and significant effect of income on responsiveness The magnitude of the coefficients is smaller than in the OPROBIT (up to 1/2 for India, 1/3 for Mexico and Philippines) The influence of income on responsiveness is overestimated by the OPROBIT model

Conclusions   Preliminary results on the extent of reporting bias, using data from three countries contained within the WHS. Heterogeneity in reporting behaviour exists, it is a function of income and education, but does not appear to be strongly related to age and gender. Adjusting for reporting bias impacts on the estimated coefficients of the responsiveness equation when comparing the OPROBIT and HOPIT model Future extensions: compare heath system responsiveness across countries THANKS