The role of patient reported outcome (PRO) data in HTA: issues in measuring and valuing health Professor Nancy Devlin Director of Research OHE PSI conference, May 16th 2017

Content Patient reported outcomes (PROs) – what do they measure? Why are they important? Descriptive systems and profile data Analysing profile data 3. Patients’ assessments of overall health 4. Scoring and values/utilities Sources/properties of values; implications for statistical analysis of utilities 5. Differences between the 3L and 5L; implications for HTA And I will end with some concluding remarks

Much of what I will say today is drawn from these papers

1.PROs: what do they measure? The goal of most health care is to improve patients’ health Patients arguably the best judge of how they feel “The use of PRO instruments is part of a general movement toward the idea that the patient, properly queried, is the best source of information about how he or she feels” (FDA 2006) PROs: aim to capture patients’ subjective account of their own health in a structured way. Psychometric properties: validity; reliability PRO data a useful complement (not substitute) for clinical endpoints.

How is health described? Types of PROs How is health described? How is health summarised? Generic Condition-specific Scores Eg. SF-36 Eg. Oxford Hip Score, Oxford Knees Score (literally thousands of such PROs exist) Values/utilities Eg. EQ-5D HUI SF-6D AQOL 15D Mapping to preference based measures (various methods) Some condition specific instruments accompanied by value sets eg. EORTC-8D, derived from the EORTC QLQ-C30 for use in patients with cancer Notes: - generic and condition specific instruments have a different role: generic allows comparisons across disease areas (to weigh up benefit gained with benefit foregone elsewhere); condition-specific provide rich data on specific disease/health problem. Common to map from disease specific to preference based instrument (various methods; map to utilities or map to generic descriptive system) but preferable to collect generic data Best practise: collect both generic and condition specific.

Uses of PROs Clinical trials Observational studies Population health surveys Routine collection in health service delivery, eg. ‘PROMs’ in English NHS UK private health care sector (PHIN) Alberta Health Services, Canada New Zealand Southern Cross.

EQ-5D descriptive system and profile One of the most common generic, preference-based PROs. Available 5 level (shown here) or 3L (used since 1990s; recommended by NICE)

EQ-VAS: Patients’ overall assessment of their health

Utilities Single number summaries of how good or bad each health state is Anchored at 1= full health, 0 = dead, < 0 worse than dead (as required by QALYs) Based on stated preferences of general public considering hypothetical states With exceptions eg. Sweden’s TLV prefers patient’s values Both could be argued to be relevant: “health economic guidelines could require analysis of benefit in terms of QALYs based on both patient and general public preferences” (Versteegh & Brouwer 2017)

Utility/happiness Quality of life What EQ-5D and other PROs measure is nested within broader concepts of QoL and utility. Conceptually, the borders between HR-QoL and QoL and between QoL and utility are not precisely defined. Quality of life Health-related quality of life Health status EQ-5D profile captures specific aspects of HR-QoL EQ-VAS (a ‘feeling thermometer’) which captures the patients’ overall assessment of how good or bad their health is; Utilities (values) which capture the general public’s affective forecasts of utility in those states Increasing interest in broader concepts: social care outcomes; subjective wellbeing and ‘happiness’

Profiles: simple descriptive analysis Nature of profiles observed + value set characteristics drive observed distribution of utilities eg. in these 3L data, no level 3 observed on mobility ‘N3’ term on 3L utilities important characteristic of utility This sort of analysis aggregates data by dimensions/levels. It doesn’t show how dimensions/levels combine on a within-respondent basis

Pareto classification of health change Comparing a patient’s PRO profile at any 2 points in time: The same Better on at least 1 dimension, no worse on others = improvement Worse on at least 1 dimension, and no better on others = worsening Better on some dimensions, worse on others = mixed

What is the EQ-VAS measuring?

4. Scores and utilities Act to weight the levels/dimensions in profiles so they can be aggregated into a single number Preference based (values/utilities) or non- preference based (scores) Utilities: 1= full health, 0 = dead, < 0 worse than dead Neither are ‘neutral’ – there is no objective way of aggregating profile data Which value set is used introduces an exogenous source of variance, and can bias statistical inference. The paper where we show this point is: Parkin D, Devlin N, Rice N. (2010) Statistical analysis of EQ-5D profiles: does the use of value sets bias inference? Medical Decision Making 2010; 30:556-565

What causes the distribution of observed utilities? The distribution of utilities is caused both by the properties of the value set and by the properties of the profile data Profile data are often clustered, and highly ‘concentrated’ ie a surprisingly small number of profiles account for the majority of observations. This is important to know since the characteristics of the profile data at baseline will determine what improvements (specific dimensions/levels) are possible. For more on this, see: Parkin Devlin Feng (2016) What determines the shape of an EQ-5D Index distribution? Medical Decision Making. http://journals.sagepub.com/doi/abs/10.1177/0272989X16645581

5L value set Both papers free to download: www.ohe.org

EQ-5D-5L valuation protocol

Comparing 3L and 5L value sets UK 3L: Dolan 1997 UK ‘Crosswalk’ van Hout et al 2013 England 5L: Devlin et al 2016 The 3L value set, and the crosswalk value set which drew off the same data, had what we call a bimodal property. The 5L value set does not have that property.

Comparison with 3L and ‘crosswalk’   5L value set Crosswalk value set 3L value set % health states worse than dead 5.1% (159 out of 3,125) 26.7% (833 out of 3,125) 34.6% (84 out of 243) Preferences regarding dimensions (ordered from most to least important in terms of level 5 weight) Pain/Discomfort Anxiety/Depression Mobility Self-care Usual Activities Value of 55555 (33333) -0.285 -0.594 Value of 11112* 0.922 0.879 0.848 Value of 11121* 0.937 0.837 0.796 Value of 11211* 0.950 0.906 0.883 Value of 12111* 0.846 0.815 Value of 21111* 0.942 0.877 0.850 Minimum value Maximum value 1.000

Implications of the results The EQ-5D-5L value set for England has a lower range of values than the current UK EQ-5D-3L value set Proportion of states with negative values is considerably lower Implies that treatments for very severe conditions generate smaller gains than is currently assumed e.g. interventions that reduce the level of anxiety/depression from extreme to severe Treatments seeking to alleviate pain/discomfort and anxiety/depression are highly valued and most likely to be prioritised Mobility is less influential than before Life-extending treatments may generate larger gains than is currently assumed

Concluding remarks PRO data are often under-analysed. Be aware of the nature of the underlying profile data in your sample Look at patients’ EQ-VAS data Be aware of the characteristics of the value set (or scoring system) used to summarise profiles – results may be affected by it (sensitivity analysis) Understand the normative issues determining choice of value set, and whether relevant to your study (eg if not estimating QALYs, utilities may not be the most relevant)