1. The role of patient reported outcome (PRO) data in HTA: issues in measuring and valuing health
Professor Nancy Devlin
Director of Research
OHE
PSI conference, May 16th 2017

2. Content
Patient reported outcomes (PROs) – what do they measure? Why are they important?
Descriptive systems and profile data
Analysing profile data
3. Patients’ assessments of overall health
4. Scoring and values/utilities
Sources/properties of values; implications for statistical analysis of utilities
5. Differences between the 3L and 5L; implications for HTA
And I will end with some concluding remarks

3. Much of what I will say today is drawn from these papers

4. 1.PROs: what do they measure?
The goal of most health care is to improve patients’ health
Patients arguably the best judge of how they feel
“The use of PRO instruments is part of a general movement toward the idea that the patient, properly queried, is the best source of information about how he or she feels” (FDA 2006)
PROs: aim to capture patients’ subjective account of their own health in a structured way.
Psychometric properties: validity; reliability
PRO data a useful complement (not substitute) for clinical endpoints.

5. How is health described?
Types of PROs
How is health described?
How is health summarised?
Generic
Condition-specific
Scores
Eg. SF-36
Eg. Oxford Hip Score, Oxford Knees Score (literally thousands of such PROs exist)
Values/utilities
Eg. EQ-5D
HUI
SF-6D
AQOL
15D
Mapping to preference based measures (various methods)
Some condition specific instruments accompanied by value sets eg. EORTC-8D, derived from the EORTC QLQ-C30 for use in patients with cancer
Notes:
- generic and condition specific instruments have a different role: generic allows comparisons across disease areas (to weigh up benefit gained with benefit foregone elsewhere); condition-specific provide rich data on specific disease/health problem.
Common to map from disease specific to preference based instrument (various methods; map to utilities or map to generic descriptive system) but preferable to collect generic data
Best practise: collect both generic and condition specific.

6. Uses of PROs Clinical trials Observational studies
Population health surveys
Routine collection in health service delivery, eg.
‘PROMs’ in English NHS
UK private health care sector (PHIN)
Alberta Health Services, Canada
New Zealand Southern Cross.

7. EQ-5D descriptive system and profile
One of the most common generic, preference-based PROs.
Available 5 level (shown here) or 3L (used since 1990s; recommended by NICE)

8. EQ-VAS: Patients’ overall assessment of their health

9. Utilities
Single number summaries of how good or bad each health state is
Anchored at 1= full health, 0 = dead, < 0 worse than dead (as required by QALYs)
Based on stated preferences of general public considering hypothetical states
With exceptions eg. Sweden’s TLV prefers patient’s values
Both could be argued to be relevant:
“health economic guidelines could require analysis of benefit in terms of QALYs based on both patient and general public preferences” (Versteegh & Brouwer 2017)

10. Utility/happiness Quality of life
What EQ-5D and other PROs measure is nested within broader concepts of QoL and utility.
Conceptually, the borders between HR-QoL and QoL and between QoL and utility are not precisely defined.
Quality of life
Health-related
quality of life
Health status
EQ-5D profile captures specific aspects of HR-QoL
EQ-VAS (a ‘feeling thermometer’) which captures the patients’ overall assessment of how good or bad their health is;
Utilities (values) which capture the general public’s affective forecasts of utility in those states
Increasing interest in broader concepts: social care outcomes; subjective wellbeing and ‘happiness’

11. Profiles: simple descriptive analysis
Nature of profiles observed + value set characteristics drive observed distribution of utilities
eg. in these 3L data, no level 3 observed on mobility
‘N3’ term on 3L utilities important characteristic of utility
This sort of analysis aggregates data by dimensions/levels. It doesn’t show how dimensions/levels combine on a within-respondent basis

12. Pareto classification of health change
Comparing a patient’s PRO profile at any 2 points in time:
The same
Better on at least 1 dimension, no worse on others = improvement
Worse on at least 1 dimension, and no better on others = worsening
Better on some dimensions, worse on others = mixed

13. What is the EQ-VAS measuring?

14. 4. Scores and utilities
Act to weight the levels/dimensions in profiles so they can be aggregated into a single number
Preference based (values/utilities) or non- preference based (scores)
Utilities: 1= full health, 0 = dead, < 0 worse than dead
Neither are ‘neutral’ – there is no objective way of aggregating profile data
Which value set is used introduces an exogenous source of variance, and can bias statistical inference.
The paper where we show this point is: Parkin D, Devlin N, Rice N. (2010) Statistical analysis of EQ-5D profiles: does the use of value sets bias inference? Medical Decision Making 2010; 30:

15. What causes the distribution of observed utilities?
The distribution of utilities is caused both by the properties of the value set and by the properties of the profile data
Profile data are often clustered, and highly ‘concentrated’ ie a surprisingly small number of profiles account for the majority of observations.
This is important to know since the characteristics of the profile data at baseline will determine what improvements (specific dimensions/levels) are possible.
For more on this, see: Parkin Devlin Feng (2016) What determines the shape of an EQ-5D Index distribution? Medical Decision Making.

16. 5L value set
Both papers free to download:

17. EQ-5D-5L valuation protocol

18. Comparing 3L and 5L value sets
UK 3L: Dolan 1997
UK ‘Crosswalk’ van Hout et al 2013
England 5L: Devlin et al 2016
The 3L value set, and the crosswalk value set which drew off the same data, had what we call a bimodal property. The 5L value set does not have that property.

19. Comparison with 3L and ‘crosswalk’
5L value set
Crosswalk value set
3L value set
% health states worse than dead
5.1%
(159 out of 3,125)
26.7%
(833 out of 3,125)
34.6%
(84 out of 243)
Preferences regarding dimensions
(ordered from most to least important in terms of level 5 weight)
Pain/Discomfort
Anxiety/Depression
Mobility
Self-care
Usual Activities
Value of (33333)
-0.285
-0.594
Value of 11112*
0.922
0.879
0.848
Value of 11121*
0.937
0.837
0.796
Value of 11211*
0.950
0.906
0.883
Value of 12111*
0.846
0.815
Value of 21111*
0.942
0.877
0.850
Minimum value
Maximum value
1.000

20. Implications of the results
The EQ-5D-5L value set for England has a lower range of values than the current UK EQ-5D-3L value set
Proportion of states with negative values is considerably lower
Implies that treatments for very severe conditions generate smaller gains than is currently assumed
e.g. interventions that reduce the level of anxiety/depression from extreme to severe
Treatments seeking to alleviate pain/discomfort and anxiety/depression are highly valued and most likely to be prioritised
Mobility is less influential than before
Life-extending treatments may generate larger gains than is currently assumed

21. Concluding remarks PRO data are often under-analysed.
Be aware of the nature of the underlying profile data in your sample
Look at patients’ EQ-VAS data
Be aware of the characteristics of the value set (or scoring system) used to summarise profiles – results may be affected by it (sensitivity analysis)
Understand the normative issues determining choice of value set, and whether relevant to your study (eg if not estimating QALYs, utilities may not be the most relevant)