1. Patient-Reported Outcomes Measurement Group Elizabeth Gibbons Senior Research Officer elizabeth.gibbons@dphpc.ox.ac.ukhttp://phi.uhce.ox.ac.uk/

2. Overview PROM Group Oxford PROM Group Oxford Outline of what PROMs are Outline of what PROMs are Criteria for selection Criteria for selection Examples Examples UK context UK context Discussion Discussion

3. History of group DPHPC DPHPC Part of National Centre for Health Outcomes Development Part of National Centre for Health Outcomes Development (NCHOD) and funded by the Department of Health through the NHS Information Centre for Health and Social Care. Now core funding from DH

4. Web resources systematic of PROMs relevant to specific disease and population groups systematic reviews of PROMs relevant to specific disease and population groups bibliographic database (up to 2005) comprising over 16000 records relating to PROMs with keyword search facility bibliographic database (up to 2005) comprising over 16000 records relating to PROMs with keyword search facility general information on instrument selection general information on instrument selection links to related websites including instruments, resources, organisations, research groups, and journals links to related websites including instruments, resources, organisations, research groups, and journals information about the Oxford Orthopaedic scores including PDFs of the questionnaires and guides to their usage. information about the Oxford Orthopaedic scores including PDFs of the questionnaires and guides to their usage.

6. What are PROMs? Patient’s or public’s reports of health Patient’s or public’s reports of health Obtained by questionnaire: Obtained by questionnaire: Self-completed Self-completed Interview Interview Computer terminal Computer terminal Internet Internet Palm held devices Palm held devices Mobile phones Mobile phones

7. Dimensions Physical function Physical function Symptoms Symptoms Global judgments of health Global judgments of health Psychological well-being Psychological well-being Social well-being Social well-being Cognitive function Cognitive function Role activities Role activities Personal constructs (eg stigma, satisfaction with bodily appearance) Personal constructs (eg stigma, satisfaction with bodily appearance) Satisfaction with care Satisfaction with care Fitzpatrick R., Davey C, Buxton M., Jones D., Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment 1998; 2: (14)1- 74.

8. Type of PROM Generic Generic-SF-36 Condition-specific Condition-specific -Audit of Diabetes-Dependent Quality of Life/ADDQoL Population-specific Population-specific -Child Health Questionnaire Dimension-specific Dimension-specific -psychological well-being: Patient Health Questionnaire (PHQ-9) -psychological well-being: Patient Health Questionnaire (PHQ-9) Utility measure Utility measure-EQ-5D Individualised measure Individualised measure -Patient Generated Index

9. Generic vs Specific Generic/utility Generic/utility relevant to broader groups of patients relevant to broader groups of patients Condition-specific Condition-specific more sensitive to specific problems Commonly recommended to use both

10. What are patient-reported outcome measures for? Individual patient care Individual patient care ~ ? Disease-specific Clinical effectiveness-RCTs Clinical effectiveness-RCTs ~ generic and/or disease-specific Cost effectiveness / cost utility Cost effectiveness / cost utility ~EQ-5D utility value to construct a QALY Evaluation of services~ ? Evaluation of services~ ?

11. Patient-reported outcome measures Systematic search of 11 major databases Systematic search of 11 major databases 1275 separate instruments in 2002 1275 separate instruments in 2002 3215 separate instruments in 2007 3215 separate instruments in 2007

12. Evaluation of bibliography 0 500 1000 1500 2000 2500 Rheu/musc Cancer Mental health Neurological Elderly Paed/adol Cardiovascular Respiratory Diabetes Population or health condition Cummulative number of records 1999 2005

13. Instrument type 1999 & 2005 Instrument type 1999 & 2005

14. Choosing an instrument A potentially complex task A potentially complex task Structured reviews of evidence Structured reviews of evidence Selection criteria Selection criteria Web-based advice: Web-based advice: http://phi.uhce.ox.ac.uk/ http://www.proqolid.org/ NHS Guidance: currently PROMs for elective procedures in acute services NHS Guidance: currently PROMs for elective procedures in acute services

15. Structured reviews Chronic conditions (2006; 2009) Chronic conditions (2006; 2009) Breast and prostate cancer Breast and prostate cancer Kidney Kidney Children Children Mental health Mental health Elective procedures Elective procedures

16. Review methodology Fitzpatrick R., Davey C, Buxton M., Jones D., Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment 1998; 2: (14)1-74. Fitzpatrick R., Davey C, Buxton M., Jones D., Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment 1998; 2: (14)1-74. LSHTM (2004) LSHTM (2004) Patient-Reported Outcome Measures (PROMs) for routine use in Treatment Centres: recommendations based on a review of the scientific evidence. Terwee (2007) Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology 60 (34-42) Terwee (2007) Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology 60 (34-42)

17. Methods for establishing a PROM Clarify goal of proposed instrument Clarify goal of proposed instrument Generate items from patient-oriented interviews, focus groups Generate items from patient-oriented interviews, focus groups Item selection Item selection Formatting and testing of format of instrument Formatting and testing of format of instrument Studies to examine measurement properties Studies to examine measurement properties

18. Measurement properties Reliability: Does the instrument produce results that are reproducible and internally consistent? Reliability: Does the instrument produce results that are reproducible and internally consistent? Validity: Does the instrument measure what it claims to measure? Validity: Does the instrument measure what it claims to measure? Content validity: Qualitative evidence Content validity: Qualitative evidence

19. Measurement properties Construct validity: High correlations between the scale and relevant constructs preferably based on a priori hypothesis with predicted strength of correlation High correlations between the scale and relevant constructs preferably based on a priori hypothesis with predicted strength of correlation Statistically significant differences between known groups and/or a difference of expected magnitude Statistically significant differences between known groups and/or a difference of expected magnitude

20. Measurement properties Responsiveness: Does the instrument detect changes over time that matter to patients? Responsiveness: Does the instrument detect changes over time that matter to patients? ~ Statistically significant changes on scores from pre to post-treatment and/or difference of expected magnitude Precision How precise are the scores of the instrument? Precision How precise are the scores of the instrument? ~ Floor/ceiling effects for summary scores <15%

21. Measurement properties Interpretability How interpretable are the scores of the instrument? Clinically important difference- minimally important difference

22. Operational characteristics Acceptability: Low levels of incomplete data or non-response Acceptability: Low levels of incomplete data or non-response Feasibility/burden: Reasonable time and resources to collect, process and analyse the data Feasibility/burden: Reasonable time and resources to collect, process and analyse the data

23. Appraisal Rate quality of evidence for each instrument identified in relation to each of key criteria Rate quality of evidence for each instrument identified in relation to each of key criteria Reliability, validity, responsiveness, interpretability, acceptability, feasibility Reliability, validity, responsiveness, interpretability, acceptability, feasibility

24. Appraisal Each aspect for each instrument rated on 5 point scale: Each aspect for each instrument rated on 5 point scale: 0 No evidence _ Negative evidence + Limited evidence in favour ++ Some good evidence in favour +++ Good evidence in favour

25. Examples Patient Health Questionnaire (PHQ- 9)- QOF-DEP2 Patient Health Questionnaire (PHQ- 9)- QOF-DEP2 Audit of Diabetes-Dependant Quality Of Life (ADDQoL) Audit of Diabetes-Dependant Quality Of Life (ADDQoL) Euroqol-EQ-5D Euroqol-EQ-5D

26. Scoring PHQ Scores range from 0 to 27 with a three point scale for the 9 items. Scores range from 0 to 27 with a three point scale for the 9 items. Mild depression is considered with scores of 5 to 9 Mild depression is considered with scores of 5 to 9 Moderate for scores between 10 and 14 Moderate for scores between 10 and 14 Severe, 20 to 27 Severe, 20 to 27 (Spitzer et al., 1999).

27. Views of patients and GPs Routine implementation of measures of the severity of depression (PHQ, HADS, BDI) GPs were cautious about the validity and utility and doubtful about the motives behind their implementation. GPs considered that clinical experience was more important in identifying cases and assessment of the severity of depression. Objective measurement reduced the humanistic aspect of the consultation as well as de-skilling doctors GPs were cautious about the validity and utility and doubtful about the motives behind their implementation. GPs considered that clinical experience was more important in identifying cases and assessment of the severity of depression. Objective measurement reduced the humanistic aspect of the consultation as well as de-skilling doctors Patients on the other hand were positive about the utility of the measures and viewed the process as a structured adjunct to the consultation. GPs were more attentive to their problems. Some patients reported that the questionnaires were useful in helping them think about their depression and enable them to express themselves better. Patients on the other hand were positive about the utility of the measures and viewed the process as a structured adjunct to the consultation. GPs were more attentive to their problems. Some patients reported that the questionnaires were useful in helping them think about their depression and enable them to express themselves better. (Dowrick et al., 2009)

28. Others ADDQoL-conceptual issues ADDQoL-conceptual issues EQ-5D EQ-5D Brevity vs comprehensiveness of items

30. UK DH Policy Lord Darzi’s Interim Report on the future of the NHS recommends that (PROMs) should have a greater role in the NHS. ‘It could be useful to build on recent advances in measuring outcomes as assessed by patients themselves, and make these patient-reported outcome measures a stronger part of the approach to quality’. Lord Darzi’s Interim Report on the future of the NHS recommends that (PROMs) should have a greater role in the NHS. ‘It could be useful to build on recent advances in measuring outcomes as assessed by patients themselves, and make these patient-reported outcome measures a stronger part of the approach to quality’. The new Standard NHS Contract for Acute Services includes a requirement to report on evidence from PROMs from April 2009. The new Standard NHS Contract for Acute Services includes a requirement to report on evidence from PROMs from April 2009.

32. Current policy For the first-time payments to NHS hospitals will be adjusted according to patient satisfaction and health outcomes. For the first-time payments to NHS hospitals will be adjusted according to patient satisfaction and health outcomes.

33. LTC Pilot study Pilot evaluation of the feasibility of collecting PROM data in primary care Pilot evaluation of the feasibility of collecting PROM data in primary care Six LTCs: asthma, COPD, diabetes, epilepsy, heart failure, stroke Six LTCs: asthma, COPD, diabetes, epilepsy, heart failure, strokeReviews Postal survey Interviews with non-responders Interviews with GPs. commissioners

34. Impact of PROMs: the commissioners of services PROMs in the context of routine measuring of quality of NHS trusts PROMs in the context of routine measuring of quality of NHS trusts Will raise an number of interesting methodological challenges if they are to inform decisions about quality Will raise an number of interesting methodological challenges if they are to inform decisions about quality PROMs less clearly understood outside the context of research- based RCTs where role is clearer PROMs less clearly understood outside the context of research- based RCTs where role is clearer

35. The Office Department of Public Health