1. Constructing and Validating the Short Recovering Quality of Life (ReQoL) Measure for Use in a Mental Health Population.
Anju Keetharuth, John Brazier, Janice Connell, Jill Carlton, Lizzie Taylor Buck, Tom Ricketts and Michael Barkham
School of Health and Related Research
Contact:
Website:

2. Rationale for commissioning a new measure
Concerns with limitations of EQ-5D in calculating QALYs for assessing cost-effectiveness in the areas of mental health
Policy context
Use of PROMs in commissioning value for money services
Drive to expand use of PROMs
Traditional measures focused
on symptoms or too generic
Recovery themes
(CHIME) (Leamy, 2011)
In terms of the rationale for developing a new measure, there are 3 main points:
First, traditionally measures have focussed on symptoms, for example PHQ-9 and have also been through clinical assessment, for example HoNos.
Second, in a recent report submitted to the DH, Jed Boardman, Mike Slade and Geoff Shepherd mentioned that existing measures do not adequately capture the wide range of mental health conditions and severity
Finally a measure is needed that adequately captures what matters to service users. Such concerns include those identified by Leamy and colleagues. These are often referred as CHIME which stands for Connectedness, Hope, Identity, Meaning and Purpose, and Empowerment
Connectedness
Hope
Identity
Meaning and Purpose
Empowerment

3. ReQoL (QoL literature)
Stage 1: Identifying themes
ReQoL (QoL literature)
Connell et al. 2012
Recovery literature
Leamy et al. 2011
Methods used
1. Belonging and Relationships
Connectedness
Systematic review of qualitative literature
2. Hope
Hope
3. Self perception
Identity
Qualitative interviews
with 19 service users
4. Activity
Meaning
5. Autonomy, Choice and
Control
Empowerment
Framework analysis
to identify themes
6. Well-Being
Well-being/Symptoms
There are 5 main stages of the project.
The first stage will involve the confirmation of themes for the measure and this runs for 3 months until the end of April 2014.
Stage 2 is where we generate a long list of items from existing measures and the work in Stage 1.
From Aug – Oct, we will be testing those items to see if they make sense to service users.
In Stage 4a, we will ask service users to fill in the long measure to allow us to carry out analyses to construct a short measure.
In Stage 4b we expect the brief measure to be filled routine in a number of NHS sites and other settings across England. We will test the psychometric properties of the measure.
We are hoping to get a measure by October 2015 at the latest.
Stage 5 is the ultimate stage where we will obtain preferences of the general public so that this measure can be used to calculate QALY which is how health benefits are valued in assessing cost-effectiveness.
7. Physical health

4. Stage 2: Generating items
Methods used
Identified items under each theme from:
Manuscripts from
qualitative interviews
(mainly)
Existing QoL measures
- Existing Recovery measures
2. Applied Streiner and Norman criteria
ReQoL team
N = 87
Scientific Group
N = 101
Expert User Group
N = 180
ReQoL Sheffield team
N = 122
Initial round of selection
N = 1597

5. Stage 3: Face to face validity testing
59 adult service users interviewed across 4 NHS trusts
17 younger service users aged 16-18
22 service users from South Asian and Polish backgrounds
Translatability assessment (MAPI Group)
Methods used
Participants were presented with a subset of 87 questions
Commented on the questions
Chose their favourite ones and rephrased items
Wide range of self-reported diagnosis:
Schizophrenia, Depression, Bipolar, Psychosis, OCD, Anxiety, Asperger’s syndrome, PTSD, Personality disorder, Eating disorder, Alcohol problems, Cognitive impairment, Agoraphobia, Tourette’s syndrome.
Items falling in the following categories were eliminated:
Potentially distressing
Judgmental
Difficult to respond to
Not too relevant to everyone
Difficulties with interpretation
Potentially distressing (e.g. very negative items – I detested myself)
Judgemental (e.g. advocating a way of life)
Difficult to respond to (e.g. making inference about feelings of other people)
Not too relevant to everyone (e.g. I felt loved versus I felt cared for)
Difficulties with interpretation (e.g. I felt guilty)

6. 6514 service users required A subset were followed up (N = 1237)
Stages 4 and 5: Psychometric testing
6514 service users required
A subset were followed up (N = 1237)
Methods used
Recruitment took place in secondary care, primary care and voluntary sector
20 Trusts 64%
Age Category
6 GP practices 20%
16 to %
Casper Trial Cohort 6%
30 to %
Service users completed item pools
Online panels 8%
60 to %
Voluntary sector 2%
79 and over 22%
Self report main diagnosis
Mode of administration
Face to face in clinics
Post
Online
Depression %
Anxiety %
Psychotic disorders %
Potentially distressing (e.g. very negative items – I detested myself)
Judgemental (e.g. advocating a way of life)
Difficult to respond to (e.g. making inference about feelings of other people)
Not too relevant to everyone (e.g. I felt loved versus I felt cared for)
Difficulties with interpretation (e.g. I felt guilty)
Bipolar %
Personality disorder %
Eating disorders %

7. Principles informing item selection
All six mental health domains identified in Stage 1 are
represented in the final measures
Service user acceptability (qualitative data collected in
Stages 3, 4 and 5)
Clinician acceptability and usefulness (qualitative data based
on focus groups with clinicians)
Robust psychometric properties following the application of
various analyses
In terms of the rationale for developing a new measure, there are 3 main points:
First, traditionally measures have focussed on symptoms, for example PHQ-9 and have also been through clinical assessment, for example HoNos.
Second, in a recent report submitted to the DH, Jed Boardman, Mike Slade and Geoff Shepherd mentioned that existing measures do not adequately capture the wide range of mental health conditions and severity
Finally a measure is needed that adequately captures what matters to service users. Such concerns include those identified by Leamy and colleagues. These are often referred as CHIME which stands for Connectedness, Hope, Identity, Meaning and Purpose, and Empowerment

8. Criteria to assess psychometric evidence
Analyses
Guidance/Judgement/rule
Notes
Missing data at item level
If any item has ≥5% missing data, this item should be dropped
None identified
2. Factor Analyses
Identify items with high residual correlations (>0.1)
15 pairs identified
(19 items)  
3. IRT - Misfitting items
Identify items with p values < 0.05 in at least 3 datasets. However, do not use the item level fit to exclude items. Acknowledge the misfit but retain item in the item pool.
5 items identified
4. IRT - Information functions
Ensure that items cover the whole measurement range (i.e., intensity) by choosing items to balance maximization of information over the total range and content validity (including items from all themes)
Items were chosen to cover the whole range
5.Differential Item Functioning
Exclude items with DIF
(age, ethnicity, gender, condition)
6. Sensitivity to change
Ensure selected items show change in response over time
Limited change: 2 items

9. Validation Test-retest reliability
(2 weeks interval and those reporting same health)
Intra-class coefficient
ReQoL-10 = (n = 488 general population; n = 279 patients)
ReQoL ICC = (n = 249 general population)
ICC = (n = 100 patients)
Cronbach alpha > 0.8
2. Convergent validity
Correlation with SWEMWBS and CORE-10 (summative scores) > 0.88
3. Known Group validity using Standardised effect size (SES)
General population versus:
Patient population SES ReQoL -10 = ReQoL-20 = 1.05
Common mental health SES ReQoL -10 = ReQoL-20 = 1.33
Schizophrenia SES ReQoL -10 = ReQoL-20 = 0.73
Bipolar SES ReQoL -10 = ReQoL-20 = 1.11
Personality Disorder SES ReQoL -10 = ReQoL-20 = 2.17
4. Responsiveness to change
Standardised response mean (SRM)

10. Next steps Valuation – TTO Validation Implementation
Cross-cultural adaptation work
ReQoL licences obtained from: Oxford Innovation Ltd
Link also from

11. The Meadowhead Acknowledgments Group Practice
Funder: Policy Research Programme, Department of Health, UK
Usual disclaimer applies.
Members of the governance groups: Scientific, Advisory, Stakeholder and Expert Users
All participants in the study, NIHR – CRN Mental health, staff at all participating trusts, charities and GP surgeries
MAPI Group for translatability assessment
The Meadowhead
Group Practice

12. EXAMPLE – Item selection
BELONGING & RELATIONSHIPS Top item:BEL2 – I felt lonely
Relationships – requiring a judgement on other people
Relationship –self
BEL3P
WB5
BEL1
BEL4
BEL5P
WB6
BEL2
I felt lonely
Do we need another item in this theme?
If so, which one of: BEL4 BEL3P BEL1 BEL5P WB5 and WB6
ACT2P
ACT5P

13. Relationships with others
2nd BELONGING ITEM
Relationships with others