1. Recovering Quality of Life (ReQoL)
Development of a new outcome measure for mental health services
Anju Keetharuth, John Brazier,
Janice Connell, Michael Barkham,
Tom Ricketts, Lizzie Taylor-Buck, Jill Carlton,
Kirsty McKendrick.

2. Demand for an outcome measure
DH published: ‘No health without mental health: a cross-government mental health outcomes strategy for people of all ages’ with emphasis on recovery from user point of view
DH – Closing the Gap 2014
NHS Outcome Framework:
NHS England: National Tariff Payment System
NICE – concerns with limitations of EQ-5D in calculating QALYs for assessing cost-effectiveness
The demand for this measure is supported by various parties mainly at the government level.
First, in the document No Health without mental health, there is growing interest in measuring recovery from the point of view of the service user.
This has been reiterated in a more recent document Closing the Gap published in January 2014.
For some time, there has been an NHS Outcomes Framework across the board looking at QoL and recovery. There are specific domains looking at these. There are placeholders for a measure and there is mention of this work.
Next, in the context of introducing a currency in the area of mental health, there is the need to collect a PROM and various sites are currently piloting the short version of the Warwick Edinburgh Well- Being Scale to kick start that process.
Finally in assessing cost-effectiveness there are concerns around the use of EQ-5D in measuring benefits in the area of mental health. Therefore, there needs to be a brief measure that can capture benefits.

3. Rationale for developing a new measure
Traditionally measures focused on symptoms (e.g. PHQ-9) and/or involved clinical assessment (e.g. HONoS), or too generic (EQ-5D and sWEMWBS)
Existing measures not able to capture wide spectrum of mental health conditions and severity (Boardman et al, 2013)
Measure required that captures the concerns of service users, such as those identified by Leamy et al (2011): Connectedness, Hope, Identity, Meaning and Purpose, and Empowerment (CHIME)
In terms of the rationale for developing a new measure, there are 3 main points:
First, traditionally measures have focussed on symptoms, for example PHQ-9 and have also been through clinical assessment, for example HoNos.
Second, in a recent report submitted to the DH, Jed Boardman, Mike Slade and Geoff Shepherd mentioned that existing measures do not adequately capture the wide range of mental health conditions and severity
Finally a measure is needed that adequately captures what matters to service users. Such concerns include those identified by Leamy and colleagues. These are often referred as CHIME which stands for Connectedness, Hope, Identity, Meaning and Purpose, and Empowerment

4. Aim and scope of research project
to develop a brief measure of recovery and quality of life for users of mental health services
Scope
For mental health populations aged 16 and over
Generic – suitable across all mental health populations including common mental health problems, severe and complex and psychotic disorders (clusters 1-17) (not dementia or learning disabilities)
For primary, secondary and tertiary mental health services
For self completion – though will have interview and proxy versions
Long version (20-30 items) and brief version (around 10 items)
The aim is to develop a brief measure of recovery and quality of life for users of mental health services
We anticipate that it will be used by adult mental health populations
It will be suitable across all mental health populations including common mental health problems, severe and complex and psychotic disorders (clusters 1-17). We are excluding dementia and learning disabilities.
The measure will be used in a range of settings: in primary, secondary and tertiary services
The measure is primarily for self-completion though we may go on to develop proxy versions or interview based one.
There will be a long version of the measure with items and for routine use a shorter measure with a max of 10 items or questions.

5. Project stages
Stage 1 Agree themes (from service users) (Connell 2012; Boardman 2013)
Output: List of domains
Stage 2 Generate items
Output: Long list of items
Stage Test face and content validity with service users
(focus groups and interviews)
Output: Refined long questionnaire
Stage 4a Item reduction
(service users across clusters and diagnoses in various settings)
Output: A pool of items (around 40 ) to be administered in Stage 4b
Stage 4b Psychometric testing of questionnaires and further item reduction
(service users )
Output: Brief questionnaire and long questionnaires
Stage Valuation (May 16 – July 16)
(Survey of general population)
Output: simple summative and QALY scoring methods
There are 5 main stages of the project.
The first stage will involve the confirmation of themes for the measure and this runs for 3 months until the end of April 2014.
Stage 2 is where we generate a long list of items from existing measures and the work in Stage 1.
From Aug – Oct, we will be testing those items to see if they make sense to service users.
In Stage 4a, we will ask service users to fill in the long measure to allow us to carry out analyses to construct a short measure.
In Stage 4b we expect the brief measure to be filled routine in a number of NHS sites and other settings across England. We will test the psychometric properties of the measure.
We are hoping to get a measure by October 2015 at the latest.
Stage 5 is the ultimate stage where we will obtain preferences of the general public so that this measure can be used to calculate QALY which is how health benefits are valued in assessing cost-effectiveness.

6. Stage 1 Identifying the domains
Systematic Review
Reviewed 13 studies
Canada/UK/Sweden/USA/Australia/NZ
Service user interviews
19 in-depth interviews
Broad range diagnosis and severity
Analysis: data extracted and reviewed by Framework to develop themes of how mental health problems impacted on people’s lives
A literature search identified 7000 studies from which 200 full texts were retrieved. A total of 13 studies were relevant for this study and they were all set in developed countries: Canada, UK, Sweden, USA, Australia and New Zealand. As there were more recovery papers than papers on Quality of Life, recovery papers were excluded for pragmatic reasons. We will come back to this point later. There was a slight bias for occupational therapy and nursing. The main limitation was that the papers identified focussed on severe mental health conditions namely schizophrenia.

7. Themes from qualitative analysis
Self-Perception
“The worse part is the lack of self worth and having to accept that one is disabled, one has a stigma which stops you from doing anything else really other than being worthless”
Well
“I would lose this anxiety that I seem to be constantly carrying with me”
Hope
“The one thing that I used to do a lot is not think about the future, think only a couple of days ahead … now, I’m more willing to think further ahead, I’m more willing to say, well in a year’s time I’d like to be at this place”
Activity
“I went on a year’s course at engineering and I was absolutely scared about going on that, but I did it and I did it, you know, quite well … and at that time, I was really happy in my life”
Well-Being
Self
Perception
Autonomy Control Choice
Relationships Belonging
Activity
Hope
Physical health
Relationships and Belonging
“I have feelings of err not belonging to the human race…, it’s not an outcast. I just don’t feel a connection…I would just like to be supported by other people.. and helping other people, that’s all I ever wanted to do”
Physical Health
“It feels physical as well as mental .. my body aches and like I think I just become really tense and that is what makes my body ache and I feel like erm I feel like my chest is being crushed and erm I can’t breath“
Included Partially included Not included
Autonomy, control and choice
“I do feel reliant on my parents … I would like to be able to have my own house and be on the property ladder and like live the life like I thought that I was going to”

8. Recovery and Quality of Life
Connell et al.
Recovery*
Leamy et al.
Belonging/Relationships
Connectedness/support/stigma and discrimination/community participation
Hope
Self perception
Identity
Activity (meaningful/enjoyable)
Meaning
Autonomy/Choice/Control
Empowerment
Well-Being
Well-Being/Symptoms
Physical Health
When we compared our findings with the domains identified by Boardman and colleagues (2013), there seems to be a lot of agreement between the domains identified from the Quality of Life literature and the recovery literature.
*Boardman et al - Assessing recovery: seeking agreement about the key domains (domain 2)

9. Extending ReQoL Young adults and adolescents
During the development of ReQoL the team found that there was an interest in whether the questionnaire could be used with young people aged between 16 and 18.
Face to face interviews and focus groups with 17 CAMHS service users.
Participants were asked to reflect on the relevance and wording of 61 items
Findings in line with Stage 3a and will be used to inform the final item selection along with results from psychometrics.
Cross-culture work
3 focus groups with Polish and Urdu speakers in Bradford

10. PROCESS FLOWCHART GE 3 STAGE 4b STAGE 3 STAGE 4a
40 items filled by 3600 participants at baseline. Follow up in progress
STAGE 2
Identifying items
n = 1597
GE 3
STAGE 3
Content validity with 59 service users
STAGE 4a
61 items filled by 2262 participants at one time point
Sifting using set of criteria
n = 87
Items added
n = 97
Qualitative feedback, psychometrics to reduce no of items
n = 40
Review by scientific group
n = 61

11. Psychometric analyses
Factor analysis aims to identify underlying unobservable (latent) variables or domains
Initial Exploratory factor analysis undertaken
Standard analysis assuming no correlation of errors
Confirmatory factor analysis and other techniques undertaken to explore further whether the items are consistent with the original themes or some other hypothesised set of domains
Rasch/ Item Response theory for reducing the number of items

12. Stage 4b
Further validation of ReQoL 40 item pool and comparison to other measures
3600 participants completed baseline questionnaires.
Follow up data collection until end of February
Four types of analysis
Responsiveness
Comparing ReQoL with other measures
Analysis by cluster, diagnoses
Other measures
WEMWBS, EQ-5D, CORE, (PHQ-9, GAD7, WSAS)

13. Looking forward Criteria for selecting items
ReQoL may have only 2 or 3 key mental health dimensions: Negative ‘distress; Positive well-being; Belonging and relationships
Criteria for selecting items
1) All the domains identified in Stage 1 of the project are represented in the final measures.
2) Items will be selected based on service user acceptability (based on data collected in stage 2, 3, 4A and 4B.  No further interviews or focus groups will be required) and on clinician acceptability and usefulness (Focus groups with clinicians will be starting in January 2016)
3) Outcomes of psychometric analyses.

14. ReQoL and the future Available for use April/May 2016
ReQoL in routine practice
In routine practice in all mental health services (primary care, secondary care, voluntary sector)
Commissioning (outcomes- based commissioning)
In clinical decision-making (longer version of the ReQoL)
ReQoL in trials, studies and economic evaluation
Summer 2016: Preference weights available for ReQoL and it can be used in cost-effectiveness analyses (trials)

15. Routine collection in the NHS – potential challenges
DH/NHS England - Mental health outcomes framework
Measuring outcomes – not part of the culture
Perception that service-users lack capacity to consent
MH – different kettle of fish compared to elective surgery
Timings- dependant on how frequently services are accessed
Administration- where, when, who with
Presentation of questionnaires- lack of engagement when unwell
Part of clinical workflow?
Feedback - Presentation of results IT systems - ££s

17. Acknowledgements All participants in the study
CRN (mental health) for their assistance with recruitment - Divisions 4 &5 Yorkshire and Humber
All participating Trusts, GP practices, charities and CASPER trial

18. THANK YOU QUESTIONS ??