User Involvement in Mental Health Research

User Involvement in Mental Health Research
Diana Rose and Jenny Walke Service User Research Enterprise (SURE) Institute of Psychiatry King’s College London

Purpose To demonstrate how service user-focused research can add something to the methods and findings of conventional research To show why service user researchers are best-placed to do this Involving service users at all stages in the research process

Overview A little bit about one of us (DR) Some introductory slides
Service user research User-focussed monitoring Systematic reviews Outcome measures Criticisms and answers The NIHR continuum

Having a double identity (DR)
Service user all my adult life academic career 1986 medically retired – ‘living in the community’ 1996 two identities come together to do user-led research

Colney Hatch Lunatic Asylum (est 1854)
This is Colney Hatch Lunatic Asylum originally built on the land outside North London. Gradually the city encroached and it became part of a suburb. At it height in the mid 1950s it accommodated over 2,000 people. It was renamed Frien Hospital in the 20th century. This was how people with mental health problems were treated for nearly 200 years.

Friern Hospital corridor 1976 – I walked the line
Friern Hospital was reputed to have the longest corridors of any institution in Europe. They were certainly long when I walked down them as a patient in But things were already changing and the hospitals were closing so you were not necessarily admitted to a long-stay ward as in the past. You could be there for just a few weeks.

Friern Hospital patient (1984)
This is one of the last patients in Friern Hospital. We do not know if she is suffering from a form of dementia or is on very high levels of sedative medication. Or both.

Camden Mental Health Consortium Resettling Friern Patients
Camden – central and north London Friern – patients to be resettled to this area Camden Mental Health Consortium – one of first local user groups Formed to make sure the resettlement was done well Not user-led at this point – that came later Camden is an area in North London. Camden Mental Health Consortium (CMHC) was one of the UK’s first local user groups. One of its aims was to make sure the final people left in Friern, like the lady in the last slide, had a decent quality of life when they were resettled into the community. The group also did some radical things but was not user-led at this point (1980s).

Service user research - history
1988 – CMHC and GPMH project on new inpatient provisions in District General Hospital 1996 – two user-led projects based in NGOs Strategies for Living (Mental Health Foundation) User-Focused Monitoring (Sainsbury Centre for Mental Health) Political: workers members of the user movement and took research questions from the movement To make a contribution – our ‘evidence’ for the movement This slide contains a brief history of the beginnings of service user-led research in mental health. Apart from the work of Viv Lindow in Bristol, the first piece of research was a collaboration between Camden Mental Health Consortium (CMHC) and a voluntary organisation called Good Practices in Mental Health. So CMHC was active in both policy and research. The project was about service users’ view of teh new inpatient provision in a District General Hospital that had replaced Friern. Some years later, there were two initiatives in charities. One was called Strategies for Living and was based in the Mental Health Foundation. This looked at people’s own strategies for dealing with mental distress alongside of or instead of statutory services. The coordinator was a service user and so were all the interviewers. The other project was a peer evaluation of community and hospital services and evaluated these in specific localities. Again, the coordinator was a service user who trained up local service users in research skills. Both these projects published reports in the year The service users who led this research were members of the service user movement and took their questions and sometimes their methods from it. They were openly political which some critics understood as biased. We will come back to that later.

User research moves on (or backwards)
Early 2000s two groups in universities SURESearch (Birmingham) Service User Research Enterprise (SURE) Service user researchers have ‘insider knowledge’ SURE has ‘collaborative’ management structure – one conventional university researcher and one user researcher SURESearch more complex but ‘head’ is conventional university researcher Service user researchers know things about distress, treatments and services that are closed to conventional researchers and clinicians. This relevant personal experience is in addition to, not instead of, conventional academic training/qualifications.

Activity Now test your knowledge - correct answers revealed on final slide. 1. What was the original name of Friern Hospital? Bedlam Colney Priory Colney Hatch Lunatic Asylum 2. How many patients did Friern accommodate by the 1950s? 500 2000 5000

3. What does CMHC stand for?
Community Mental Health Clinic Centralised Mental Hospital Care Camden Mental Health Consortium 4. When were two reports from service user-led projects published? 2000 2002 2005

Further Reading Taylor B: The Last Asylum. London: Penguin; 2014.
Survivors’ History Group, Mental health and survivors' movements and context:

User-Focussed Monitoring

Method of Peer-Review of Mental Health Services
Started in Central London in 1996 Community services Hospital services Expanded across UK and across service types Now also used in Nordic countries Norway “User ask user”

UFM is User-Led Co-ordinator who is a user but also with research skills Visits local area to identify service users interested in project Get together over lunch to make final decision about whether they want to be involved

Some UFM projects questionnaire-based
Group devises questionnaire on the basis of their experience of the service to be assessed Takes many meetings as co-ordinator synthesises ideas and then brings back to group for amendment

Training Some involved in UFM projects have never done an interview
Some basic training But mostly role-play: opportunity to experience answering as well as asking the questions

More qualitative projects
Focus groups have been used Topic guide devised in the same way as for questionnaires but shorter as want to give group scope to expand on their views Some training for this too

Collecting the information
Interviewers go to many different venues: CMHTs, hospitals, participants’ homes, charities Focus groups not usually held in NHS venues – more neutral De-briefing – co-ordinator speak with interviewer immediately after each interview to check no problems

Analysing the data For questionnaires there is some statistical analysis Problematic from involvement perspective as few service users have these skills Easier for members of UFM team to be involved in analysing qualitative data Can bring their experience to bear

What happens to the report?
UFM projects are locally commissioned e.g. Trusts, local authorities, charities Report goes to commissioners Up to them what they do with it Some take very seriously and results in measurable changes Some take not so seriously! Sometimes have feedback days for staff

Activity 5. When did peer review of mental health services start in Central London? 1983 1990 1996 6. Why aren’t focus groups typically held in NHS venues? Lack of space Lack of neutrality Lack of insurance

7. At what level are UFM projects commissioned?
Local National International

Further Reading Rose D, Fleischmann P, Schofield P: Perceptions of User Involvement: a User-Led Study. International Journal of Social Psychiatry 2010, 56(4):

New method – patient-centred systematic reviews
Systematic reviews a gold standard in conventional research. Attempt to estimate the effect of a treatment by pooling together and averaging different studies on that treatment. The studies together provide a better estimate of effect than any one study alone.

Example - ECT Consumers’ perspectives on ECT
Two main researchers had experienced ECT – “insider knowledge” Adapted method of systematic review to make it user-focused Included peer-reviewed literature as normal in systematic reviews Included the ‘grey’ literature Included qualitative data – ‘testimonies’ Assembled 26 clinical papers and 9 authored by service user groups Testimonies sourced from a video archive and the internet Orthodox systematic reviews don’t include the grey literature or qualitative testimonies. Thought insufficiently scientific. Included so the user voice would be present.

Main themes Perceived benefit Retrograde memory loss Information
Consent and perceived coercion Many people who have had ECT complain of long-term (retrograde) memory loss. In England, must sign a consent form before ECT is administered. Concerned to see whether this consent was freely given.

Perceived benefit Conventional research showed much higher levels of satisfaction with ECT than user research – no overlap in the estimates of benefit between the two groups Critique of method in grey literature We answered with critique of method in clinical papers Some said user research was biased in its sampling – only discontented people would return the surveys (why?). User researchers suggested mainstream work biased because being asked about satisfaction by someone connected to the hospital would likely overestimate the numbers saying they benefitted. Patients do not want to complain.

ECT continued BOTH conventional and grey literature showed high levels of memory loss though conventional did not discuss this or even said it was not important BOTH showed people did not feel informed but again this absent from discussion in clinical papers Perceived coercion – testimonies showed some people felt coerced into signing form Same results but different interpretations – not like the finding on perceived benefit where there were different results Argument that if a finding is not discussed in a paper then that paper is less scientific than one that does discuss it. In this case papers and reports written by user groups more scientific than mainstream ones.

The Dispute Paper published in a high-profile medical journal
Royal College of Psychiatrists issued press release disputing what we had said about memory loss Mendacious – they mis-quoted themselves

Policy NICE new guidelines on ECT as our research work being done
Consumer review influenced Especially around information and consent including about risk of memory loss User research can have an effect on national policy Now patients considering ECT must be fully informed of the risk of memory loss and professionals must take care not to coerce.

Activity 8. Which two of the following are omitted from orthodox systematic reviews? Grey literature Randomised controlled trials (RCTs) Peer-reviewed literature Qualitative testimonies 9. Issues of consent and memory loss are associated which of these treatments? Cognitive Behavioural Therapy (CBT) Occupational therapy Mindfulness Electro-convulsive therapy (ECT)

Further Reading Rose D, Fleischmann P, Wykes T: Consumers' views of electroconvulsive therapy: A qualitative analysis. Journal of Mental Health 2004, 13(3):

Patient Generated Patient Reported Outcome Measures (PG-PROMs)
Outcome measures are needed in research to assess the effectiveness of treatments. Also used clinically. Most outcome measures are compiled and filled in by clinicians and academics. May not measure the things important to service users.

The claim of neutrality in Randomised Controlled Trials (RCTs)
RCTs considered the ‘gold standard’ in medicine Neutrality depends on blinding (not knowing certain details) But is everything in an RCT neutral? Outcome measures devised by clinicians and academics May not be the outcomes that matter to service users Try to develop measures that are valued by service users (and others) in mental health Not everything in an RCT is neutral. The outcome measures reflect clinicians’ values. ‘Blinding’ means concealing information that may bias the trial until it is completed. ‘Double blind’ is when neither participant nor researcher knows who is assigned to which experimental condition/group. This can reduce the likelihood of subjective influences and placebo effects.

Patient Reported Outcome Measures (PROMs)
Much talk of PROMs But only filled out by patients – no say in which questions are asked Patient-Generated PROMs (PG-PROMs) Medication side-effects Experiences of inpatient care In the last few years, move to something called Patient Reported Outcome Measures. But these are merely filled in by patients. Our method is to generate PROMs entirely from the service user perspective, to reflect their views and priorities.

Method Participatory research
Attempts to reduce the power relations between researcher and researched In user-focused research, researchers have the same or similar experiences as the participants All are mental health service users A new development even within participatory research Potential influences on treatment and policy The method used is participatory research and it aims to reduce the power imbalance in the research process. The researchers are also service users and so share something with the participants in the research. Does not have to be exact. A method called a Discrete Choice Experiment (choosing the ‘least worst’ side effects) can inform drug developers of key side-effects to avoid from the service user’s perspective.

Procedure Focus Groups which meet twice
Recruited because they have experience of what the measure is attempting to tap Facilitators/researchers have experience of the treatment or service that is being evaluated On basis of focus group discussions, researchers draw up draft measure Taken to Expert Panels for amendment and refinement and that the language is their own language Feasibility study to make sure it is easy to complete – refinements all the way Psychometric testing About 200 participants involved in all Mixed methods. The focus groups and expert panels draw on qualitative research methods. Assessing the stability and validity of the measure (psychometrics) is done quantitatively.

Example – in-patient care
Much anecdotal evidence and evidence in the ‘grey’ literature that profoundly disliked My experience too Wanted to do something more rigorous Finally became the main outcome measure in an RCT evaluating the introduction of psychological therapies on acute wards Collaborative but our part is user-led

User Measure: VOICE Participants
People who had been in-patients in the local Trust within the previous two years One group specifically made of participants who had been detained Measure is called VOICE (Views of Inpatient Care). Participants in focus groups had been inpatients in the local Trust within the previous two years. Convened a specific group for people who had been detained although in the event all groups contained people who had been detained.

Focus groups 6-8 people: 4 groups
One facilitator with experience of in-patient care and other also a service user Meet twice to make sure we have accurately captured their views Thematic analysis using Nvivo software after both 1st and 2nd wave Respondent validation – check with participants that we have accurately captured their views. Nvivo is software for analysing qualitative data.

Drafting the measure Done by the researchers on the basis of the qualitative analysis Quantitative and qualitative questions Researchers also use own experience The measure has sets of scaled questions followed by a box where people can write what they like.

Expert Panels One drawn from focus group members and one independent
Also been inpatients in previous two years Tasked with amending and refining the measure and making sure that language and layout are appropriate Usually quite a few changes made at this stage Expert panel from focus groups to make sure the measure is consistent with what they wanted. Independent expert panel for a final and fresh look at the measure. Language and layout are important as people will be filling this in themselves.

Feasibility study Final stage of constructing measure
~50 people complete the measure and we find out which parts are easy to complete and which not Iterative process In this project the participants were actually in hospital Purpose of feasibility study is to make sure the measure is easy to complete. Give to participants in groups of ten and progressively refine until all find it easy to complete. In this case, participants actually in hospital.

Psychometrics Mixed methods research
We do this because it is appropriate but also to show that user-led research can be rigorous One mainstream researcher said users could never produce measures because they would never understand factor analysis ‘Mixed methods’ research combines the collection and analysis of both quantitative (numeric) and qualitative (descriptive) data. Psychometrics are techniques for finding out whether a measure is good or not – stable, reliable, measuring what it is supposed to measure. Important that user-led research should be rigorous. Not all mainstream researchers think we are up to this.

Psychometrics Continued
Test-retest reliability: the same people fill in the measure twice with an interval of a week in between Are the scores the same the 2nd time? This would mean the measure is stable. VOICE very stable People in this exercise mostly a diagnosis of psychosis which might compromise stability. Good test-retest reliability because measure developed by service users? Two key considerations are: Validity - the extent to which an indicator measures what it intends to measure Reliability - the extent to which a measure produces consistent results.

Psychometrics cont Criterion validity: assess measure against an existing one Expect some differences because of means of development: close relationship but there were differences Acceptability: how does it feel to fill it in Enjoyable? Distressing? Right length? Criterion validity: How far does the measure correlate with another (known to be valid) indicator of the same concept? E.g. Reported self-alcohol consumption and blood alcohol level. However, there are rarely if ever true ‘gold standard’ measures for things; rather, we look for indicators where there is most agreement on the extent to which they indeed measure what they claim to measure.

Activity 10. Who compiles most existing outcome measures?
Patients’ families and carers Clinicians and academics Hospital administrators 11. What is ‘blinding’? Conducting research without obtaining consent Assigning participants to different groups Temporarily withholding information to reduce bias

12. What name is given to techniques for finding out whether a measure is stable and valid?
Feasibility study Psychometrics Expert panels Quantitative methods

13. What is the main purpose of a feasibility study?
To ensure a measure is easy to complete To check cost-effectiveness of a measure To advertise your research

Further Reading Evans J, Rose D, Flach C, Csipke E, Glossop H, McCrone P, Craig T, Wykes T: VOICE: Developing a new measure of service users' perceptions of inpatient care, using a participatory methodology. Journal of Mental Health 2012, 21(1):57-71.

Challenges to user-led research and user-produced knowledge

Frank Scepticism Peter Tyrer, past editor of the British Journal of Psychiatry, writes: “The engine of user involvement, while welcome in principle,……….may drive mental health research into the sand.”

Power Most of the projects we have been involved with are ‘collaborative’ Nearly always headed up by professor(s) of psychiatry or psychology Not just status or naked power although that exists– more subtle Are you a researcher or are you a patient? “I wonder what your diagnosis is, then.” Undermining user-produced knowledge Rose, D. (2003). “Having a diagnosis is a qualification for the job.” British Medical Journal 326 (14 June): 1331.

Hierarchies of Evidence
RCT is the ‘gold standard’ But RCTs are not neutral – outcome measures devised by clinicians ‘Expert opinion’ counts as evidence but only as the weakest form Experts are psychiatrists Users as experts – a different knowledge perspective The Cochrane Hierarchy needs revisiting Evidence-based medicine hierarchies rank study types based on the strength and precision of their research methods. There are different hierarchies for different question types, and even experts may disagree on the exact rank of information in the evidence hierarchies. Still, most agree that current, well designed systematic reviews and meta-analyses are at the top of the pyramid, and that expert opinion and anecdotal experience are at the bottom. The Cochrane Collaboration is an international voluntary organization that prepares, maintains and promotes systematic reviews of the effects of healthcare.

The charge of bias 1 Said, mostly implicitly, that user-research is biased, anecdotal and carried out by people who are over-involved – ENMESH conference We make no pretence of neutrality But all research comes from a certain standpoint Conventional researchers think what they do is ‘obvious’ – nothing is obvious User researchers more explicit about this than mainstream researchers In my opinion the word ‘bias’ should be banished from research discourse and all researchers should clearly say where they are coming from. ENMESH = European Network for Mental Health Service Evaluation

Bias 2 Although implicit some seem to think that irrational people (the mad) cannot engage in the supremely rational activity of science We epitomise ‘unreason’ (Foucault) So more difficult to find legitimacy and credibility than consumer researchers in other medical disciplines Paradox – user research in mental health ahead of that in other specialties ...modern man no longer communicates with the madman [...] There is no common language: or rather, it no longer exists; the constitution of madness as mental illness, at the end of the eighteenth century, bears witness to a rupture in a dialogue, gives the separation as already enacted, and expels from the memory all those imperfect words, of no fixed syntax, spoken falteringly, in which the exchange between madness and reason was carried out. The language of psychiatry, which is a monologue by reason about madness, could only have come into existence in such a silence. —Foucault, Madness and Civilization (1961)

Some answers?

A new epistemology for user-led research – learning from feminism
‘Timeless oppositions’: Reason / unreason Culture / nature Intellect / emotion First are male attributes and valorised So have women been excluded from science and science not attentive to the concerns of women

Timeless oppositions and madness
The mad positioned as nature, emotion and crucially unreason The Enlightenment valorised reason and so positioned the mad as its antithesis: Unable to reason Unable to labour The Great Confinement According to Foucault, in the mid-seventeenth century, in the depths of the age of reason, the rational response to the mad, who until then had been consigned to society's margins, was to separate them completely from society by confining them, along with prostitutes, vagrants, blasphemers and the like, in newly created institutions all over Europe – a process he calls "the Great Confinement”.

Standpoint epistemology and strong objectivity
Standpoint epistemology has been seen as essentialist Sandra Harding and ‘strong objectivity’ We have access to two discourses Our own That of conventional science A more complete picture – even a challenging one Sandra Harding used the term ‘strong objectivity’ to describe research that starts from the experiences of those who have traditionally been left out of the production of knowledge. She contrasted this with the ‘weak objectivity’ of (supposedly) impartial scientific investigation.

Implications User-led research can add to the jig saw or it can challenge conventional research But needs to critique the methodologies and epistemologies of conventional research to do this successfully

Other ways service users may become involved in research
Everybody in SURE employee of IoP BRC Service User Advisory Group (SUAG) Reference groups for projects e.g. The Interface Study Sitting on Steering Groups Problem of tokenism although in SURE also Should be parity of esteem and payment Training resources for service users Ongoing research: Maudsley Toolkit, EQUIP etc - developing training materials for mental health service users to strengthen their participation in research and help them to set up their own projects.

INVOLVE (NIHR) Continuum
Consultation Collaboration User-control Researcher-initiated Collaboration split Jointly initiated User-initiated INVOLVE is a national advisory group that supports greater public involvement in NHS, public health and social care research. It is funded by and part of the National Institute of Health Research (NIHR) and has around 30 members, including health and social care service users and practitioners, carers, people from voluntary organisations, managers and researchers.

Activity 14. The Cochrane Collaboration is associated with which type of evidence? Systematic reviews Focus groups Ethnography Service user research 15. Which two of the following statements are true? RCTs are neutral RCTs are considered the ‘gold standard’ of evidence Service user research is neutral User-led research can reduce the power imbalance in the research process.

16. Sandra Harding used which term to describe research by groups traditionally excluded from knowledge production? Enlightenment Strong objectivity Unreason Evidence-based medicine 17. Which is NOT a level of user engagement on the INVOLVE continuum? Consultation Collaboration Facilitation User-control

Further Reading Sweeney A, Beresford P, Faulkner A, Nettle M, Rose D (eds.): This is Survivor Research. Ross-on-Wye: PCCS Books; 2009.

Activity Answers Q. ANSWER(S) 1. c 10. b 2. 11. 3. 12. 4. a 13. 5. 14.
6. 15. b, d 7. 16. 8. a, d 17. 9. d

User Involvement in Mental Health Research

Similar presentations

Presentation on theme: "User Involvement in Mental Health Research"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

User Involvement in Mental Health Research

Similar presentations

Presentation on theme: "User Involvement in Mental Health Research"— Presentation transcript:

Similar presentations

About project

Feedback