1. VOICE Developing a New Measure of Service User Perceptions of Inpatient Care, using Participatory Research Methods Jo Evans Service User Research Enterprise (SURE) Institute of Psychiatry jo.evans@iop.kcl.ac.uk

2. Service User Research Enterprise Undertakes research that tests the effectiveness of services and treatments from the perspective of people with mental health problems Employs people with experience of using mental health services Involves service users collaboratively in the whole research process: from design to data collection, through to data analysis and dissemination of results

3. Why Involve Service Users? The inclusion of the service user perspective is vital in developing evidence based practice Traditional approaches can ignore issues around acceptability of treatment New policies and practices can be made on the basis of an inadequate evidence base Research priorities and perspectives of service users can be different to those of traditional researchers Consumers Perspectives on ECT (Rose et al, 2003): professional papers reported much higher satisfaction with ECT than user reports. Use of a different methodology: participatory research Involves service users in the whole process and assumes they are experts Reduces the power relations between researcher and researched Allows participants to feel more relaxed and give honest and open responses Develops different outcomes and measures

4. Outline The development of a service user generated measure of perceptions of acute psychiatric care. The major issues raised by service users, which were included in the measure

5. Phases of Measure Development Sampling and Recruitment Developing a Topic Guide Data Collection Analysis and Measure Development Psychometric Testing

6. Sample and Recruitment Purposive Sample:Service users admitted to hospital in the previous 2 years. Qualitative Phase: Local MIND day centre and community mental health teams Quantitative Phase:Acute and forensic wards and intensive care units. Demographics: Qualitative Phase (n=37)Quantitative Phase (n=185) Gender43% male 57% female 62% male 38% female. Age20-66 years Median 45 years 19-76 Median 37 years Ethnicity49% White 51% Black and Minority Ethnic Groups 51% White 48% Black and Minority Ethnic Groups 1% Not Known Diagnosis51% Schizophrenia/Schizo-Affective Disorder 49% Other 41% Schizophrenia/Schizo-Affective Disorder 59% Other

7. Developing a Topic Guide Literature Review Comprehensive literature search produced suggested themes Reference Group Service users and representatives from mental health organisations discussed and expanded on the draft topic guide Pilot Study Tested the topic guide and focus group process The session was audio-taped, transcribed and the data analysed The topic guide was amended and used as the basis for discussion in the focus groups

8. Data Collection 4 sets of focus groups of service users met to discuss their experiences of acute care Held at non-NHS sites to enable fuller disclosure Facilitated by 2 service user researchers Each session was audio-taped and transcribed Each group met twice for the purpose of respondent validation At the first group, the topic guide was used to prompt discussion The data was subject to a thematic analysis and the main issues were fed back to the second group 90% of participants came back to the second group

9. Measure Generation Item Generation The whole data was subject to a second thematic analysis 22 items were generated in the form of brief statements, with a 6 point Likert Scale response format I find it easy to keep in contact with family and friends when Im on the ward. Refining the Measure 2 expert panels and the original reference group were asked for their comments and the measure was adjusted accordingly. Strongly Agree AgreeSlightly Agree Slightly Disagree DisagreeStrongly Disagree

10. Psychometric Testing Feasibility Study 1 (n=40) Completed VOICE and the Service Satisfaction Scale: Residential (Greenfield, Atkinson & Pascoe, 1985) for criterion validity 5-15 minutes to complete VOICE, with minimum support required 97.5 % found it easy to understand and complete Feasibility Study 2 (n=105) 82% found it an appropriate length Only 2% disliked completing the measure and 93.5% did not find the questions distressing Suitable for a wide range of diagnoses Flesch-Kincaid grade level of 4.6 (approx age 10) Test Retest (n=40) Completed VOICE on 2 occasions to assess the stability of the measure over time Factor Analysis (n=210) Will be conducted using the baseline data from the randomised controlled trial

11. Validity and Reliability Content and During the focus groups, service users identified a wide range of Face Validity: domains which they considered most important. Expert panels confirmed that the measure reflected their own experiences and views Criterion Validity:There is a strong, positive association between VOICE and SSS:RES (Pearsons correlation coefficient, r =.881) Internal Reliability: One item was removed from the scale, leaving a Cronbachs Alpha of.938. This indicates that the items are measuring the same underlying construct Test Retest Reliability: Lins concordance coefficient: total score, rho = 0.906 and mean score, rho = 0.886. This shows the measure has strong test retest reliability Inter-Rater Reliability: Between 97 and 99% agreement

12. Main Issues in the Measure Admission (1) Care and Treatment (2) Medication (3) Staffing (5) Therapy and Activities (3) Environment (5) Diversity (2)

13. Admission Fear and lack of understanding I was plonked down in a chair and I was left there for nearly an hour, not knowing what was going on and watching the patients, you know, both aggressive and very, very withdrawn. You know, it literally frightened the living daylights out of me because I was there for nearly an hour, without anyone coming up to me. When somebody is brought with a section, its like a big war. Youre trying to show them that youre sane. Youre trying to show them that youre … capable of doing what you want to do for yourself without hurting anybody. And theyre there trying to tell you … well boy, we dont believe and youve got to prove it to us by, by succumbing and … giving into whatever we want. And thats how it feels. I feel that the system … is making more schizophrenics violent than if they werent getting the treatment. I think that the system actually makes a lot of … the violence and the anger and the … sort of war against, from users … or survivors... against the authorities. Qu: I was made to feel welcome when I arrived on this ward

14. Care, Treatment and Medication Little involvement in care They keep saying that youve got the opportunity to but they never actually give, physically give you the opportunity to do anything about your care. They dont have, it comes back to the time, this has been said. Ward rounds five, ten minutes at the most and then youre kicked out so you cant really get into a discussion about your care in that short a period of time. They wouldnt take the trouble to come and sit, just take one of the staff to sit in your bedroom with you and maybe talk over whats making you feel like that. Yeah rather than do that, they just sit back and just let you go over the top and then they pound on you with the needle. Qu: I have a say in my care and treatment Qu: Staff give me medication instead of talking to me

15. Staffing Interaction and communication A lot of the time you had to go to them to say I need to talk and theyd say five minutes and half an hour later youre getting worse and worse and worse. Youre spiralling down, you need that person to talk to and they finally decided to turn up. They think because youre mentally disturbed, you dont deserve like to be treated like a, youre treated like a subnormal human being because youve got the disturbance. When I was feeling down, he picked me up you know. He told me the good things about myself, the positive things about myself you know. Qu: Staff are available to talk to when I need them

16. Activities and Therapies Little to do, boredom, need for talking therapies I was just absolutely bored out of my nut and found it very difficult to motivate myself to get up, to eat, to do anything just because of the whole boredom issue. Just dumped. Sleep, eat, poo. Thats it. The situation where that girl … was trying to hang herself. If group therapy were able to listen to her and understand her because, you know, everyones walking around oh shes at it again. Shes trying to self harm. Whatever. But if they sat her down … in a therapeutic room and listened to her, what got her to that stage, then perhaps she wouldve understood, and then give positive thoughts back to her as to how she can deal with it. Qu: I think the activities on the ward meet my needs Qu: I find sharing experiences in a group with other patients helpful

17. Environment Feeling unsafe and unprotected I think acute wards are quite stressful places to be. Because like youve got people with all different kinds of illnesses. You might have someone with depression in with someone with paranoid schizophrenia or really aggressive and the two dont match. I was in with some very dangerous people that kept threatening me and making my life a hell and doing things to me and the staff done absolutely nothing to help me, you know. Absolutely nothing. They were really violent. When I mean violent, they are the kind of people that one minute they are ok, everything is ok, then they just flip. Just minor little things. Qu: I feel safe on the ward

18. Diversity Respect for difference This particular nurse said, do not trouble me, or any of you. This is Sunday, its Bible day, so dont ask me for anything. Ive had another one say to me, you lot are evil because you dont believe in Jesus. Now its fine for people to have their religion. Im not against anybodys religions, but do not use their religions on us because were not there for religion. Were there to get ourselves well. Well I presume most Black Caribbean … they label you dont they, if they cant find exactly what it is. Because all I had was a breakdown and now its gone to … schizophrenia and paranoia, paranoid schizophrenia. So how did they go from a breakdown to that? Qu: I feel able to practice my religion whilst Im in hospital

19. In Summary VOICE is a brief self-report questionnaire measuring service users perceptions of acute care It is directly drawn from the experiences of service users and encompasses the issues that they consider most important It also has strong psychometric properties VOICE is currently being used as the primary service user measure evaluating health and social care interventions in a randomised controlled trial of acute inpatient care

20. VOICE Developing a New Measure of Service User Perceptions of Inpatient Care, using Participatory Research Methods Jo Evans Service User Research Enterprise (SURE) Institute of Psychiatry jo.evans@iop.kcl.ac.uk