Therapy Outcome Measure Workshop Pam Enderby Alex John p.m.enderby@sheffield.ac.uk

Therapy Outcome Measures Background Context Purpose Use of the tool

Data to support quality assurance 3 Donabedian (1980) Process Structure Outcome Staff grades, costs of assessment tools, quality of accommodation No. of interventions provided; no of patients seen Changes in patients communication, wellbeing Clearly when we are trying to assess the value of a service and quality assure, we need to use all kinds of information to analyse and understand services. Donabedian (1980) was one of the first – he talked about structure process and outcome

Outcomes ‘address the effects, not the process, of particular interventions’ (Hesketh & Sage, 1999) “ results or visible effects of interventions…. forms part of the quality cycle….. provides information on the impact of interventions…. identifies the effectiveness of practices….” (Enderby, John & Petheram, 2006) What all these have in common, is the understanding of what we mean by outcomes:

How can we improve our therapy? Reflection Learning from research Learning from others Learning from experience Data collection

Why has outcome measurement got to the top of the agenda? Increasing demands on the health service Knowledge of variation in provision Financial constraints

NHS Atlas of variation February 2016 NHS patients are suffering from "unwarranted variation" in their care, which cannot be linked to levels of illness or patient-preference, a joint report from leading health bodies shows. “Our challenge now is to consider how we can better understand and tackle the underlying causes. This is not a straightforward task, but exploring the data that lies behind these variations will be an important starting point,” Chief Medical Officer

National Director for Commissioning Development emphasises the importance of good governance for CCGs. ' Where CCGs wish to make changes to their commissioning support arrangements, it is critical that the rationale behind these decisions is transparent and properly documented.’

Commissioning Guidance for Rehabilitation 2016 Good rehabilitation services will: 1.Optimise physical, mental and social wellbeing and have a close working partnership with people to support their needs 2.Recognise people and those who are important to them, including carers, as a critical part of the interdisciplinary team 3. Instil hope, support ambition and balance risk to maximise outcome and independence 4.Use an individualised, goal-based approach, informed by evidence and best practice which focuses on people’s role in society 5. Require early and ongoing assessment and identification of rehabilitation needs to support timely planning and interventions to improve outcomes and ensure seamless transition

Commissioning Guidance for Rehabilitation 2016 Good rehabilitation services will: 6.Support self-management through education and information to maintain health and wellbeing to achieve maximum potential 7.Make use of a wide variety of new and established interventions to improve outcomes e.g. exercise, technology, Cognitive Behavioural Therapy 8. Deliver efficient and effective rehabilitation using integrated multi-agency pathways including, where appropriate, seven days a week 9.Have strong leadership and accountability at all levels – with effective communication 10.Share good practice, collect data and contribute to the evidence base by undertaking evaluation/audit/research

Commissioning Guidance for Rehabilitation 2016 Good rehabilitation services will: 6.Support self-management through education and information to maintain health and wellbeing to achieve maximum potential 7.Make use of a wide variety of new and established interventions to improve outcomes e.g. exercise, technology, Cognitive Behavioural Therapy 8. Deliver efficient and effective rehabilitation using integrated multi-agency pathways including, where appropriate, seven days a week 9.Have strong leadership and accountability at all levels – with effective communication 10.Share good practice, collect data and contribute to the evidence base by undertaking evaluation/audit/research

will it enable benchmarking with other services? Commissioning Guidance for Rehabilitation When considering what outcome data to request from providers, the following should be considered: ‘what outcome data is already collected locally (by the team managers and clinicians)? what outcome measurement tools are appropriate for the client group, health condition and method of service delivery? will it enable benchmarking with other services? will it show how existing inequalities have been reduced in terms of access to services, experiences of services and outcomes achieved?’

Context ‘ ‘Funding constraints mean that double running costs are no longer viable. The need to close the spending gap in public services means that priority-setting involves difficult decisions and funding choices.’

How can outcome measurement help you with your service? Examine changes over time Investigate particular issues e.g. intensity of therapy Identifying areas of strength Identifying areas of weakness Communicate with the client, other professionals and commissioners

Collecting data on patient experience is not enough: they must be used to improve care Coulter A., Locock L., Ziebland S., Calabrese J. BMJ 2014; 348 :g2225

Rehabilitation/Enablement The process of trying to help people who have suffered some injury/disease or developmental delay to maximise psychological well being, functional ability and social integration (Wade, 1992) An often complex process which enables individuals after impairment by illness, developmental delay or injury to regain as far as possible control over their own lives (King’s Fund, 1999)

What For? Impairment/disorder reduction Improved Function Psycho social gain Wellbeing Disorders have become more complex

Measures of Performance Health Gain Social Gain

Therapy treatment goals to identify and reduce the disorder/ dysfunction to improve or maintain the function and ability to assist to achieve potential or integration and to alleviate anxiety or frustration.

It’s what it says on the tin!

Choosing an Outcome Measure Relevance Validity Reliability Other considerations Ease of use Communication

Different outcome measures Patient Reported Outcome Measures Goal Attainment Scales Clinical Assessments Generic measures e.g. SF36, Teller, COPM etc Therapy Outcome Measure Patient Experience Measures---- not an ‘outcome measure’!

The Therapy Outcome Measure Note

Therapy Outcome Measures for Rehabilitation Professionals Pamela Enderby and Alexandra John (2015) ISBN 978-1-907826-29-0 PUBLISHED BY J&R  

“So when do we do it?”

Referral /case history/ assessment Aim/Goal Intervention End of episode of care

Now for something different!

We have added wellbeing as a domain What is the International Classification of Function, Disability and Health (ICF)? An international classification of function and disability and its effects on the individual: Classifies body structure and function Classifies activity/independence Classifies social participation Classifies how the environment impacts upon the disabled individual We have added wellbeing as a domain

Therapy Outcome Measures The Dimensions

Disease/Disorder/Developmental Delay Impairment Disease/Disorder/Developmental Delay Physical Mental/ cognitive An injury, illness, or congenital condition that causes or is likely to cause a loss or difference of physiological or psychological function as compared to those without such.

Impairments are problems of body function or structure as a significant deviation or loss A deviation from accepted population standards Can be temporary or permanent Slight or severe or vary over time Not dependent upon the presence of disease Broader than disease or disorder Impairments may result from other impairment

Activity Limitation Abilities Person Difficulties an individual may have in the performance of activities/level of independence.

Activity is the execution of a task or action by an individual Occurs at the level of the individual Is essential Composite of various functions Covers full range of life areas Examples: Maintaining a body position Reaching and grasping Walking Self care Communicating

Participation Disadvantages Circumstances Society Disadvantages an individual may have in the manner or extent of involvement in life situations.

Participation is involvement in life situations Occurs at the social level Covers the full range of life areas Limitation may not occur as a result of impairment Is culturally defined Examples: Shopping Voting Attending school Working Being part of a relationship Being a carer

Achieving potential in social situations Confidence in social settings Autonomy Integration Controlling life Social activity Participation Life role Achieving potential in social situations Confidence in social settings Involvement in life situations Sustaining work role Interpersonal interactions Community life Tasks and Actions for living life Restrictions on everyday life Self-esteem Life situations

Well-being Upset Feelings Satisfaction Degree of upset, distress, or satisfaction with status Incorporates frequency and severity

Concern Frustration Anger Embarrassment Withdrawal Apathy Emotional control Emotional expression Distress Well-being Satisfaction Mood Depression Emotional detachment Happiness Sadness

Carer Well-being Upset Feelings Satisfaction Emotional effect resulting in an upset, distress, or satisfaction with status

THERAPY OUTCOME MEASURE (TOM) Based on: World Health Organisation Classification - ICF 11 point ordinal scale with 6 defined points

The Core Scale

Impairment ----TOM 0 The most severe presentation of this impairment .5 1 Severe presentation of this impairment 2 Severe/moderate presentation 3 Moderate presentation 4 Just below normal/mild presentation 5 No impairment

Activity--- TOM 0 Totally dependant/unable to function .5 1 Assists/co-operates but burden of task/achievement falls on professional carer 2 Can undertake some part of task /needs a high level of support to complete 3 Can undertake task/function in familiar situation but required some verbal/physical assistance 4 Requires some minor assistance occasionally/or extra time to complete task 5 Independent/able to function

Participation TOM 0 No autonomy, isolated, no social/family role .5 1 Very limited choices, contact mainly with professionals, no social / family role, little control over life 2 Some integration, value and autonomy in one setting 3 Integrated, valued and autonomous in limited number of settings 4 Occasionally some restriction in autonomy, integration, or role 5 Integrated, valued, occupies appropriate role

Well-being TOM 0 Severe constant: High and constant levels of distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy, unable to express or control emotions appropriately. 1 Frequently severe: Moderate distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy. Becomes concerned easily, requires constant reassurance/support, needs clear/ tight limits and structure, loses emotional control easily. 2 Moderate consistent: Distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy in unfamiliar situations, frequent emotional encouragement and support required. 3 Moderate frequent: Distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy. Controls emotions with assistance, emotionally dependant on some occasions, vulnerable to change in routine, etc., spontaneously uses methods to assist emotional control. 4 Mild occasional: Distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy. Able to control feelings in most situations, generally well adjusted/stable (most of the time/most situations), occasional emotional support/encouragement needed. 5 No inappropriate: Distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/ withdrawal/ severe depression/ or apathy. Well adjusted, stable and able to cope emotionally with most situations, good insight, accepts and understands own limitations.

Adapted scales 1. Anorexia Nervosa and Bulimia Nervosa-scale under development 2. Augmentative and Alternative Communication (AAC) 3. Autistic Spectrum Disorder 4. Cardiac Rehabilitation 5. Cerebral Palsy 6. Child Language Impairment 7. Challenging Behaviour and Forensic Mental Health 8. Chronic Pain 9. Cleft Lip and Palate 10. Cognition 11. Complex and Multiple Difficulty 12. Dementia 13. Diabetes 14. Dietetic Intervention for the Prevention of Cardiovascular Disease 15. Dietetic intervention for Enteral Feeding – Paediatrics

Adapted scales 2 16. Dietetic intervention for Home Enteral Feeding – Adult 17. Dietetic intervention for Irritable Bowel Syndrome 18. Dietetic intervention for Obesity – Paediatric 19. Dietetic intervention for Obesity – Adult 20. Dietetic intervention for Undernutrition – Paediatrics 21. Dietetic intervention for Undernutrition – Adults 22. Dysarthria 23. Dysfluency 24. Dysphagia 25. Dysphasia 26. Dysphonia 27. Dyspraxia –Developmental Co-Ordination Difficulties 28. Equipment Services 29. Head Injury 30. Hearing Therapy/ Aural Rehabilitation

Adapted scales 3 32. Laryngectomy 33. Learning Disability – Communication 34. Mental Health 35. Mental Health – Anxiety 36. Multi-Factorial Conditions 37. Musculo-Skeletal 38. Neurological Disorders (Including Progressive Neurological Disorders) 39. Palliative Care 40. Phonological Disorder 41. Podiatric Conditions - scale under development 42. Post Natal Depression 43. Respiratory Care- Chronic Obstructive Pulmonary Disease (COPD) 44. Schizophrenia 45. Stroke 46. Tracheostomy 47. Wound Care

0 = severe, 3 = moderate and 5 = normal---+ 1 /2 points A cross-disciplinary method of gathering information on a broad spectrum of issues associated with therapy/enablement/rehabilitation It allows description of the abilities of a patient/client in four ways: Impairment (problems in body structure or function) Activity (performance of activities) Participation (disadvantages experienced in living) Wellbeing (emotional level of upset or distress) 11 point ordinal scale: 0 = severe, 3 = moderate and 5 = normal---+ 1 /2 points Administered at the beginning and again at the end of episode of care.

Ten programmes of care Patient/client does not need intervention Patient needs prevention/maintenance programme Patient needs convalescence Patient needs slow-stream rehabilitation Patient needs regular rehabilitation programme Patient needs intensive rehabilitation Patient needs specific treatment for individual acute disabling condition Patient needs medical care and rehabilitation Patient needs rehabilitation for complex, profound, disabling condition Palliative Care

Therapy Outcome Measure Ordinal Rating Scale

How many scores? Impairment Impairment 1 Impairment 2 Not impairment 3 (use multifactorial or multiple difficulties)

How many scores? Disability/Activity Activity 1--- is the most usual Participation Participation 1 Well-being Well-Being- patient /client Well-Being—carer (only if involved)

Commissioning Guidance for Rehabilitation 2016 Good rehabilitation focuses on good outcomes that are set by the people we treat and driven by their goals Centres on people’s needs, not their diagnosis Aims high and includes vocational outcomes Is an active and enabling process – not passive care Relies on interdisciplinary team working Responds to changes in people’s needs Integrates specialist and generalist services Requires leadership for transformational change Gives hope

Impairment Participation Wellbeing Activity

Impairment Participation Wellbeing Activity Stephen Hawking

Conducting a Benchmarking Study Internal Benchmarking: Assess own performance External Benchmarking: Assess performance against benchmarking partners First benchmark own service and make changes you want to make. Find benchmarking partners - as close to own service as possible.

Beware of causality