T10 OUTCOME ASSESSMENT Why, what and how? Dr. Frederike van Wijck & John Dennis

Learning Outcome Plan a safe, effective and appropriate intervention, i.e.: Design and adapt appropriate progressive physical activity programme(s) after stroke using findings from the physical/ exercise assessments, etc… Demonstrate competency in relevant assessment procedures: Monitor clients’ progress against agreed goals

Content 1.Measuring outcomes: why? 2.Measurement: general principles 3.Measuring outcomes: what? –General framework: the ICF –Specific suggestions for the exercise-after- stroke setting 4.Using outcome measures in an exercise after stroke setting: how?

Outcome measure – a definition: “ a test or scale administered by therapists that has been shown to measure accurately a particular attribute of interest to patients and therapists and is expected to be influenced by the intervention” (Mayo, 1995)

1.Measuring outcomes: why? 1.Screening: testing eligibility for exercise 2.Baseline assessment: establishing starting point for exercise programme 3.Follow-up assessment: charting change following exercise 4.Monitoring: to chart adherence and identify adverse effects This session: baseline and follow-up assessment using outcome measures

1. Measuring outcomes: why not? Common reasons for not using outcome measures: It takes time away from the actual exercise It is a burden for participants It’s complex and a hassle for the instructors You need training – we don’t have time for that What do these measures tell you anyway – I know if something works! Do you??!

SCIENCE VS. COMMON SENSE Science: “knowledge, ascertained by observation and experiment, critically tested, systematised and brought under general principles” Cambridge English Dictionary Common sense: “normal understanding, good practical sense in every day affairs, general feeling (of mankind or community)” Oxford English Dictionary

Common sense is not good enough for exercise instructors/ health care professionals; Exercise/ rehabilitation/ health care needs to be based on science!

Content 1.Measuring outcomes: why? 2.Measurement: general principles 3.Measuring outcomes: what? –General framework: the ICF –Specific suggestions for the exercise-after- stroke setting 4.Using outcome measures in an exercise after stroke setting: how?

2. Measurement: general principles Characteristics of good outcome measures: 1.Relevant 2.Valid 3.Reliable 4.Sensitive to change 5.Practicable 6.Results can be easily communicated (Wade, 1992)

Characteristics of good outcome measures Relevance: the pertinence of the information Consider: Is this information useful – what does it tell me? What am I going to do with the information?

Characteristics of good outcome measures Validity: the measure does what it is purported to do Consider: Which idea/ construct does this measure address?

Characteristics of good outcome measures Reliability: the measure gives the same result each time the same quantity is measured. Consider: –Intra-rater variation –Inter-rater variation -> Importance of protocols! (tutorial)

Characteristics of good outcome measures Sensitivity to change: the measure can detect changes that are relevant Consider: On what scale is/ are the item(s) scored? E.g.: –0/ 1 or Yes/ No –0-10 (Visual Analogue Scale) Floor and ceiling effects

Characteristics of good outcome measures Practicability: the measure is quick and easy to use Consider: Amount of information required Duration of the process Complexity of the process Burden on client (and you!)

Characteristics of good outcome measures Communicability: The results can easily be reported and understood Consider: Amount Format Standardisation of the information

Summary - general principles of measurement: Characteristics of good outcome measures: 1.Relevant 2.Valid 3.Reliable 4.Sensitive to change 5.Practicable 6.Results can be easily communicated (Wade, 1992)

Content 1.Measuring outcomes: why? 2.Measurement: general principles 3.Measuring outcomes: what? –General framework: the ICF –Specific suggestions for the exercise-after- stroke setting 4.Using outcome measures in an exercise after stroke setting: how?

3. Measuring outcomes: what? A general framework for outcome measurement in clinical practice: the ICF International Classification of Functioning, Disability and Health “Aim of the ICF classification is to provide a standard language and framework for the description of health and health-related states.”

ICF definitions  Impairments are problems in body function or structure such as a significant deviation or loss.  Activity limitations are difficulties an individual may have in executing activities.  Participation restrictions are problems an individual may experience in involvement in life situations.

ICF model

The ICF: A general framework for outcome measurement in rehabilitation Activity Limitations ImpairmentsParticipation Restrictions Health Condition

Outcome measures and the ICF Which outcome measures do you use in your work ? Where do they fit within the ICF? Can you think of one outcome measure in each of the ICF domains for a person who has had a stroke?

Outcome measures for exercise after stroke Activity Limitations Example? Impairments Example? Participation Restrictions Example? Person with stroke

3. Measuring outcomes: what/ how? Suggested outcome measures for exercise after stroke specifically : 6 min. walk/ 10 m. walk Timed up and Go Visual Analogue Scale (VAS) Stroke Impact Scale + Register: for monitoring adherence

Onto: Measuring Outcomes: how? 6 min. walk/ 10 m walk VAS Timed up and Go Stroke Impact Scale

6-minute walk test Construct: maximum walking distance in 6 minutes Relevance: functional test for exercise endurance, O 2 uptake Validity: good Reliability: high Sensitivity: ? Practicability: good Reporting: easy (distance (m)) Normative data for healthy people aged yrs: m (Steffen et al., 2002)

Timed Up and Go Construct: time to stand up from arm chair, walk 3 m, turn, walk back, sit down Relevance: functional test for basic mobility for frail elderly in community Validity: acceptable Reliability: moderate - high Sensitivity: ? Practicability: good Reporting: easy (time (s)) Normative data for healthy people aged yrs: 7-12 s (Steffen et al., 2002)

VAS Construct: person’s perception of a particular construct Relevance: depending on what is assessed. Can be used to assess individual goal attainment Validity: generally good Reliability: generally high Sensitivity: high Practicability: caution with stroke, esp. higher cortical problems and neglect (Price et al., 1999) Reporting: easy

Correct use of VAS in stroke Price et al. (1999), p

Correct use of VAS in stroke Price et al. (1999), p

Stroke Impact Scale Construct: the person’s perceived impact of stroke across range of domains (incl. movement, ADL, cognition, communication, emotion, participation) Relevance: high Validity: good (devised with target population) Reliability: moderate – very high Sensitivity: each item on 5-point scale + one VAS item Practicability: mixed Reporting: time-consuming but can be done by mail Normative data: not applicable Interpretation: change between points clinically meaningful (Duncan et al., 2003)

Pitfalls of measurement – and how to fix them Problem Error: –Systematic –Random Wrong signals: –False + –False - Solution?

Pitfalls of measurement – and how to fix them Problem Error: –Systematic –Random Wrong signals: –False + –False - Solutions: Errors: –Calibrate your instrument –Use standardised protocol Wrong signals: –Check sensitivity –Verify with other information

Pitfalls of measurement – and how to fix them Problem Error: –Systematic –Random Wrong signals: –False + –False - Solutions: Errors: –Calibrate your instrument –Use standardised protocol Wrong signals: –Check sensitivity –Verify with other information Errors are inherent in any form of measurement! Always be aware and try to reduce.

Summary Outcome measures are necessary to: Establish baseline for exercise Evaluate change following exercise -> science underpinning your work EVIDENCE BASED PRACTICE

Choosing your Outcome Measure how to go about it Safe? NO YES Relevant? NO YES Science Robust? NO YES Practicable? NO YES GO THINKAGAINTHINKAGAIN Safe? NO YES Relevant? NO YES Science Robust? NO YES Practicable? NO YES GO T

References DUNCAN, P. W., LAI, S. M., BODE, R. K., PERERA, S. & DEROSA, J. (2003b) Stroke Impact Scale-16: A brief assessment of physical function. Neurology, 60, DUNCAN, P. W. Stroke Impact Scale (SIS). Rehabilitation Outcomes Research Centre, US Department of Veteran Affairs. Available from: (last accessed 05/12/05). WADE, D. T. (1992) Measurement in Neurological Rehabilitation, Oxford, Oxford University Press. WORLD HEALTH ORGANISATION (2001). International Classification of Functioning, Disability and Health. Available from

OM’s Applied What outcome measures would be appropriate for your case study?