1. Outcome Measurements Study Design Liz Lingard BPhty, MPhil, MPH Research Coordinator Department of Orthopaedics Freeman Hospital

2. Orthopaedic Clinical Research Traditionally focused on reporting clinical and radiographic results Traditionally focused on reporting clinical and radiographic results Rates of complications following procedures Rates of complications following procedures Survivorship analysis of the life of a prosthesis Survivorship analysis of the life of a prosthesis Many studies are observational Many studies are observational

3. Need For Outcome Measures Technological advances, epidemiologic and demographic trends and rising public demand had rapidly increased health care expenditure over the past three decades Technological advances, epidemiologic and demographic trends and rising public demand had rapidly increased health care expenditure over the past three decades As health care resources are limited they need to be allocated according to clinical effectiveness and cost-effectiveness As health care resources are limited they need to be allocated according to clinical effectiveness and cost-effectiveness

4. Orthopaedic Interventions Surgical treatment for one or more of the following musculoskeletal problems Surgical treatment for one or more of the following musculoskeletal problems  Pain  Loss of range of motion  Instability  Functional limitation  Deformity  Trauma  Tumour

5. International Classification of Functioning, Disability and Health (ICF) Functioning and Disability Functioning and Disability  Body function and structures  Activities and participation Contextual Factors Contextual Factors  Environmental factors  Personal factors

6. Interaction Between ICF Components

7. Uses of Outcome Measures Clinical research Clinical research Audit and quality assurance Audit and quality assurance Assess health care needs of populations Assess health care needs of populations Assist in individual patient care Assist in individual patient care

8. Outcome Measures Assessment of disease or injury severity Assessment of disease or injury severity Measure change over time due to: Measure change over time due to:  Natural history  Intervention Supplement mortality, clinical, radiological and laboratory measures Supplement mortality, clinical, radiological and laboratory measures Outcome instruments Outcome instruments  Observer administered  Patient administered

9. Observer Administered Functional Tests Functional Tests  Timed Up & Go  6 minute walk  Activity Monitoring Global rating scores Global rating scores  Knee Society Score  Harris Hip Score  Foot and Ankle Score

10. ActivPal Monitor Ambulatory activity monitor worn for 7 days Activity Monitoring

11. matched walking periods Subject A: 72 year old THR patientSubject B: 65 year old spouse

12. Activity Summary

13. Stepping Summary

14. Global Rating Scores May combine measures of pain, joint range of motion, stability, muscle power, deformity, function and even complications and investigations May combine measures of pain, joint range of motion, stability, muscle power, deformity, function and even complications and investigations Weight of scoring of each item varies Weight of scoring of each item varies Inter / intra-rater reliability & observer bias Inter / intra-rater reliability & observer bias

15. Pitfalls of global scores Different scoring systems assign different weights to parts of the score and the summary results are often inconsistent giving contrary measures of success in the same patient. (Bryant et al. JBJS-Br 1993) Different scoring systems assign different weights to parts of the score and the summary results are often inconsistent giving contrary measures of success in the same patient. (Bryant et al. JBJS-Br 1993)

16. Patient Administered Report summary scores for different health dimensions Report summary scores for different health dimensions  Physical Function  Symptoms – pain  Global judgements of health  Psychological well-being  Social well-being  Cognitive functioning  Role activities  Personal constructs  Satisfaction

17. Patient Administered Disease-specific (WOMAC, AIMS) Disease-specific (WOMAC, AIMS) Site or region-specific (KOOS, DASH, ODI) Site or region-specific (KOOS, DASH, ODI) Dimension-specific (McGill Pain Questionnaire) Dimension-specific (McGill Pain Questionnaire) General health status (SF-36, NHP) General health status (SF-36, NHP) Health state utility (EQ-5D) Health state utility (EQ-5D) Satisfaction (Management / Outcome) Satisfaction (Management / Outcome)

18. WOMAC Western Ontario & McMaster University OA index Western Ontario & McMaster University OA index Tri-dimensional scale (Likert and VAS versions): Tri-dimensional scale (Likert and VAS versions):  Pain (5 items)  Stiffness (2 items)  Function (17 items) Validated on patients with OA of hip and knee Validated on patients with OA of hip and knee  Sensitive and responsive for detecting change over time due to NSAIDs and joint arthroplasty Translated into over 50 different languages Translated into over 50 different languages

19. WOMAC Responses for each item are none, mild, moderate, severe or extreme (pain, stiffness or difficulty) Responses for each item are none, mild, moderate, severe or extreme (pain, stiffness or difficulty) Scores transformed to 0-100 (100 best) Scores transformed to 0-100 (100 best)  100 = none  75 = mild  50 = moderate  25 = severe  0 = extreme

20. Preoperative WOMAC None Mild Moderate Severe Extreme Note: Data on Primary Joints – 1,061 TKA, 760 THA & 174 BHR

21. 1-Year WOMAC None Mild Moderate Severe Extreme Note: Data on Primary Joints - 490 TKA, 387 THA & 75 BHR

22. Developed and validated by Ewa Roos Developed and validated by Ewa Roos Meniscus injury, ACL injury and/or OA Meniscus injury, ACL injury and/or OA Adds dimensions to the WOMAC scale: Adds dimensions to the WOMAC scale:  Sport and recreation function  Knee-related quality of life Improves sensitivity of evaluation of younger or more physically active persons Improves sensitivity of evaluation of younger or more physically active persons Knee injury and Osteoarthritis Outcome Score (KOOS)

23. Squatting Squatting Running Running Jumping Jumping Twisting/pivoting Twisting/pivoting Kneeling Kneeling KOOS – Sports/Recreation

24. How often are you aware of your knee problems? How often are you aware of your knee problems? Have you modified your lifestyle to avoid potentially damaging activities to your knee? Have you modified your lifestyle to avoid potentially damaging activities to your knee? How much are you troubled with lack of confidence in your knee? How much are you troubled with lack of confidence in your knee? In general, how much difficulty do you have with your knee? In general, how much difficulty do you have with your knee? KOOS – Knee Related QOL

25. KOOS Profile Roos EM et al, Osteoarthritis & Cartilage 1999; 7:216-221 WOMAC KOOS

26. KOOS Profile Roos EM et al, JOSPT 1998; 78:88-96

27. WOMAC Pain and Function None Mild Moderate Severe Extreme Patients at 10 years after primary TKR

28. How often are you aware of your knee problem? Patients at 10 years after primary TKR

29. In general, how much difficulty do you have with your knee? Patients 10 years after primary TKR

30. Short-Form 36 (SF-36) Reports on 8 health dimensions Reports on 8 health dimensions  Physical Functioning  Bodily Pain  Role Physical  General Health Perception  Mental Health  Role Emotional  Vitality  Social Functioning Each subscale given a 0-100 score (100 best) Each subscale given a 0-100 score (100 best) Physical component score Mental component score

31. SF-36 – Primary TKA Note: Data on 1,061 at preop, 490 1-year & 243 at 2-years

32. SF-36 – Primary THA Note: Data on 760 at preop, 387 at 1-year & 186 at 2-years

33. SF-36 – Primary BHR Note: Data on 174 at preop, 75 at 1-year & 35 at 2-years

34. Satisfaction with outcome Asked on follow-up questionnaires Asked on follow-up questionnaires Patient rates how satisfied they are with the results of surgery: Patient rates how satisfied they are with the results of surgery:  Overall  Pain relief  Ability to do ADL  Ability to do recreational activities Responses are very satisfied, somewhat satisfied, somewhat dissatisfied and very dissatisfied Responses are very satisfied, somewhat satisfied, somewhat dissatisfied and very dissatisfied

35. Satisfaction – Primary TKA Note: Data on 490 primary TKA patients at 1-Year

36. Satisfaction – Primary THA Note: Data on 388 primary THA patients at 1-Year

37. ICF Quiz

38. Choosing an Outcome Measure Formulate a precise study question Formulate a precise study question Know your patient population Know your patient population Know what independent variables may affect the outcome Know what independent variables may affect the outcome Commonly used outcome measure does not necessarily mean it is the best measure to use for your study Commonly used outcome measure does not necessarily mean it is the best measure to use for your study

39. Independent Variable Examples Socio-demographics Socio-demographics Body mass index Body mass index Comorbid medical conditions Comorbid medical conditions Comorbid musculoskeletal conditions Comorbid musculoskeletal conditions Mental health status Mental health status Common geriatric problems Common geriatric problems

40. Appropriateness Disease / injury to be studied Disease / injury to be studied Patient population Patient population Intervention Intervention Dimension of health likely to be affected Dimension of health likely to be affected Acceptability Acceptability Feasibility Feasibility

41. Acceptability Minimise burden to patients: Minimise burden to patients:  Avoid distress to patients  Reduce refusal rate  Reduce incomplete data Need to consider: Need to consider:  Literacy of patients  Legibility and layout of questionnaire  Time taken to complete

42. Feasibility Impact on clinical staff / researchers Impact on clinical staff / researchers  Staff attitudes are key determinant of data collection Data collection Data collection  Brief questionnaires  Lengthy questionnaires with interviews  Postal / telephone Coding and entering data Coding and entering data Analysis of data Analysis of data

43. Validity Measure what it says it does Measure what it says it does Content – appropriate range of items Content – appropriate range of items Face – measures what they aim to measure Face – measures what they aim to measure Construct – correlates to other measures Construct – correlates to other measures Criterion – correlates to a ‘gold standard’ Criterion – correlates to a ‘gold standard’

44. Reliability Measures accurately Measures accurately Internal consistency Internal consistency Test – re-test Test – re-test Inter-rater Inter-rater Intra-rater Intra-rater

45. Responsiveness Sensitive to detecting changes over time that are important to the patient Sensitive to detecting changes over time that are important to the patient Correlates with changes in disease severity Correlates with changes in disease severity Effect size index = Effect size index = ( (mean 1 – mean 2 )/ pooled SD Standardised response mean = Standardised response mean = (SD of change in score (mean 1 – mean 2 )/ SD of change in score

46. Precision Ceiling and floor effects Ceiling and floor effects Same amount of change across different parts of the scale may not be equal Same amount of change across different parts of the scale may not be equal Instruments that make more distinctions are not more precise Instruments that make more distinctions are not more precise

47. Interpretation Outcome measures produce numeric scores Outcome measures produce numeric scores Score ranges need to be clear Score ranges need to be clear Need to know what is the smallest clinical meaningful changes that are perceptible to a patient for that particular instrument Need to know what is the smallest clinical meaningful changes that are perceptible to a patient for that particular instrument

48. Study Design

49. Retrospective Retrospective  Deliberate intervention  Observational studies Prospective Prospective  Deliberate intervention  Observational studies

50. Study Design Cross-sectional studies Cross-sectional studies  Disease description  Diagnosis and staging  Abnormal ranges  Disease severity  Disease process

51. Study Design Longitudinal studies Longitudinal studies  Prospective  Randomised  Non-randomised

52. Randomised vs. non-randomised RCT is seen as the ‘gold standard’ RCT is seen as the ‘gold standard’  Should ensures patients differ only by their exposure to the intervention  Often difficult to generalise results due to inclusion/exclusion criteria  Blinded trials in orthopaedics almost impossible introducing observer/patient bias Non-randomised trials Non-randomised trials  Cohort and case-controlled studies  More inclusive but may have selection bias

53. Study design For studies that compare health care interventions to be valid they must produce results that are: For studies that compare health care interventions to be valid they must produce results that are:  Generalisable to to other potential patients (high external validity)  Estimate outcome effects that can be reliably attributed to the intervention (high internal validity)

54. Solutions to threats of validity Internal validity Internal validity  Allocation bias (confounding) – risk adjustment and subgroup analysis  Patient preference – preference arms or adjustment for preference External validity External validity  Exclusions – expand inclusion criteria  Non-participation – multi-centre, pragmatic design  Non-participation – less rigorous consent

55. Potential Problems Recruitment Recruitment  Need to know what proportion of the reference population have been recruited  Reasons for non-participation – does this group of patients differ from participants. Knowing baseline details about this group strengthens your study  Volunteerism likely to be associated with age, gender, socioeconomic status, education etc. which may influence outcomes

56. Potential Problems Compliance Compliance  Depends on how well informed patients are at start of trial of what is expected  If intervention is ongoing (e.g. physical vs. surgical intervention) then poor compliance with intervention can affect statistical power  If patients are non-compliant with parts of study this is preferable to total non-compliance  Need to track all patients and document reasons for lost to follow-up

57. Statistical Support Speak to a statistician Speak to a statistician  After you have reviewed the literature, selected your primary outcome measure and drafted a protocol  Sample size calculation  Method of data collection  Advice about coding of data  Analysis of data BUT  Before you submit to either R&D or LREC