Rasch analysis of the Roland-Morris Disability Questionnaire Megan Davidson, PhD School of Physiotherapy, La Trobe University, Melbourne

Questionnaires Functioning Disability Ability Health status Activity limitations Participation restrictions Quality of life Well being Level of assistance Typically have n items summed to give a total score Higher score indicates more or less of the “thing” being measured

The scores are ordinal Rank order Distance between ranks is unknown Distance between adjacent scores are not equivalent units Arguably, ordinal scores cannot be manipulated mathematically

How far will the cars go? 4 cars are filled with petrol to see how far they go on a full tank of fuel. 3 observers are positioned along the route.

Green: Score 3 Blue: Score 2 Red: Score 2 Yellow: Score 1

Rasch analysis Modern Test Theory (Item response) Models the probability that a person of θ ability will be able to do activity of δ difficulty Locates item difficulty and person ability on an interval-level logit scale Logit = log-odds unit probability a person can perform the task divided by the probability they cannot

Advantage of Rasch modelling Measure what is measurable, and make measurable what is not so. (Galileo Galilei) “Rasch… provides an operational criterion for fundamental measurement of the kind found in the physical sciences” David Andrich

Easy activities Hard activities Least able person Most able person Get out of bedHouseworkGardeningSport

Roland-Morris Questionnaire (RDQ) A low-back specific disability questionnaire 24-items from the Sickness Impact Profile Patient self-completed Tick those that apply “today” Number of items selected = score Possible score 0-24 Higher score indicates greater disability

Roland-Morris Disability Questionnaire 1. I stay at home most of the time because of my back 2. I change position frequently to try and get my back comfortable 3. I walk more slowly than usual because of my back 4. Because of my back, I am not doing any of the jobs that I usually do around the house

RDQ content housework self-care walking sleeping sitting irritability appetite pain

Short-form versions RDQ 18-item version Stratford & Binkley item version Williams & Myers 2001

Classical (Traditional) Test Theory Reject/retain items on some basis Very low or high response frequency Very low or high item-item correlations Low or high corrected item-total correlations Cronbach’s alpha in range considered desirable

18-item versions Stratford & Binkley Response frequency 90% Item-item correlations > 0.75 Item-total correlations <.40 Increased Cronbach’s alpha Williams & Myer Response frequency 80% Item-item correlations > 0.75 Item-total correlations <.20 Cronbach’s alpha >.80

Items removed in 18-item versions Stratford & Binkley 2 change position 15 appetite not good 17 walk short distance 19 dress with help 20 sit most of day 24 stay in bed Williams & Myers 2 change position 15 appetite not good 19 dress with help 20 sit most of day 22 more irritable 24 stay in bed

Aim: To examine fit to a Rasch model of the 24- item and two 18-item versions of the RDQ To explore whether decisions to reject items on the basis of Rasch analysis would differ from that made by the developers of two 18- item versions of the RDQ.

Is RDQ unidimensional? Items drawn from several SIP domains Williams & Myers 2001 Many low item-item and some low item-total correlations 4 factors explaining 55% of total variance

Method Data for 140 people from a previous study Battery of questionnaires including RDQ Participants were seeking physiotherapy treatment for a low back problem aged 18 years or older read and write English. Recruited from public hospitals, community health centres and private practices. RUMM2020 Rasch analysis software

Results (n = 140) Mean age 51 years (sd 17, range 18-89) 66% female 41% employed 43% pain < 6 weeks 34% pain > 6 months 70% pain that referred into the buttock or leg. RDQ score 9 (sd 5.6), median 8 (IQR 5-14).

Item-Trait Interaction Item Fit Residual Item Fit  2 and F-stat PSI 24-item RDQ (-2.47) 17 (-2.28) 22 (2.19)  2 p all >.01 F 17 p = item Stratford (-2.41)  2 p all >.01 F 9 p = item Williams (-2.18) 17 (-2.15) 18 (2.22)  2 p all >.01 F 17 p = Poor fit if item-trait p ±2,  2 p <.01, F p <.01 PSI = Person Separation Index COMPARISON OF FIT TO THE RASCH MODEL

Differential Item Functioning (DIF) An item may attract systematically different responses on the basis of some characteristic other than item difficulty Age Gender DIF by age and gender in all versions Item 5 Because of my back, I use a handrail to get upstairs

Which items would Rasch reject? 17 walk short distances 9 dress slowly Negative residuals indicate redundancy 5 use handrail upstairs DIF by age/gender 16 trouble putting on socks DIF by age, another item at same location on logit scale

20-items fit the Rasch model Item-Trait interaction Total item chi square >.05: p =.424 Item Fit Item residuals all < ±2.0 Item Chi Square and F-stat all p >.01 No DIF by age and gender PSI = 0.83

Items removed Stratford & Binkley version: 2,15,17,19,20,24 Williams & Myers version: 2,15,19,20,22,24 Rasch 21-item version: 5,9,16,17

24-item RDQ: Targetting Increasing item difficulty Increasing person ability Decreasing item difficulty Decreasing person ability 19 dress with help avoid heavy jobs -2.35

24-item RDQ: Targetting Increasing item difficulty Increasing person ability Decreasing item difficulty Decreasing person ability gap cluster

24-item RDQ 18-item RDQ Stratford 17 Walk short distances 2 Change position frequently 19 Dress with help 24 Stay in bed 15 Appetite not good 20 Sit most of the day

24-item RDQ 18-item RDQ Williams

24-item RDQ 20-item Rasch selection

Raw Scores Vs Rasch Measure Change of 5 points from 10 to 5 = 1.16 logits Change of 5 points from 5 to 0 = 2.58 logits

Conclusions Traditional and Modern Test Theory approaches reject different items Rejecting items of very low/high frequency results in truncated scale RDQ can be made to fit Rasch model, but targetting is poor Gaps in item difficulty locations No items of sufficient difficulty for high ability persons

Rasch Group Meeting Swinburne Hawthorn Monday 5 th Dec, 4.00pm Room AR103 Graduate School of Research (next door to Haddon’s coffee shop in campus centre) Megan Davidson Julie Pallant