1. Complications with Assessing Disability in Aged Care: When Does “Does Not Apply” Apply? E. Helmes & A. Campbell Department of Psychology James Cook University

2. Disabilities are common in residential care facilities Disabilities are common in residential care facilities Severe levels of disability may affect other domains of function Severe levels of disability may affect other domains of function – Example: impaired mobility limits social interactions

3. Many rating scales and self-report instruments include a neutral, or “Cannot Say” option Many rating scales and self-report instruments include a neutral, or “Cannot Say” option Examples: early MMPI, 16PF (5 th edition), Likert scales with uneven number of options (5-, 7-, or 9-point scales)

4. Responses to such neutral points are ambiguous: Responses to such neutral points are ambiguous: –Neutral? –Indifferent? –Lacks understanding of content? –Lacks knowledge need to answer? –Hostility?

5. What of seemingly more direct “Does Not Apply” or “Not Applicable” options? What of seemingly more direct “Does Not Apply” or “Not Applicable” options? Bristol Activities of Daily Living Scale (Bucks et al., 1996) – all 19 items Bristol Activities of Daily Living Scale (Bucks et al., 1996) – all 19 items MOSES (Helmes, et al., 1987) – 18 of 40 items MOSES (Helmes, et al., 1987) – 18 of 40 items

6. 11. FINDING WAY AROUND INSIDE: (For example, ability to find his room, the washroom, the dining room) How often during the daytime in the past week did the resident become disoriented (confused) in finding his or her way around the inside of the residence? 1. Not at all 2. Seldom (only one to three times during the week) 3. At times (either once or twice a day on more than three days, or several times a day on one to three days) 4. Often (several times a day or on more than three days) 5. Question does not apply ‑‑ the resident never moved around inside the building without assistance from the staff

7. Content Interpretation? Pruchno et al. (1988): Pruchno et al. (1988): 5 of 18 items – inability to speak implies greater levels of disability, so score as 5>4 5 of 18 items – inability to speak implies greater levels of disability, so score as 5>4 11 items equivalent to non-occurrence, so equate with “Not at All”, so score as 5=0 11 items equivalent to non-occurrence, so equate with “Not at All”, so score as 5=0 536 nursing home residents, 24/40 items retained after changed scoring & confirmatory factor analysis 536 nursing home residents, 24/40 items retained after changed scoring & confirmatory factor analysis

8. Samples Norming sample; 2921 unique cases Norming sample; 2921 unique cases –Psychogeriatric – 397 –Nursing home – 918 –Home for the Aged – 563 –Continuing Care – 447 924 (31.6%) males, 1985 (68%) females 924 (31.6%) males, 1985 (68%) females Mean age 78.9 (SD = 10.9) Mean age 78.9 (SD = 10.9) 490 Single, 688 Married, 1588 Widowed, 123 Divorced or Separated 490 Single, 688 Married, 1588 Widowed, 123 Divorced or Separated

9. Scoring Variations All “Does Not Apply” coded as “5” All “Does Not Apply” coded as “5” Pruchno et al. variation Pruchno et al. variation Listwise deletion of any case with a “Does Not Apply” score (as in 1987 components analysis) Listwise deletion of any case with a “Does Not Apply” score (as in 1987 components analysis)

10. Analysis Scoring key as target: 8 items on each of 5 dimensions Scoring key as target: 8 items on each of 5 dimensions 12 covariance matrices (3 scoring variations x 4 samples) 12 covariance matrices (3 scoring variations x 4 samples) EQS confirmatory factor analysis EQS confirmatory factor analysis M-Plus distribution-free confirmatory analysis M-Plus distribution-free confirmatory analysis

11. Results All solutions not optimal: cross-loading items All solutions not optimal: cross-loading items Fewer model mis-specifications with M- Plus Fewer model mis-specifications with M- Plus No clear pattern: M-Plus suggests poorer fit with ‘Exclude’ scoring option (CFI; but not RMSEA) No clear pattern: M-Plus suggests poorer fit with ‘Exclude’ scoring option (CFI; but not RMSEA)

12. Results: Method of Analysis

13. Results: Method of Compensation - CFI

14. Results: Method of Compensation - RMSEA

15. Results: Method of Compensation – Number of Low Loadings

16. Results Pruchno approach more model mis- specifications Pruchno approach more model mis- specifications Pruchno approach more marginal loadings Pruchno approach more marginal loadings Exclude approach fewest marginal loadings, mis-specifications with EQS (not so with M-Plus) Exclude approach fewest marginal loadings, mis-specifications with EQS (not so with M-Plus) Deletion method results in fewer items with low loadings (i.e. clearer structure) Deletion method results in fewer items with low loadings (i.e. clearer structure)

17. Conclusions Minimal differences across methods of compensation for “Does Not Apply” option Minimal differences across methods of compensation for “Does Not Apply” option No method gives univocally better fit No method gives univocally better fit Listwise deletion gives clearer structure but at cost of smaller and likely biased sample Listwise deletion gives clearer structure but at cost of smaller and likely biased sample

18. Alternative: Item Response Theory IRT provides information on performance of response options IRT provides information on performance of response options Preliminary results of analysis of nursing home data using GGUM (Roberts et al., 2004): Generalized Graded Unfolding Model Preliminary results of analysis of nursing home data using GGUM (Roberts et al., 2004): Generalized Graded Unfolding Model

19. Disorientation Item 16: 5 more extreme than 48 of 18 items

20. Withdrawal Item 40: “Does not Apply” = Most Severe 10 of 18 items