1 ESCAP/WHO Pilot Study Overview Wei Liu Statistics Division, ESCAP Sixth Meeting of the Washington Group on Disability Statistics, Kampala, Oct , 2006

2 1. Objective 2. Background and methodology 3. Questions tested 4. Analysis and results 5. Moving forward 1. Objective 2. Background and methodology 3. Questions tested 4. Analysis and results 5. Moving forward Outline

3 Objectives  Comparing the Washington Group 6- question “short set” with a WHO “long set” of questions  Studying the relative importance of different domains  Considering differences in underlying concepts and phrasings of questions

4 Background Pilot countries: Fiji, Indonesia, India, Mongolia, Philippines Test Design: Study One: Testing sensitivity and specificity Study Two: Test-Retest Reliability Study Three: Linguistic Evaluation & Cognitive interviewing Other Partners: Australian Bureau of Statistics (ABS )

5 Sample Selection India: Meerut District, Uttar Pradesh State Indonesia: Central Jakarta Mongolia: Ulaanbaatar Philippines: A selection of urban and rural geographic “clusters” geographic “clusters” Fiji: 2 rural and 8 urban enumeration areas in Central Division (capital city & surrounding areas)

6 Sample Sizes FijiIndiaIndonesiaMongoliaPhilippines Study Study Study

7 Sample Summary: Age Structure

8 Pilot Study Packets  Study Instructions  Questionnaire  Version A  Version B  Cognitive Interview Schedule

9 Support Materials  Translation  Guidelines  List of key terms  Linguistic evaluation form  Interview Guidelines & Q- by-Q specifications  Disability Statistics Training Manual - Draft  Quality assurance standards & guidelines  Data entry program

10 Questions Tested Question set 1: Washington Group’s general disability measures, 6 Qs Question set 2: -World Health Survey body structure/functions domains, 17 Qs; -WHO Disability Assessment Schedule II (WHO DAS II), activity and participation domains, 36 Qs; Australian Bureau of Statistics, need-for-assistance domains, 4 Qs

11 Study 1: Sensitivity and Specificity of WG Q’s n=4512 SensitivitySpecificity “Difficulty…seeing, even if wearing glasses” (W1), compared to: “…Seeing a person across the road” (B1.6) 59%90% “…Seeing an object at arm's length” (B1.7) 62%90% “Difficulty…hearing, even if using a hearing aid” (W2), compared to: “…Hearing someone talking on the other side of the room in a normal voice? ” (B1.8) 39%98% “…Hearing what is said in quiet room” (B1.9) 38%96%

12 Study 1 (cont.) Walking or climbing steps (W3) * Walking a long distance >1km (D2.5) 51%91% Remembering or concentrating (W4) * Remembering to do important things (D1.2) 48%92% Self care (W5) * Washing whole body (D3.1) 50%98% Self care (W5) * Getting dressed (D3.2) 56%98% Self care (W5) * Need for assistance: Self Care (E1) 84%98% Communicating (W6) * Understanding what people say (D1.5) 32%95%

13 Study 2: Kappa Statistics for Test-retest Reliability W1 Seeing 0.57 W2 Hearing 0.54 W3 Walking & climbing 0.64 W4 Remembering & concentrating 0.59 W5 Self Care 0.71 W6 Communicating 0.53 B1.1 Pain 0.60 B1.2 Discomfort 0.53 B1.5 Using hands & fingers 0.58 B1.6 Seeing (short sighted) 0.65 B1.7 Seeing (long sighted) 0.53 B1.8 Hearing (far) 0.58 B1.9 Hearing (close) 0.38 B1.12 Short of breath (rest) 0.48 B1.13 Short of breath (exercise) 0.52

14 Study 2 (cont.) B1.14 Coughing 0.47 B1.15 Sleeping 0.49 B1.17 Worry or anxiety 0.50 D1.1 Concentrating 0.55 D1.2 Remembering 0.55 D1.5 Understanding what people say 0.42 D1.6 maintaining conversation 0.42 D2.1 Standing 30min D2.2 Standing up 0.58 D2.3 Moving inside 0.43 D2.4 Getting out 0.60 D2.5 Walking >1km 0.59 D3.1 Washing 0.55 D3.2 Dressing 0.62

15 Study 3: Cognitive Debriefing  Most questions are understood well  95% of the respondents had no difficulties responding to questions  Minor difficulties with questions on:  Mental disability (i.e. cognition, anxiety, getting along with people, etc.)  Need-for-assistance questions  W6 communication question  Translation

16 Identifying “Core Domains”  ESCAP/WHO 4 th Workshop on Improvement of Disability Statistics and Measurement (Bangkok, June, 2006)  Factor analysis research

17 Factor Loadings Disability Factor 1 LifeActivity Factor 2 Hearing Factor 3 Self Care Factor 4 Participation Factor 5 Mobility Factor 6 Vision

18 Domain Selections Factor Analysis  Life Activities  Hearing  Self Care  Participation  Mobility  Vision 4 th Bangkok Workshop Recommendation  Life Activities  Hearing  Self Care  Communication  Mobility  Vision

19 Moving Forward  Further analysis of “core” domains  To select/reconsider the phrasings of the questions for the selected domains  Follow-up pilot studies for census recommendations and for an “extended” question set for use in surveys  Regional training and advisory missions

20 Thank You!