1. What information to collect?
Jan Smit
Statistics Division, ESCAP
With thanks to Wei Liu
What information to collect?
29/07/2019

2. Outline Overview of ESCAP/WHO project Disability question set testing
What questions were asked?
Sampling & sample characteristics
What info should be collected?
Comparison between Washington Group and WHO questions
What information to collect?
29/07/2019

3. ESCAP/WHO project ESCAP/WHO Project on Improving Disability Data
Goal: improve national disability statistics by applying the ICF framework and using international measurement standards
Timeframe: this is one of the various last activities, although an extension of the project is currently negotiated with the donor (ROK)
What information to collect?
29/07/2019

4. ESCAP/WHO project (cont.)
Target group: statisticians and experts in disability from 20 countries
Expected outcome: capacity to produce better disability data that meet policy needs
Partners: Australian Bureau of Statistics, Washington Group
What information to collect?
29/07/2019

5. Project countries What information to collect? 29/07/2019 Afghanistan
Brunei Darussalam
Cambodia
China
DPR Korea
Islamic Republic of Iran
Japan
Hong Kong, China
Laos
Macao, China
Republic of Korea
Singapore
Sri Lanka
Thailand
Vietnam
Taking part in pilot test
Fiji
Indonesia
India
Mongolia
Philippines
What information to collect?
29/07/2019

6. Motivation for project
Biwako Millennium Framework for Action (BMF) Principle 6 to “Strengthen national capacity in data collection and analysis concerning disability statistics”
However, many data issues within ESCAP region
What information to collect?
29/07/2019

7. Regional disability data issues
Data not available to inform policy
Disability prevalence is underestimated
Traditional view of disability prevails at every layer of societies, incl. government
No comparability of data; need for use of international standards and guidelines
Lack of even basic information on persons with disabilities at the national level
Data that is available is not comparable across sources
Under-reporting of true incidence of disability
Data does not sufficiently reflect the lived experience and needs of persons with disabilities
The current standard classification system (ICF) not fully understood or implemented in most countries in the region
What information to collect?
29/07/2019

8. Lack of data comparability
What information to collect?
29/07/2019

9. Project components Phase 1:
Training in the use of ICF for disability collection, in standard methodologies and in design of collection tools
National action plans for improving disability information systems
Preparation of disability statistics training manual & material
What information to collect?
29/07/2019

10. Project components (cont.)
Phase 2:
Field testing of standard sets of disability questions through identical protocols
Evaluation of results for application to real data collections
Building a network of experts
What information to collect?
29/07/2019

11. Main project activities
4 regional workshops for advocating and training on the ICF approach and its implementation
Pilot testing of disability questionnaires for use in census and surveys and results analysis and dissemination
The ESCAP/WHO Disability Statistics Training Manual
What information to collect?
29/07/2019

12. Disability question set testing
Study 1: Sensitivity and specificity testing (and more)
Study 2: Test-retest reliability
Study 3: Cognitive testing
Study 2: 100 respondents re-visited and asked same questions again. Level of agreement expressed in kappa statistic generally OK.
Study 3: 100 willing participants were asked about their understanding of the questions.
What information to collect?
29/07/2019

13. Pilot test: Study 1 Through NSOs of five countries
Fiji (FJI)
Indonesia (IDN)
India (IND)
Mongolia (MNG)
Philippines (PHL)
Data collection period: May-July 2005
In local languages
What information to collect?
29/07/2019

14. Questions Demographic characteristics 6 Washington Group questions (W)
53 WHO questions (B and D)
17 body structure/function questions from World Health Survey
36 activity & participation questions from Disability Assessment Scale (DAS II)
4 Australian Bureau of Statistics questions on need for assistance (E)
What information to collect?
29/07/2019

15. Activity & participation quest.
Understanding & communicating (D1)
Getting around (D2)
Self care (D3)
Getting along with people (D4)
Life activities (D5)
Participation in society (D6)
Incl. 2 questions (D6.2-3) that might be considered as addressing environmental factors
What information to collect?
29/07/2019

16. Sampling
Each NSO was requested to administer the questionnaire to a sample of at least 1,000 respondents of 18 years and older representative of the national population in terms of age, sex and educational attainment
Interpreted by the collaborating NSOs in different ways
Sample not representative for whole population
What information to collect?
29/07/2019

17. Sample characteristics: Sex
Country
Sex IND IDN PHL FJI MNG Total
Female
Male
Total
Col % Col % Col % Col % Col % Col %
Female
Male
Total
What information to collect?
29/07/2019

18. Age
What information to collect?
29/07/2019

19. Education
What information to collect?
29/07/2019

20. Marital status
What information to collect?
29/07/2019

21. Employment status
What information to collect?
29/07/2019

22. What info should be collected?
Depends on question(s) to be answered (already discussed)
What domains are most important?
No question about respondents overall judgement of their own health (or perhaps happiness, quality of life)
What are the main underlying ‘factors’?
(Exploratory) factor analysis
What information to collect?
29/07/2019

23. Factors
Day-to-day life (D1.3, D5.1, D5.2, D5.3, D5.4, D5.5, D5.6, D5.7, D5.8 and D6.8)
Self-care (W5, D3.1, D3.2, E1 and E2)
Resources and emotion (D6.4, D6.5, D6.6, D6.7 and D6.8)
Getting along with people (D4.1, D4.2, D4.3 and D4.4)
Mobility (W3, B1.13, B2.1, B2.2 and B2.5)
What information to collect?
29/07/2019

24. Factors (cont.) Hearing (W2, B1.8 and B1.9)
Learning and understanding (D1.4 and D1.5)
Seeing (W1, B1.6 and B1.7)
Pain and discomfort (B1.1 and B1.2)
Sadness and anxiety (B1.16 and B1.17)
What information to collect?
29/07/2019

25. Prevalence
What information to collect?
29/07/2019

26. Sensitivity & specificity
Base of comparison + _
Test questions
True positives (a)
False positives (b)
False negatives (c)
True negatives (d)
Sensitivity: __a___
a + c
Specificity: __d__
b + d
What information to collect?
29/07/2019

27. Sensitivity, specificity, gamma
WHO question Sensitivity Specificity Gamma
Seeing B B
B1.6 and B
Hearing B B
B1.8 and B
Mobility B D
B1.13 and D
What information to collect?
29/07/2019

28. Sensitivity, specificity, gamma
WHO question Sensitivity Specificity Gamma
Cognition D D
D1.1 and D
Self care D D
D3.1 and D
Communication D D
D1.5 and D
What information to collect?
29/07/2019

29. Ongoing research…
Can info on some ‘important’ domains be recovered from a limited number of questions/variables?
E.g. can D5.5 be explained by the 6 WG questions (or WHO counterpart questions) + demographic info?
Initial results in terms of model fit promising
Difficulty is accuracy of prediction
What information to collect?
29/07/2019

30. Not only collect data…
But document and preserve data collection exercise
Instrument, scope, coverage, etc.
Response options, labels, etc.
Microdata Management Toolkit
And preferably disseminate micro data (unit records)
Possibly anonymized
What information to collect?
29/07/2019