Transcript What information to collect? Jan Smit Statistics Division, ESCAP With thanks to Wei Liu What information to collect? 07/11/2015

What information to collect?
Jan Smit
Statistics Division, ESCAP
With thanks to Wei Liu
What information to collect?
1
07/11/2015
Outline
Overview of ESCAP/WHO project
 Disability question set testing

What questions were asked?
 Sampling & sample characteristics
 What info should be collected?
 Comparison between Washington Group
and WHO questions

What information to collect?
2
07/11/2015
ESCAP/WHO project
ESCAP/WHO Project on Improving
Disability Data
 2004-2006
 Goal: improve national disability
statistics by applying the ICF
framework and using international
measurement standards

What information to collect?
3
07/11/2015
ESCAP/WHO project (cont.)
Target group: statisticians and experts
in disability from 20 countries
 Expected outcome: capacity to
produce better disability data that
meet policy needs
 Partners: Australian Bureau of
Statistics, Washington Group

What information to collect?
4
07/11/2015
Project countries
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Afghanistan
Brunei Darussalam
Cambodia
China
DPR Korea
Islamic Republic of Iran
Japan
Hong Kong, China
Laos
Macao, China
Republic of Korea
Singapore
Sri Lanka
Thailand
Vietnam
Taking part in pilot test
1.
Fiji
2.
Indonesia
3.
India
4.
Mongolia
5.
Philippines
What information to collect?
5
07/11/2015
Motivation for project
Biwako Millennium Framework for
Action (BMF) Principle 6 to
“Strengthen national capacity in data
collection and analysis concerning
disability statistics”
 However, many data issues within
ESCAP region

What information to collect?
6
07/11/2015
Regional disability data issues
Data not available to inform policy
 Disability prevalence is
underestimated
 Traditional view of disability prevails
at every layer of societies, incl.
government
 No comparability of data; need for
use of international standards and
guidelines

What information to collect?
7
07/11/2015
Lack of data comparability
Prevalence of persons with disabilities in Asian
and Pacific Countries (%)
25%
20%
15%
10%
5%
0%
1985
Admin.
Records
1981
Census
1982
Survey
2000
Census
1995
Census
2001
Survey
Singapore Pakistan BangladeshPhilippines Philippines Thailand
What information to collect?
1986
Survey
2001
Census
2002
Survey
2000
Survey
2005
Survey
1987
Survey
2002
Survey
2001
Survey
2003
Survey
Sri Lanka
India
Turkey
Hong
Kong
Korea
(Republic
of)
China
Japan
New
Zealand
Australia
8
07/11/2015
Project components

Phase 1:
Training in the use of ICF for disability
collection, in standard methodologies
and in design of collection tools
 National action plans for improving
disability information systems
 Preparation of disability statistics
training manual & material

What information to collect?
9
07/11/2015
Project components (cont.)

Phase 2:
Field testing of standard sets of disability
questions through identical protocols
 Evaluation of results for application to
real data collections
 Building a network of experts

What information to collect?
10
07/11/2015
Main project activities
4 regional workshops for advocating
and training on the ICF approach and
its implementation
 Pilot testing of disability
questionnaires for use in census and
surveys and results analysis and
dissemination
 The ESCAP/WHO Disability Statistics
Training Manual

What information to collect?
11
07/11/2015
Disability question set testing
Study 1: Sensitivity and specificity
testing (and more)
 Study 2: Test-retest reliability
 Study 3: Cognitive testing

What information to collect?
12
07/11/2015
Pilot test: Study 1

Through NSOs of five countries
Fiji (FJI)
 Indonesia (IDN)
 India (IND)
 Mongolia (MNG)
 Philippines (PHL)

Data collection period: May-July 2005
 In local languages

What information to collect?
13
07/11/2015
Questions
Demographic characteristics
 6 Washington Group questions (W)
 53 WHO questions (B and D)

17 body structure/function questions
from World Health Survey
 36 activity & participation questions
from Disability Assessment Scale (DAS II)


4 Australian Bureau of Statistics
questions on need for assistance (E)
What information to collect?
14
07/11/2015
Activity & participation quest.
Understanding & communicating (D1)
 Getting around (D2)
 Self care (D3)
 Getting along with people (D4)
 Life activities (D5)
 Participation in society (D6)


Incl. 2 questions (D6.2-3) that might be
considered as addressing environmental
factors
What information to collect?
15
07/11/2015
Sampling
Each NSO was requested to administer
the questionnaire to a sample of at
least 1,000 respondents of 18 years
and older representative of the
national population in terms of age,
sex and educational attainment
 Interpreted by the collaborating NSOs
in different ways


Sample not representative for whole
population
What information to collect?
16
07/11/2015
Sample characteristics: Sex
Country
Sex
IND
IDN
PHL
FJI
MNG
Total
Female 245
449
606
527
596
2423
Male
345
351
433
468
466
2063
Total
590
800
1039
995
1062
4486
Col %
Col %
Col %
Col %
Col %
Col %
Female 41.5
56.1
58.3
53
56.1
54
Male
58.5
43.9
41.7
47
43.9
46
Total
100
100
100
100
100
100
What information to collect?
17
07/11/2015
Age
What information to collect?
18
07/11/2015
Education
What information to collect?
19
07/11/2015
Marital
status
What information to collect?
20
07/11/2015
Employment
status
What information to collect?
21
07/11/2015
What info should be collected?
Depends on question(s) to be
answered (already discussed)
 What domains are most important?



No question about respondents overall
judgement of their own health (or
perhaps happiness, quality of life)
What are the main underlying
‘factors’?

(Exploratory) factor analysis
What information to collect?
22
07/11/2015
Factors
1.
2.
3.
4.
5.
Day-to-day life (D1.3, D5.1, D5.2,
D5.3, D5.4, D5.5, D5.6, D5.7, D5.8
and D6.8)
Self-care (W5, D3.1, D3.2, E1 and E2)
Resources and emotion (D6.4, D6.5,
D6.6, D6.7 and D6.8)
Getting along with people (D4.1,
D4.2, D4.3 and D4.4)
Mobility (W3, B1.13, B2.1, B2.2 and
B2.5)
What information to collect?
23
07/11/2015
Factors (cont.)
6.
7.
8.
9.
10.
Hearing (W2, B1.8 and B1.9)
Learning and understanding (D1.4
and D1.5)
Seeing (W1, B1.6 and B1.7)
Pain and discomfort (B1.1 and B1.2)
Sadness and anxiety (B1.16 and
B1.17)
What information to collect?
24
07/11/2015
Prevalence
What information to collect?
25
07/11/2015
Sensitivity & specificity
Base of comparison
+
_
Test
+ True positives (a) False positives
questions
(b)
_ False negatives
True negatives
(c)
(d)
Sensitivity: __a___
a+c
What information to collect?
Specificity: __d__
b+d
26
07/11/2015
Sensitivity, specificity, gamma
WHO question Sensitivity
Specificity
Gamma
Seeing B1.6
0.60
0.91
0.83
B1.7
0.63
0.90
0.84
B1.6 and B1.7 0.58
0.94
0.85
Hearing B1.8
0.39
0.98
0.91
B1.9
0.38
0.97
0.88
B1.8 and B1.9 0.34
0.98
0.90
0.47
0.93
0.80
0.51
0.91
0.79
B1.13 and D2.5 0.43
0.96
0.82
Mobility B1.13
D2.5
What information to collect?
27
07/11/2015
Sensitivity, specificity, gamma
WHO question Sensitivity
Specificity
Gamma
0.46
0.88
0.70
0.48
0.92
0.79
D1.1 and D1.2 0.43
0.94
0.79
0.51
0.98
0.96
0.58
0.98
0.96
D3.1 and D3.2 0.49
0.99
0.96
Cognition D1.1
D1.2
Self care D3.1
D3.2
Communication D1.5
0.31
0.96
0.80
D1.6
0.31
0.96
0.80
D1.5 and D1.6 0.28
0.97
0.80
What information to collect?
28
07/11/2015
Ongoing research…
Can info on some ‘important’ domains
be recovered from a limited number
of questions/variables?
 E.g. can D5.5 be explained by the 6
WG questions (or WHO counterpart
questions) + demographic info?

Initial results in terms of model fit
promising
 Difficulty is accuracy of prediction

What information to collect?
29
07/11/2015
Not only collect data…

But document and preserve data
collection exercise
Instrument, scope, coverage, etc.
 Response options, labels, etc.
 Microdata Management Toolkit



www.internationalsurveynetwork.org
And preferably disseminate micro
data (unit records)

Possibly anonymized
What information to collect?
30
07/11/2015