Joint UNECE/Eurostat Meeting on Population and Housing Censuses (13-15 May 2008) Sample results expected accuracy in the Italian Population and Housing Census Giancarlo Carbonetti,

Download Report

Transcript Joint UNECE/Eurostat Meeting on Population and Housing Censuses (13-15 May 2008) Sample results expected accuracy in the Italian Population and Housing Census Giancarlo Carbonetti,

Joint UNECE/Eurostat Meeting on Population and Housing
Censuses (13-15 May 2008)
Sample results expected accuracy in the
Italian Population and Housing Census
Giancarlo Carbonetti, Marco Fortini
Istat – Italian National Statistical Institute
General Censuses Directorate
May 13th 2008
Joint
UNECE
Eurostat
Meeting
Outline
 Introduction
 Some aspects related to the use of samples of households
for long form enumerations
 Sampling strategies
 Simulation study
 Some results
 Conclusions
2
Joint
UNECE
Eurostat
Meeting
Introduction - 1
Main critical issue of the last Census
 Huge organizational (and economical) effort of Municipal
Census Offices
 sudden and time-concentrated increase of workload
 for largest municipalities, massive network of enumerators
and coordinators to be trained and managed
 lack of adequately skilled resources, high turn over rates
Main objectives for the next Census
 to improve the census operations efficiency
 to reduce the municipalities workload
 to keep an high level of quality
3
Joint
UNECE
Eurostat
Meeting
Introduction - 2
 Innovations proposed to reach the objectives



the use of population registers
mail out of census forms
mixed mode of data collection mainly based on mail and web
 Expected consequences with the innovations


the increase of “back office” work
the reduction of enumerators number (“front office” work)
 How it is possible

increasing the response rates
A proposal: the use of a “short form” version of the
questionnaire is considered to reach high response rates.
4
Joint
UNECE
Eurostat
Meeting
Introduction - 3
Consequences of the use of short form
 increasing the response rates
 reducing as much as possible the response time delay
This approach risks information loss!!!
How to preserve the richness of the census information
 by a selection of a sample of households to which a
“long form” version of the questionnaire is supplied
Strategy: the simultaneously use of short and long forms.
5
Joint
UNECE
Eurostat
Meeting
Some aspects related to the use of samples of households
for long form enumeration - 1
Which type of information can be surveyed by means of a
sample of long forms and which must be collected on the
whole population?
 The overall set of census variables is partitioned into
two subsets


the demographic variables (gender, date of birth, marital status,
nationality, …)
the remaining variables (educational level, occupational status,
commuting)
 Short form accounts for merely the first set of variables
whereas long form accounts for the whole set
6
Joint
UNECE
Eurostat
Meeting
Some aspects related to the use of samples of households
for long form enumeration - 2
Which is the population municipality threshold under which
the sampling strategy cannot be adopted?
 An option we are taking into consideration is to sample
in municipalities with more than 5,000 inhabitants


long forms will be submitted to a sample of households
short forms will be administered to remaining households
 In municipalities smaller than 5,000 inhabitants long
forms will be submitted to the whole population
7
Joint
UNECE
Eurostat
Meeting
Some aspects related to the use of samples of households
for long form enumeration - 3
Which domains have to be considered to plan the sample
and to produce accurate estimates?
 New “census domains” have been defined


an appropriate methodology was adopted to build up census
domains by aggregating the smallest census areas
the new “areas” are referred to sub-municipal level
 Accuracy of sampling estimates for different territorial levels


a similar precision is expected for estimates among areas
higher precision is expected for larger territorial reference
(from sub-municipal to nationwide level)
8
Joint
UNECE
Eurostat
Meeting
Some aspects related to the use of samples of households
for long form enumeration - 4
Which statistical methodology performs the most accurate
estimation?
… in terms of …
 sampling design
 use of appropriate lists
 efficient estimation methods
 sampling error assessment
The answer to this question is the aim of the study of which
some results will be presented.
9
Joint
UNECE
Eurostat
Meeting
Sampling strategies
Two different sampling designs have been tested
 Simple Random Sampling of HOUseholds from Administrative
Registers (SRSHOU) managed by municipalities
 Area Frame Sampling based on a Simple Random Sampling of
ENumeration Areas (SRSENA) which implies a complete data
collection of households dwelling in the selected enumeration
areas (from Digital Geocoded Database)
Different studies have been conducted
 To compare the two different approaches (with a sampling ratio
of about one third of the whole population considered)
 To evaluate in the SRSHOU the improvement of the estimates
precision for increasing sampling ratio (10%, 15%, 20%, 33%)
 To introduce some stratifications of the units involved
10
Joint
UNECE
Eurostat
Meeting
Simulation study - 1
The sampling strategies were compared to each other through
Monte Carlo sampling replications (carried out on 2001 Italian
Census data) in order to assess the sampling error defined by
the coefficient of variation (CV) which represents an accuracy
measurement of the sampling estimates.
Main features of the sampling designs
 Domains: the “new areas” referred to sub-municipal districts
 Target variables: “variables” related to cross-classification of
educational level, employment status and commuting with
demographic variables
 Sampling units: “households” or “enumeration area”
 Estimator: “calibrated estimators” by using final weights
properly modified so to make the sample more representative
11
Joint
UNECE
Eurostat
Meeting
Simulation study - 2
 Because of the strong differences among the Italian
municipalities, 40 of them with different population size and
from different regions of Italy were considered
Geographical
area
Classes of population size (a)
more than
100,000
Total
10,000-20,000
20,000-100,000
North
4
6
6
16
Center
2
3
3
8
South
4
6
6
16
Total
10
15
15
40
(a) It has been considered the legal (official) population date referred to the 2001 Census
of Population.
12
Joint
UNECE
Eurostat
Meeting
Simulation study - 3
Amount of units involved by the simulation study
Sampled Units
Areas
497
Enumeration areas
30,890
Universe
%
3,347(*) 14.85%
382,534
8.08%
Households
2,243,511 21,810,676 10.29%
Individuals
5,537,582 56,594,021
(*) Estimated number
13
9.78%
Scatter plot of cv and p (estimates) for each census area.
SRSHOU design (sampling ratio=33%). City of Perugia.
100
90
80
70
60
cv%
Joint
UNECE
Eurostat
Meeting
50
40
30
20
10
0
0
3% 5
2%
10
15
20
25
30
p%
1%
14
35
40
45
50
55
Joint
UNECE
Eurostat
Meeting
Distribution of median cv for classes of p for SRSHOU design and
SRSENA design (both with sampling ratio=33%).
Comparison of 4 municipalities.
Classes
of p
Milano (111 areas)
Bologna (32 areas)
Padova (18 areas)
Livorno (13 areas)
SRSHOU
SRSENA
SRSHOU
SRSENA
SRSHOU
SRSENA
SRSHOU
SRSENA
< 0.05%
97.78
94.12
96.52
94.31
99.65
98.34
101.61
0.05%├0.1%
51.61
51.59
50.69
52.06
0.1%├0.25%
34.67
THIS50.67
IS DUE
THE54.13
49.54 TO
51.70
34.92
35.00
35.20
35.37
36.03
CLUSTER
EFFECT
102.21
35.08
35.67
0.25%├0.5%
22.96
24.38
24.17
24.73
25.58
26.45
23.70
24.37
0.5%├1%
16.86
18.71
16.85
18.32
16.95
17.81
17.16
18.72
1%├2.5%
10.61
12.21
10.74
11.95
11.07
12.00
11.34
12.90
2.5%├5%
7.02
8.53
7.07
8.25
7.35
8.48
7.17
9.00
5%├10%
4.84
5.97
4.75
5.74
5.05
5.85
4.88
6.39
10%├15%
3.17
4.41
3.09
4.09
3.19
4.37
3.06
4.82
15%├20%
2.44
3.46
2.38
3.12
2.44
3.14
2.44
3.39
20%├30%
1.89
2.61
1.92
2.48
2.08
2.73
2.05
2.88
≥ 30%
1.35
1.78
1.32
1.60
1.39
1.72
1.40
2.00
15
Joint
UNECE
Eurostat
Meeting
Loss of efficiency (in terms of CV for classes of p) of estimation with
SRSENA with respect to SRSHOU design (both with sampling
ratio=33%). Comparison of 4 municipalities.
[CV(SRSHOU_s.r. 33%)-CV(SRSENA_s.r. 33%)]
Classes
of p
Milano
(111 areas)
Bologna
(32 areas)
Padova
(18 areas)
Livorno
(13 areas)
< 0.05%
3.65
2.21
1.31
0.60
0.05%├0.1%
0.03
1.13
-2.43
-1.37
0.1%├0.25%
-0.25
-0.20
-0.66
-0.59
0.25%├0.5%
-1.42
-0.56
-0.87
-0.68
0.5%├1%
-1.85
-1.47
-0.87
-1.56
1%├2.5%
-1.60
-1.22
-0.93
-1.56
2.5%├5%
-1.51
-1.18
-1.13
-1.83
5%├10%
-1.13
-0.99
-0.80
-1.51
10%├15%
-1.24
-1.01
-1.18
-1.76
15%├20%
-1.02
-0.73
-0.70
-0.95
20%├30%
-0.72
-0.56
-0.65
-0.82
≥ 30%
-0.43
-0.29
-0.33
-0.60
16
Joint
UNECE
Eurostat
Meeting
Distribution of median cv for classes of p. Comparison of 4 different
sampling ratios with the SRSHOU design.
Classes of p
sampling ratio=
10%
sampling ratio=
15%
sampling ratio=
20%
sampling ratio=
33%
170 areas
140 areas
111 areas
204 areas
< 0.05%
220.51
157.20
142.00
98.21
0.05%├0.1%
111.48
87.22
74.20
51.14
0.1%├0.25%
75.57
59.83
49.97
34.76
0.25%├0.5%
50.70
39.92
33.97
23.44
0.5%├1%
35.54
28.10
23.74
16.56
1%├2.5%
23.62
18.56
15.33
10.68
2.5%├5%
15.50
12.29
10.09
7.04
5%├10%
10.46
8.26
6.93
4.82
10%├15%
7.06
5.40
4.40
3.13
15%├20%
5.57
4.27
3.54
2.42
20%├30%
4.50
3.48
2.84
1.93
≥ 30%
3.20
2.42
1.94
1.34
17
Joint
UNECE
Eurostat
Meeting
Gain of efficiency (in terms of CV for classes of p) of estimation with
SRSHOU design by increasing sampling ratio from 10% to 33% .
[CV(SRSHOU_s.r. 10%)-CV(SRSHOU_s.r. N%)]x100/[CV(SRSHOU_s.r. 10%)]
Classes of p
increasing s.r.
from 10% to 15%
increasing s.r.
from 10% to 20%
increasing s.r.
from 10% to 33%
< 0.05%
28.71
35.60
55.46
0.05%├0.1%
21.76
33.44
54.13
0.1%├0.25%
20.83
33.88
54.00
0.25%├0.5%
21.26
33.00
53.77
33.20
53.40
1%├2.5%
20.93
Gain between
21-23 percent
21.42
54.78
2.5%├5%
20.71
35.10
Gain between
33-38 percent
34.90
5%├10%
21.03
33.75
54.58
Gain between
53-58 percent
53.92
10%├15%
23.51
37.68
55.67
15%├20%
23.34
36.45
56.55
20%├30%
22.67
36.89
57.11
≥ 30%
24.38
39.38
58.13
0.5%├1%
18
Joint
UNECE
Eurostat
Meeting
Distribution of median cv for five classes of p and three classes of
area (according to population size). Comparison of 4 different
sampling ratios with the SRSHOU design.
Classes of p
0.1%├0.25%
0.5%├1%
2.5%├5%
10%├15%
20%├30%
Population
by area
(thousands)
Sampling ratio
10%
15%
20%
33%
<10
90.00
71.27
59.48
40.20
10├12
76.23
60.03
50.45
34.41
≥ 12
66.65
53.04
43.51
30.58
< 10
43.11
33.50
28.97
19.53
10├12
35.08
27.46
22.99
16.48
≥ 12
31.25
24.95
20.97
14.85
< 10
19.12
14.68
12.22
8.25
10├12
15.58
12.36
9.89
7.08
≥ 12
14.00
10.98
9.06
6.35
< 10
8.78
6.44
5.22
3.67
10├12
7.00
5.46
4.41
3.13
≥ 12
6.29
4.79
3.89
2.83
< 10
5.46
4.16
3.44
2.27
10├12
4.57
3.42
2.94
2.01
≥ 12
4.05
3.20
2.59
1.77
19
Median CV for some classes of p and for three classes of area (according to
population size). Comparison of 4 different sampling ratios (s.r.) with the
SRSHOU design. Graph referred to area size less than 10,000 inhabitants.
s.r.=15%
130
s.r.=20%
120
s.r.=33%
110
100
90
80
70
Area size<10,000
60
50
40
30
20
10
%
30
>
0%
-3
-2
5%
0%
20
classes of p
15
10
-1
-1
0%
%
5
-5
2.
5
.5
%
-2
%
1
-1
0.
5
-0
0.
25
-0
.2
5
%
.1
%
20
.5
%
0
-0
66.7
53.0
43.5
30.6
31.3
25.0
21.0
14.9
14.0
11.0
9.1
6.4
6.3
4.8
3.9
2.8
4.1
3.2
2.6
1.8
0.
1
76.2
60.0
50.5
34.4
35.1
27.5
23.0
16.5
15.6
12.4
9.9
7.1
7.0
5.5
4.4
3.1
4.6
3.4
2.9
2.0
≥ 12
%
90.0
71.3
59.5
40.2
43.1
33.5
29.0
19.5
19.1
14.7
12.2
8.3
8.8
6.4
5.2
3.7
5.5
4.2
3.4
2.3
10├12
0.
05
10%
15%
0.1-0.25%
20%
33%
10%
15%
0.5-1%
20%
33%
10%
15%
2.5-5%
20%
33%
10%
15%
10-15%
20%
33%
10%
15%
20-30%
20%
33%
<10
s.r.=10%
140
0.
05
s.r.
150
Area size (thousands)
<
Classes
of p
median cv%
Joint
UNECE
Eurostat
Meeting
Median CV for some classes of p and for three classes of area (according to
population size). Comparison of 4 different sampling ratios (s.r.) with the SRSHOU
design. Graph referred to area size between 10,000 and 12,000 inhabitants.
s.r.=15%
130
10%
90.0
76.2
66.7
15%
71.3
60.0
53.0
120
0.1-0.25%
20%
59.5
50.5
43.5
110
33%
40.2
34.4
30.6
100
10%
43.1
35.1
31.3
15%
33.5
27.5
25.0
90
0.5-1%
20%
29.0
23.0
21.0
80
33%
19.5
16.5
14.9
70
10%
19.1
15.6
14.0
15%
14.7
12.4
11.0
60
2.5-5%
20%
12.2
9.9
9.1
50
33%
8.3
7.1
6.4
40 of
10% of efficiency
8.8
7.0
6.3 terms
The gain
(in
15%
6.4
5.5
4.8
30
10-15% for census areas with size
CV)
20%
5.2
4.4
3.9
20
between
with
33% 10,000
3.7 and
3.1 12,000
2.8
10%to census
5.5
4.6
respect
areas4.1with 10less
15%
4.2
3.4
3.2
0
20-30% 10,000 is about 14-20 percent.
than
20%
3.4
2.9
2.6
Similar
for all
33%results
2.3 are2.0obtained
1.8
s.r.=20%
s.r.=33%
21
%
30
>
0%
-3
-2
5%
0%
20
classes of p
15
10
-1
-1
0%
%
5
-5
2.
5
0.
5
.5
%
-0
%
.2
5
0.
25
-0
0.
1
-0
.1
%
%
0.
05
0.
05
<
tested sampling ratios.
Area size
between 10-12,000
.5
%
≥ 12
-2
10├12
%
<10
s.r.=10%
140
1
s.r.
150
Area size (thousands)
-1
Classes
of p
median cv%
Joint
UNECE
Eurostat
Meeting
Median CV for some classes of p and for three classes of area (according to
population size). Comparison of 4 different sampling ratios (s.r.) with the
SRSHOU design. Graph referred to area size more than 12,000 inhabitants.
s.r.=15%
130
10%
90.0
76.2
66.7
15%
71.3
60.0
53.0
120
0.1-0.25%
20%
59.5
50.5
43.5
110
33%
40.2
34.4
30.6
100
10%
43.1
35.1
31.3
15%
33.5
27.5
25.0
90
0.5-1%
20%
29.0
23.0
21.0
80
33%
19.5
16.5
14.9
70
10%
19.1
15.6
14.0
15%
14.7
12.4
11.0
60
2.5-5%
20%
12.2
9.9
9.1
50
33%
8.3
7.1
6.4
40 of
10% of efficiency
8.8
7.0
6.3 terms
The gain
(in
15%
6.4
5.5
4.8
30
10-15% for census areas with size
CV)
20%
5.2
4.4
3.9
20 to
more than
12,000
with respect
33%
3.7
3.1
2.8
10
10%areas
5.5with4.6less 4.1
census
than 10,000
s.r.=20%
s.r.=33%
%
30
>
0%
-3
-2
5%
0%
20
classes of p
15
10
-1
-1
0%
%
5
-5
2.
5
0.
5
.5
%
-0
0.
25
-0
.2
5
%
.1
%
-0
22
0.
1
0.
05
<
0.
05
%
15%
4.2
3.4
3.2
0
20-30%
is
about
22-28
percent.
As before,
20%
3.4
2.9
2.6
similar33%results
obtained
for all
2.3 are2.0
1.8
tested sampling ratios.
Area size>12,000
.5
%
≥ 12
-2
10├12
%
<10
s.r.=10%
140
1
s.r.
150
Area size (thousands)
-1
Classes
of p
median cv%
Joint
UNECE
Eurostat
Meeting
Joint
UNECE
Eurostat
Meeting
Distribution of the estimates referred to areas larger than 12,000
inhabitants for classes of cv. Comparison of percentage frequencies
for 4 different sampling ratios with the SRSHOU design.
Classes of
coefficient of
variation %
< 2%
2%├5%
5%├10%
10%├20%
20%├50%
50%├100%
100%├200%
≥ 200%
Sampling ratio
10%
15%
20%
0.57
2.69
HA – high accuracy
13.04
17.53
33%
6.39
13.14
18.40
23.64
18.02
26.28
28.64
29.14
30.16
MA – medium accuracy
25.09
19.71
23.54
16.20
16.75
13.32
7.21
5.69
3.44
3.65
2.00
1.61
1.03
0.95
-
16.18
9.32
4.40
LA – low accuracy
2.25
23
Distribution of the estimates referred to areas larger than 12,000
inhabitants for classes of cv. Comparison of percentage frequencies
for 4 different sampling ratios with the SRSHOU design - 2
Sampling ratio
Classes of cv%
10%
15%
20%
33%
< 10%
29.80
38.25
51.07
65.42
10%├50%
54.23
49.87
40.29
29.52
≥ 50%
15.97
11.89
8.64
5.05
70
HA - high accuracy
60
% of estimates
Joint
UNECE
Eurostat
Meeting
cv%<10%
50
cv% in 10-50%
40
cv%>50%
30
MA - medium accuracy
20
10
LA - low accuracy
0
10%
15%
20%
Sampling ratio
24
33%
Joint
UNECE
Eurostat
Meeting
Estimates of p referred to territory given by aggregation of areas.
Generic sampled area a
pˆ a X  CVpˆ a 
Territory RS given by
aggregation of K
sampled areas
 
Percentage expected
reduction of CV in RS
Territory R given by
aggregation of
sampled areas and
not sampled areas
γ  NS N
pˆ R S X   CV pˆ R S
1

CVpˆ a 
K
1 

red%  1 
 100
K

pˆ R X   CVpˆ R  

γ 
 100
Percentage expected red%  
1 
K

reduction of CV in R
Quote of sub-population of R
elegible for drawing the LF sample.
25
γ
CVpˆ a 
K
Joint
UNECE
Eurostat
Meeting
Conclusions - 1
 As expected, the most accurate estimates were obtained for:


simple random sampling of households from administrative
registers
largest sampling ratio
 Better efficiency of estimates for largest areas (>12,000
inhabitants)

this result could represent a suggestion for planning the sampling
design by defining larger census areas (of about 15,000 people)
 The estimates referred to large domains given by aggregation of
areas show high accuracy


the accuracy increases with the domain’s number
in case in which a part of the large domain is totally surveyed, the
estimates show a further increasing in accuracy
26
Joint
UNECE
Eurostat
Meeting
Conclusions - 2
 However area frame sampling is only slightly less efficient
than SRSHOU, thus it could be adopted where reliable
administrative registers are not available
 Sampling ratio will be chosen considering trade-off between:


needed financial savings
accuracy required at different territorial domains
 Further analyses will be conducted on small area estimation
techniques to produce more accurate estimates for:


smallest territorial levels
rare populations
27
Joint
UNECE
Eurostat
Meeting
Thank you for your attention and …
28
Joint
UNECE
Eurostat
Meeting
… have a good lunch!!!
29
Joint
UNECE
Eurostat
Meeting
30
Joint
UNECE
Eurostat
Meeting
Simulation study - 4
 Cross-classification cells


educational level, employment status, commuting and gender
90 simple estimation cells
 Calibration constraints defined by cross-classifying gender by
age, and gender by marital status
 Computational algorithm implemented by SAS code for each
municipality and for each alternative sampling design:





step 1) selection of a sample (of households or enumeration areas)
step 2) computation of final weights
step 3) estimation of the relative frequency p for each target cell
step 4) iteration of steps 1), 2) and 3) for 1,000 sampling replications
step 5) computation of sampling distribution mean and standard error
for each one of the 90 frequency cells
31
Joint
UNECE
Eurostat
Meeting
Evaluation criterion: the coefficient of variation
In order to compare the sampling strategies has been considered as
evaluation criterion the coefficient of variation CV :
σpˆ x 
cvx 
100
Epˆ x 
which represents an accuracy measurement of the sampling estimates.
Consequently, the percentage maximum expected error can be computed:
Δ% ≈ 1.96 · CV
which is implied (with a probability of 0.95) by the estimation method.


The distribution of the empirical CV’s for all the 90 target cells was
determined.
After having classified the target cells depending on their value p ,
CV’s distribution related to the cells in the same p group has been
studied.
32
Joint
UNECE
Eurostat
Meeting
Estimates of p referred to territory given by aggregation of areas.
Case 1: aggregation of sampled areas.
pˆ a X   CVpˆ a 
Estimate referred to the
generic sampled area a
ˆ R X 
p
S
Estimate referred to the territory
RS given by aggregation of K
sampled areas
 W pˆ
a
aR S
where
Wa  Na N RS
100%
 
90%
CV pˆ R S 
80%
1
K
CVpˆ a 
for K>100 → red%>90%
70%

1 
 100
red%  1 
K

for K>30 → red%>80%
60%
for K>5 → red%>50%
50%
40%
0
500
1000
1500
Riduzione percentuale "attesa" del CV
─ Percentage
expected reduction of CV
2000
2500
33
3000
Numero di aree K
Percentage expected
reduction of CV
Number of areas K
a
Joint
UNECE
Eurostat
Meeting
Estimates of p referred to territory given by aggregation of areas.
Case 2: aggregation of sampled and not sampled areas.
Territory RNS of Not Sampled areas: long form to all the households.
Territory RS referred to Sampled areas: long form to a sample of households.
pˆ R  γpˆ S  1  γ p NS
R  R S  R NS
γ  NS N
Sub-population of R elegible for drawing the LF sample.
100%
1
CVpˆ R   γ
CVpˆ a 
K
95%
90%

γ 
 100
red%  1 
K

85%
80%
100
0
400
500
1000
1500
2000
2500
3000
Numero of
di aree
K K
Number
areas
LFc_60% ─ γ=0.6
LFc_70% ─ LFc_100%
─LFc_50%
γ=1 ─ γ=0.7
γ=0.5
34
Percentage expected
reduction of CV