Joint UNECE/Eurostat Meeting on Population and Housing Censuses (13-15 May 2008) Sample results expected accuracy in the Italian Population and Housing Census Giancarlo Carbonetti,
Download ReportTranscript Joint UNECE/Eurostat Meeting on Population and Housing Censuses (13-15 May 2008) Sample results expected accuracy in the Italian Population and Housing Census Giancarlo Carbonetti,
Joint UNECE/Eurostat Meeting on Population and Housing Censuses (13-15 May 2008) Sample results expected accuracy in the Italian Population and Housing Census Giancarlo Carbonetti, Marco Fortini Istat – Italian National Statistical Institute General Censuses Directorate May 13th 2008 Joint UNECE Eurostat Meeting Outline Introduction Some aspects related to the use of samples of households for long form enumerations Sampling strategies Simulation study Some results Conclusions 2 Joint UNECE Eurostat Meeting Introduction - 1 Main critical issue of the last Census Huge organizational (and economical) effort of Municipal Census Offices sudden and time-concentrated increase of workload for largest municipalities, massive network of enumerators and coordinators to be trained and managed lack of adequately skilled resources, high turn over rates Main objectives for the next Census to improve the census operations efficiency to reduce the municipalities workload to keep an high level of quality 3 Joint UNECE Eurostat Meeting Introduction - 2 Innovations proposed to reach the objectives the use of population registers mail out of census forms mixed mode of data collection mainly based on mail and web Expected consequences with the innovations the increase of “back office” work the reduction of enumerators number (“front office” work) How it is possible increasing the response rates A proposal: the use of a “short form” version of the questionnaire is considered to reach high response rates. 4 Joint UNECE Eurostat Meeting Introduction - 3 Consequences of the use of short form increasing the response rates reducing as much as possible the response time delay This approach risks information loss!!! How to preserve the richness of the census information by a selection of a sample of households to which a “long form” version of the questionnaire is supplied Strategy: the simultaneously use of short and long forms. 5 Joint UNECE Eurostat Meeting Some aspects related to the use of samples of households for long form enumeration - 1 Which type of information can be surveyed by means of a sample of long forms and which must be collected on the whole population? The overall set of census variables is partitioned into two subsets the demographic variables (gender, date of birth, marital status, nationality, …) the remaining variables (educational level, occupational status, commuting) Short form accounts for merely the first set of variables whereas long form accounts for the whole set 6 Joint UNECE Eurostat Meeting Some aspects related to the use of samples of households for long form enumeration - 2 Which is the population municipality threshold under which the sampling strategy cannot be adopted? An option we are taking into consideration is to sample in municipalities with more than 5,000 inhabitants long forms will be submitted to a sample of households short forms will be administered to remaining households In municipalities smaller than 5,000 inhabitants long forms will be submitted to the whole population 7 Joint UNECE Eurostat Meeting Some aspects related to the use of samples of households for long form enumeration - 3 Which domains have to be considered to plan the sample and to produce accurate estimates? New “census domains” have been defined an appropriate methodology was adopted to build up census domains by aggregating the smallest census areas the new “areas” are referred to sub-municipal level Accuracy of sampling estimates for different territorial levels a similar precision is expected for estimates among areas higher precision is expected for larger territorial reference (from sub-municipal to nationwide level) 8 Joint UNECE Eurostat Meeting Some aspects related to the use of samples of households for long form enumeration - 4 Which statistical methodology performs the most accurate estimation? … in terms of … sampling design use of appropriate lists efficient estimation methods sampling error assessment The answer to this question is the aim of the study of which some results will be presented. 9 Joint UNECE Eurostat Meeting Sampling strategies Two different sampling designs have been tested Simple Random Sampling of HOUseholds from Administrative Registers (SRSHOU) managed by municipalities Area Frame Sampling based on a Simple Random Sampling of ENumeration Areas (SRSENA) which implies a complete data collection of households dwelling in the selected enumeration areas (from Digital Geocoded Database) Different studies have been conducted To compare the two different approaches (with a sampling ratio of about one third of the whole population considered) To evaluate in the SRSHOU the improvement of the estimates precision for increasing sampling ratio (10%, 15%, 20%, 33%) To introduce some stratifications of the units involved 10 Joint UNECE Eurostat Meeting Simulation study - 1 The sampling strategies were compared to each other through Monte Carlo sampling replications (carried out on 2001 Italian Census data) in order to assess the sampling error defined by the coefficient of variation (CV) which represents an accuracy measurement of the sampling estimates. Main features of the sampling designs Domains: the “new areas” referred to sub-municipal districts Target variables: “variables” related to cross-classification of educational level, employment status and commuting with demographic variables Sampling units: “households” or “enumeration area” Estimator: “calibrated estimators” by using final weights properly modified so to make the sample more representative 11 Joint UNECE Eurostat Meeting Simulation study - 2 Because of the strong differences among the Italian municipalities, 40 of them with different population size and from different regions of Italy were considered Geographical area Classes of population size (a) more than 100,000 Total 10,000-20,000 20,000-100,000 North 4 6 6 16 Center 2 3 3 8 South 4 6 6 16 Total 10 15 15 40 (a) It has been considered the legal (official) population date referred to the 2001 Census of Population. 12 Joint UNECE Eurostat Meeting Simulation study - 3 Amount of units involved by the simulation study Sampled Units Areas 497 Enumeration areas 30,890 Universe % 3,347(*) 14.85% 382,534 8.08% Households 2,243,511 21,810,676 10.29% Individuals 5,537,582 56,594,021 (*) Estimated number 13 9.78% Scatter plot of cv and p (estimates) for each census area. SRSHOU design (sampling ratio=33%). City of Perugia. 100 90 80 70 60 cv% Joint UNECE Eurostat Meeting 50 40 30 20 10 0 0 3% 5 2% 10 15 20 25 30 p% 1% 14 35 40 45 50 55 Joint UNECE Eurostat Meeting Distribution of median cv for classes of p for SRSHOU design and SRSENA design (both with sampling ratio=33%). Comparison of 4 municipalities. Classes of p Milano (111 areas) Bologna (32 areas) Padova (18 areas) Livorno (13 areas) SRSHOU SRSENA SRSHOU SRSENA SRSHOU SRSENA SRSHOU SRSENA < 0.05% 97.78 94.12 96.52 94.31 99.65 98.34 101.61 0.05%├0.1% 51.61 51.59 50.69 52.06 0.1%├0.25% 34.67 THIS50.67 IS DUE THE54.13 49.54 TO 51.70 34.92 35.00 35.20 35.37 36.03 CLUSTER EFFECT 102.21 35.08 35.67 0.25%├0.5% 22.96 24.38 24.17 24.73 25.58 26.45 23.70 24.37 0.5%├1% 16.86 18.71 16.85 18.32 16.95 17.81 17.16 18.72 1%├2.5% 10.61 12.21 10.74 11.95 11.07 12.00 11.34 12.90 2.5%├5% 7.02 8.53 7.07 8.25 7.35 8.48 7.17 9.00 5%├10% 4.84 5.97 4.75 5.74 5.05 5.85 4.88 6.39 10%├15% 3.17 4.41 3.09 4.09 3.19 4.37 3.06 4.82 15%├20% 2.44 3.46 2.38 3.12 2.44 3.14 2.44 3.39 20%├30% 1.89 2.61 1.92 2.48 2.08 2.73 2.05 2.88 ≥ 30% 1.35 1.78 1.32 1.60 1.39 1.72 1.40 2.00 15 Joint UNECE Eurostat Meeting Loss of efficiency (in terms of CV for classes of p) of estimation with SRSENA with respect to SRSHOU design (both with sampling ratio=33%). Comparison of 4 municipalities. [CV(SRSHOU_s.r. 33%)-CV(SRSENA_s.r. 33%)] Classes of p Milano (111 areas) Bologna (32 areas) Padova (18 areas) Livorno (13 areas) < 0.05% 3.65 2.21 1.31 0.60 0.05%├0.1% 0.03 1.13 -2.43 -1.37 0.1%├0.25% -0.25 -0.20 -0.66 -0.59 0.25%├0.5% -1.42 -0.56 -0.87 -0.68 0.5%├1% -1.85 -1.47 -0.87 -1.56 1%├2.5% -1.60 -1.22 -0.93 -1.56 2.5%├5% -1.51 -1.18 -1.13 -1.83 5%├10% -1.13 -0.99 -0.80 -1.51 10%├15% -1.24 -1.01 -1.18 -1.76 15%├20% -1.02 -0.73 -0.70 -0.95 20%├30% -0.72 -0.56 -0.65 -0.82 ≥ 30% -0.43 -0.29 -0.33 -0.60 16 Joint UNECE Eurostat Meeting Distribution of median cv for classes of p. Comparison of 4 different sampling ratios with the SRSHOU design. Classes of p sampling ratio= 10% sampling ratio= 15% sampling ratio= 20% sampling ratio= 33% 170 areas 140 areas 111 areas 204 areas < 0.05% 220.51 157.20 142.00 98.21 0.05%├0.1% 111.48 87.22 74.20 51.14 0.1%├0.25% 75.57 59.83 49.97 34.76 0.25%├0.5% 50.70 39.92 33.97 23.44 0.5%├1% 35.54 28.10 23.74 16.56 1%├2.5% 23.62 18.56 15.33 10.68 2.5%├5% 15.50 12.29 10.09 7.04 5%├10% 10.46 8.26 6.93 4.82 10%├15% 7.06 5.40 4.40 3.13 15%├20% 5.57 4.27 3.54 2.42 20%├30% 4.50 3.48 2.84 1.93 ≥ 30% 3.20 2.42 1.94 1.34 17 Joint UNECE Eurostat Meeting Gain of efficiency (in terms of CV for classes of p) of estimation with SRSHOU design by increasing sampling ratio from 10% to 33% . [CV(SRSHOU_s.r. 10%)-CV(SRSHOU_s.r. N%)]x100/[CV(SRSHOU_s.r. 10%)] Classes of p increasing s.r. from 10% to 15% increasing s.r. from 10% to 20% increasing s.r. from 10% to 33% < 0.05% 28.71 35.60 55.46 0.05%├0.1% 21.76 33.44 54.13 0.1%├0.25% 20.83 33.88 54.00 0.25%├0.5% 21.26 33.00 53.77 33.20 53.40 1%├2.5% 20.93 Gain between 21-23 percent 21.42 54.78 2.5%├5% 20.71 35.10 Gain between 33-38 percent 34.90 5%├10% 21.03 33.75 54.58 Gain between 53-58 percent 53.92 10%├15% 23.51 37.68 55.67 15%├20% 23.34 36.45 56.55 20%├30% 22.67 36.89 57.11 ≥ 30% 24.38 39.38 58.13 0.5%├1% 18 Joint UNECE Eurostat Meeting Distribution of median cv for five classes of p and three classes of area (according to population size). Comparison of 4 different sampling ratios with the SRSHOU design. Classes of p 0.1%├0.25% 0.5%├1% 2.5%├5% 10%├15% 20%├30% Population by area (thousands) Sampling ratio 10% 15% 20% 33% <10 90.00 71.27 59.48 40.20 10├12 76.23 60.03 50.45 34.41 ≥ 12 66.65 53.04 43.51 30.58 < 10 43.11 33.50 28.97 19.53 10├12 35.08 27.46 22.99 16.48 ≥ 12 31.25 24.95 20.97 14.85 < 10 19.12 14.68 12.22 8.25 10├12 15.58 12.36 9.89 7.08 ≥ 12 14.00 10.98 9.06 6.35 < 10 8.78 6.44 5.22 3.67 10├12 7.00 5.46 4.41 3.13 ≥ 12 6.29 4.79 3.89 2.83 < 10 5.46 4.16 3.44 2.27 10├12 4.57 3.42 2.94 2.01 ≥ 12 4.05 3.20 2.59 1.77 19 Median CV for some classes of p and for three classes of area (according to population size). Comparison of 4 different sampling ratios (s.r.) with the SRSHOU design. Graph referred to area size less than 10,000 inhabitants. s.r.=15% 130 s.r.=20% 120 s.r.=33% 110 100 90 80 70 Area size<10,000 60 50 40 30 20 10 % 30 > 0% -3 -2 5% 0% 20 classes of p 15 10 -1 -1 0% % 5 -5 2. 5 .5 % -2 % 1 -1 0. 5 -0 0. 25 -0 .2 5 % .1 % 20 .5 % 0 -0 66.7 53.0 43.5 30.6 31.3 25.0 21.0 14.9 14.0 11.0 9.1 6.4 6.3 4.8 3.9 2.8 4.1 3.2 2.6 1.8 0. 1 76.2 60.0 50.5 34.4 35.1 27.5 23.0 16.5 15.6 12.4 9.9 7.1 7.0 5.5 4.4 3.1 4.6 3.4 2.9 2.0 ≥ 12 % 90.0 71.3 59.5 40.2 43.1 33.5 29.0 19.5 19.1 14.7 12.2 8.3 8.8 6.4 5.2 3.7 5.5 4.2 3.4 2.3 10├12 0. 05 10% 15% 0.1-0.25% 20% 33% 10% 15% 0.5-1% 20% 33% 10% 15% 2.5-5% 20% 33% 10% 15% 10-15% 20% 33% 10% 15% 20-30% 20% 33% <10 s.r.=10% 140 0. 05 s.r. 150 Area size (thousands) < Classes of p median cv% Joint UNECE Eurostat Meeting Median CV for some classes of p and for three classes of area (according to population size). Comparison of 4 different sampling ratios (s.r.) with the SRSHOU design. Graph referred to area size between 10,000 and 12,000 inhabitants. s.r.=15% 130 10% 90.0 76.2 66.7 15% 71.3 60.0 53.0 120 0.1-0.25% 20% 59.5 50.5 43.5 110 33% 40.2 34.4 30.6 100 10% 43.1 35.1 31.3 15% 33.5 27.5 25.0 90 0.5-1% 20% 29.0 23.0 21.0 80 33% 19.5 16.5 14.9 70 10% 19.1 15.6 14.0 15% 14.7 12.4 11.0 60 2.5-5% 20% 12.2 9.9 9.1 50 33% 8.3 7.1 6.4 40 of 10% of efficiency 8.8 7.0 6.3 terms The gain (in 15% 6.4 5.5 4.8 30 10-15% for census areas with size CV) 20% 5.2 4.4 3.9 20 between with 33% 10,000 3.7 and 3.1 12,000 2.8 10%to census 5.5 4.6 respect areas4.1with 10less 15% 4.2 3.4 3.2 0 20-30% 10,000 is about 14-20 percent. than 20% 3.4 2.9 2.6 Similar for all 33%results 2.3 are2.0obtained 1.8 s.r.=20% s.r.=33% 21 % 30 > 0% -3 -2 5% 0% 20 classes of p 15 10 -1 -1 0% % 5 -5 2. 5 0. 5 .5 % -0 % .2 5 0. 25 -0 0. 1 -0 .1 % % 0. 05 0. 05 < tested sampling ratios. Area size between 10-12,000 .5 % ≥ 12 -2 10├12 % <10 s.r.=10% 140 1 s.r. 150 Area size (thousands) -1 Classes of p median cv% Joint UNECE Eurostat Meeting Median CV for some classes of p and for three classes of area (according to population size). Comparison of 4 different sampling ratios (s.r.) with the SRSHOU design. Graph referred to area size more than 12,000 inhabitants. s.r.=15% 130 10% 90.0 76.2 66.7 15% 71.3 60.0 53.0 120 0.1-0.25% 20% 59.5 50.5 43.5 110 33% 40.2 34.4 30.6 100 10% 43.1 35.1 31.3 15% 33.5 27.5 25.0 90 0.5-1% 20% 29.0 23.0 21.0 80 33% 19.5 16.5 14.9 70 10% 19.1 15.6 14.0 15% 14.7 12.4 11.0 60 2.5-5% 20% 12.2 9.9 9.1 50 33% 8.3 7.1 6.4 40 of 10% of efficiency 8.8 7.0 6.3 terms The gain (in 15% 6.4 5.5 4.8 30 10-15% for census areas with size CV) 20% 5.2 4.4 3.9 20 to more than 12,000 with respect 33% 3.7 3.1 2.8 10 10%areas 5.5with4.6less 4.1 census than 10,000 s.r.=20% s.r.=33% % 30 > 0% -3 -2 5% 0% 20 classes of p 15 10 -1 -1 0% % 5 -5 2. 5 0. 5 .5 % -0 0. 25 -0 .2 5 % .1 % -0 22 0. 1 0. 05 < 0. 05 % 15% 4.2 3.4 3.2 0 20-30% is about 22-28 percent. As before, 20% 3.4 2.9 2.6 similar33%results obtained for all 2.3 are2.0 1.8 tested sampling ratios. Area size>12,000 .5 % ≥ 12 -2 10├12 % <10 s.r.=10% 140 1 s.r. 150 Area size (thousands) -1 Classes of p median cv% Joint UNECE Eurostat Meeting Joint UNECE Eurostat Meeting Distribution of the estimates referred to areas larger than 12,000 inhabitants for classes of cv. Comparison of percentage frequencies for 4 different sampling ratios with the SRSHOU design. Classes of coefficient of variation % < 2% 2%├5% 5%├10% 10%├20% 20%├50% 50%├100% 100%├200% ≥ 200% Sampling ratio 10% 15% 20% 0.57 2.69 HA – high accuracy 13.04 17.53 33% 6.39 13.14 18.40 23.64 18.02 26.28 28.64 29.14 30.16 MA – medium accuracy 25.09 19.71 23.54 16.20 16.75 13.32 7.21 5.69 3.44 3.65 2.00 1.61 1.03 0.95 - 16.18 9.32 4.40 LA – low accuracy 2.25 23 Distribution of the estimates referred to areas larger than 12,000 inhabitants for classes of cv. Comparison of percentage frequencies for 4 different sampling ratios with the SRSHOU design - 2 Sampling ratio Classes of cv% 10% 15% 20% 33% < 10% 29.80 38.25 51.07 65.42 10%├50% 54.23 49.87 40.29 29.52 ≥ 50% 15.97 11.89 8.64 5.05 70 HA - high accuracy 60 % of estimates Joint UNECE Eurostat Meeting cv%<10% 50 cv% in 10-50% 40 cv%>50% 30 MA - medium accuracy 20 10 LA - low accuracy 0 10% 15% 20% Sampling ratio 24 33% Joint UNECE Eurostat Meeting Estimates of p referred to territory given by aggregation of areas. Generic sampled area a pˆ a X CVpˆ a Territory RS given by aggregation of K sampled areas Percentage expected reduction of CV in RS Territory R given by aggregation of sampled areas and not sampled areas γ NS N pˆ R S X CV pˆ R S 1 CVpˆ a K 1 red% 1 100 K pˆ R X CVpˆ R γ 100 Percentage expected red% 1 K reduction of CV in R Quote of sub-population of R elegible for drawing the LF sample. 25 γ CVpˆ a K Joint UNECE Eurostat Meeting Conclusions - 1 As expected, the most accurate estimates were obtained for: simple random sampling of households from administrative registers largest sampling ratio Better efficiency of estimates for largest areas (>12,000 inhabitants) this result could represent a suggestion for planning the sampling design by defining larger census areas (of about 15,000 people) The estimates referred to large domains given by aggregation of areas show high accuracy the accuracy increases with the domain’s number in case in which a part of the large domain is totally surveyed, the estimates show a further increasing in accuracy 26 Joint UNECE Eurostat Meeting Conclusions - 2 However area frame sampling is only slightly less efficient than SRSHOU, thus it could be adopted where reliable administrative registers are not available Sampling ratio will be chosen considering trade-off between: needed financial savings accuracy required at different territorial domains Further analyses will be conducted on small area estimation techniques to produce more accurate estimates for: smallest territorial levels rare populations 27 Joint UNECE Eurostat Meeting Thank you for your attention and … 28 Joint UNECE Eurostat Meeting … have a good lunch!!! 29 Joint UNECE Eurostat Meeting 30 Joint UNECE Eurostat Meeting Simulation study - 4 Cross-classification cells educational level, employment status, commuting and gender 90 simple estimation cells Calibration constraints defined by cross-classifying gender by age, and gender by marital status Computational algorithm implemented by SAS code for each municipality and for each alternative sampling design: step 1) selection of a sample (of households or enumeration areas) step 2) computation of final weights step 3) estimation of the relative frequency p for each target cell step 4) iteration of steps 1), 2) and 3) for 1,000 sampling replications step 5) computation of sampling distribution mean and standard error for each one of the 90 frequency cells 31 Joint UNECE Eurostat Meeting Evaluation criterion: the coefficient of variation In order to compare the sampling strategies has been considered as evaluation criterion the coefficient of variation CV : σpˆ x cvx 100 Epˆ x which represents an accuracy measurement of the sampling estimates. Consequently, the percentage maximum expected error can be computed: Δ% ≈ 1.96 · CV which is implied (with a probability of 0.95) by the estimation method. The distribution of the empirical CV’s for all the 90 target cells was determined. After having classified the target cells depending on their value p , CV’s distribution related to the cells in the same p group has been studied. 32 Joint UNECE Eurostat Meeting Estimates of p referred to territory given by aggregation of areas. Case 1: aggregation of sampled areas. pˆ a X CVpˆ a Estimate referred to the generic sampled area a ˆ R X p S Estimate referred to the territory RS given by aggregation of K sampled areas W pˆ a aR S where Wa Na N RS 100% 90% CV pˆ R S 80% 1 K CVpˆ a for K>100 → red%>90% 70% 1 100 red% 1 K for K>30 → red%>80% 60% for K>5 → red%>50% 50% 40% 0 500 1000 1500 Riduzione percentuale "attesa" del CV ─ Percentage expected reduction of CV 2000 2500 33 3000 Numero di aree K Percentage expected reduction of CV Number of areas K a Joint UNECE Eurostat Meeting Estimates of p referred to territory given by aggregation of areas. Case 2: aggregation of sampled and not sampled areas. Territory RNS of Not Sampled areas: long form to all the households. Territory RS referred to Sampled areas: long form to a sample of households. pˆ R γpˆ S 1 γ p NS R R S R NS γ NS N Sub-population of R elegible for drawing the LF sample. 100% 1 CVpˆ R γ CVpˆ a K 95% 90% γ 100 red% 1 K 85% 80% 100 0 400 500 1000 1500 2000 2500 3000 Numero of di aree K K Number areas LFc_60% ─ γ=0.6 LFc_70% ─ LFc_100% ─LFc_50% γ=1 ─ γ=0.7 γ=0.5 34 Percentage expected reduction of CV