Transcript Selective Editing Strategies for the U.S. Census Bureau Trade Statistics Programs María García, Alison Gajcowski, and Andrew Jennings U.S.

Selective Editing Strategies for
the U.S. Census Bureau Trade
Statistics Programs
María García, Alison Gajcowski, and
Andrew Jennings
U.S. Census Bureau
UNECE Work Session on Statistical Data Editing, Bonn, Germany
Trade Statistics Programs:
Background
 Official source for the U. S.
merchandise trade statistics
 Monthly publications
- Import/Export statistics
- U.S. balance of trade
 Collected by Customs and Border
protection
2
Current Data Processing
 Approximately 3.4 million import
and 1.8 million export records
per month
 Edit master
 Majority of edit failures are
automatically imputed
 Fewer than 0.5 percent of
monthly records are “rejects”
3
Selective Editing
 Prioritize manual review of edit
failing records
 Mandate: All rejected records
are to be reviewed
4
Methods Considered
 Flagging records


Flagi  max(Vi,rep ,Vi,est )  max(Qi,rep , Qi,est ) * Errori *Weighti
 Effect of changes on totals
Diffi  abs (Vi,rep  Vi,est )/ Total (V )* Errori *Weighti
 Adapted from Latouche, Berthelot (1992)
5
Methods Considered
 Hidiroglou-Berthelot method (1986)
- Applied to unit price ratios
- Series of transformation on data
- Use simple statistics
 Hidiroglou-Berthelot & effect of
changes on totals
(Jäder and Norberg, 2005)
6
Application
 Archived raw and “clean” data from
2004 export transactions
 Data adjustments
 Unit price
 Totals, medians, quartiles, computed
using the whole data set
 Two types of weights
 Data groups
7
Application
 Flagging records
– selected records that must be
examined at the commodity level
– 20% review level
 Effect on totals, Hidiroglou-Berthelot
– applied to data groups by
commodities
8
Summary
 Future work
- Production testing, adjustment
of weights, adding new weights
- Review all rejected records?
 Issue
- How do we evaluate/compare
the effectiveness of the different
score functions?
9
Thank You!
Maria M. Garcia
U.S. Census Bureau
[email protected]
10