Embed Size (px)
Citation preview
Selective Editing Strategies for the U.S. Census Bureau Trade
Statistics Programs
María García, Alison Gajcowski, and Andrew Jennings
U.S. Census Bureau
UNECE Work Session on Statistical Data Editing, Bonn, Germany
2
Trade Statistics Programs: Background
Official source for the U. S. merchandise trade statistics
Monthly publications- Import/Export statistics- U.S. balance of trade
Collected by Customs and Border protection
3
Current Data Processing
Approximately 3.4 million import and 1.8 million export records per month
Edit master Majority of edit failures are
automatically imputed Fewer than 0.5 percent of
monthly records are “rejects”
4
Selective Editing
Prioritize manual review of edit failing records
Mandate: All rejected records are to be reviewed
5
Flagging records
Effect of changes on totals
Adapted from Latouche, Berthelot (1992)
, , , ,max( , ) max ( , ) * *i i rep i est i rep i est i iFlag V V Q Q Error Weight
, ,( ) / ( ) * *i i rep i est i iDiff abs V V Total V Error Weight
Methods Considered
6
Methods Considered Hidiroglou-Berthelot method (1986)
- Applied to unit price ratios
- Series of transformation on data
- Use simple statistics
Hidiroglou-Berthelot & effect of changes on totals
(Jäder and Norberg, 2005)
7
Archived raw and “clean” data from 2004 export transactions
Data adjustments
Unit price
Totals, medians, quartiles, computed using the whole data set
Two types of weights
Data groups
Application
8
Flagging records
– selected records that must be examined at the commodity level
– 20% review level
Effect on totals, Hidiroglou-Berthelot
– applied to data groups by commodities
Application
9
Future work
- Production testing, adjustment of weights, adding new weights
- Review all rejected records? Issue
- How do we evaluate/compare the effectiveness of the different score functions?
Summary
