Upload
peregrine-tate
View
227
Download
4
Tags:
Embed Size (px)
Citation preview
United Nations Statistics Division
Benchmarking
Training Workshop on the Compilation of Quarterly National Accounts for Economic Cooperation Organization Member Countries
8-11 March 2015Tehran, Islamic Republic of Iran
Outline
What is benchmarking? Purpose of benchmarking Benchmarking methods Benchmarking software for QNA Conclusions Questions
2
What is benchmarking?
Benchmarking is a statistical technique to correct inconsistencies between estimates of the same variable obtained from data collected at different frequencies to produce a consistent time series• Example: quarterly and annual value-added
estimates which are compiled using different data sources such as quarterly and annual surveys
It is usually done retrospectively as annual benchmark data are available sometime after the quarterly data
3
Difference in coverage and sample• Annual survey has broader coverage and a more
representative sample• Differences in frame and sample size may exist
Difference in definitions and variables• Output replaces value added for growth measures
Accounting methods• Different quarterly and annual accounting methods
Estimation method, non-response treatment, respondent error, sampling error...
4
What is benchmarking?
Reasons for differences between quarterly and annual data sources
What is benchmarking?
Aspects of benchmarking• Quarterization of annual data to construct time
series of historical QNA estimates (back series) and revise preliminary QNA estimates to align them with new annual estimates when they become available
• Extrapolation to update the series from movements in the indicator for the most current period (forward series)
5
Purpose of benchmarking
Combines the relative strengths of low-frequency data (say, annual data) and high-frequency data (say, quarterly data) while preserving as much as possible the short-term movements
Creates a coherent high-frequency data series by correcting the difference between benchmark and indicator values (indicator bias)• Ensures benchmark to indicator (BI) ratio becomes 1
Example: Adjust indicator so that
6
,q yI 4
,1
BI 1y
q yq
A
I
Purpose of benchmarking
Ensures, for forward series, that the sum of the four quarters of the current year is as close as possible to the unknown future annual data
Improves quality of national accounts data in terms of comparability and coherence
7
Benchmarking methods
Numerical methods• Pro rata distribution method• Proportional Denton method
Statistical modelling methods• ARIMA-model-based methods• General least-squares regression models
Cholette Dagum Chow-Lin
8
Pro rata distribution method
For any benchmark year, distribute annual value in direct proportion to the quarterly values
,, ,
, ,
q y yq y y q y
q y q yq q
I AX A I
I I
9
Benchmarking methods
,q yX is the level of the QNA estimate for quarter q of year y
,q yI is the level of the indicator for quarter q of year y
yA is the level of the annual benchmark value for year y
,
y
q yq
A
Iis the BI ratio
Pro rata distribution method
Advantages
Easy to compute and interpret
No special software needed
Quarterly estimates can be derived each year independently
Estimates are well aligned to benchmark value and are fairly reliable when BI ratios are stable
Disadvantages
Smoothens quarterly estimates only within a year
Concentrates bias in one quarter and cause abrupt change (“step problem”)
Not recommended for longer time series
10
Benchmarking methods
Example of pro rata distribution method
11
Yellow cell shows step problem caused by change in BI ratioSource: IMF QNA manual (2001)
Benchmarking methods
Pro rata distribution method – step problem
Source: IMF QNA manual (2001)
12
Benchmarking methods
Pro rata distribution methodBenchmark-to-indicator ratio
13
Benchmarking methods
Source: IMF QNA manual (2001)
Proportional Denton method Goal: Find new estimates with minimal
deviation from original indicator series
1 4
2
1
( ,.. ., )2 1
minT
Tt t
X X Xt t t
X X
I I
1,...4 ,..,t T
2
, 1,...,T
t yt
X A y
under restriction (for flow series):
14
Benchmarking methods
t is time
tX is the derived QNA estimate for quarter t
yA is the annual data for year y
is the last year for which an annual benchmark is available
tI is the level of the indicator for quarter t
T is the last quarter for which quarterly source data are available
Proportional Denton method
Avoids the step problem seen in the pro rata distribution method by implicitly constructing from the annual observed BI ratios a time series of quarterly benchmarked QNA estimates-to-indicator (quarterly BI) ratios that• Is as smooth as possible• Average to the annual BI ratios for each year for the
back series• Are kept constant and equal to the ratio for the last
quarter of the last benchmark year Ensures quarterly growth rates are adjusted by
relatively similar amounts
15
Benchmarking methods
Example of proportional Denton method
16
Benchmarking methods
Source: IMF QNA manual (2001)
No big jump in
period-to-period
change
Proportional Denton methodSolution to step problem
17
Benchmarking methods
Source: IMF QNA manual (2001)
Proportional Denton methodBenchmark-to-indicator ratio
18
Benchmarking methods
Source: IMF QNA manual (2001)
Proportional Denton method
Method requires that the indicator contains positive values only
For series with zeros but no negative values• Replace zeroes with values infinitesimally close to zero
For series with both negative and positive values and are derived as the difference between two non-negative series (for example, changes in inventories)• Apply method to opening and closing inventory levels, or• Turn the indicator into positive series by
Adding a large constant to all periods Doing benchmarking Removing the large constant from the results
19
Benchmarking methods
Proportional Denton method
Impact on back series • QNA quarter-to-quarter growth rates differ from those in the
indicator• May introduce new turning points, or • May change timing of turning points• These changes are desirable result of incorporating
information in the annual data Impact on forward series
• Quarter-on-quarter growth rates are identical to those in the indicator
• But, annual growth rate for the first year of the forward series differs from that of the source data
20
Benchmarking methods
Proportional Denton method
Forward series estimates are of great interest to users
Forward series estimates can be improved to reduce size of future revisions by using enhanced version of proportional Denton method which forecast annual BI ratios by incorporating information on past systematic movements in the annual BI ratios• Example
A study of movements in annual BI ratios show that the indicator on average understates annual rate of growth by 2.0%
Adjust annual BI ratio for forward series by 2.0%
21
Benchmarking methods
Example of enhanced proportional Denton method
22
Benchmarking methods
Source: IMF QNA manual (2001)
Statistical modelling methods
Take into account any known information about the stochastic properties of the series being benchmarked
Were mostly actually proposed to improve survey estimates where the survey design may provide information about the stochastic (sampling) properties of the series
In QNA, information about the stochastic properties of the series is usually non-existent and non-sampling errors may be more important than sampling errors
23
Benchmarking methods
Statistical modelling methods
Unlike the Denton method, these methods may over-adjust the series by interpreting as errors, and thus removing, true irregular movements that do not fit the regular patterns of the statistical model
Models may mis-specify the error terms, resulting in inaccurate estimates
24
Benchmarking methods
BENCH ECOTRIM XLPBM
25
Benchmarking software for QNA
BENCH
Was developed by Statistics Canada Runs under Unix/DOS operating systems Allows users to choose among many alternative
options offered by the Cholette-Dagum regression-based model
Requires input data to be prepared in pre-formatted text files
26
Benchmarking software for QNA
ECOTRIM
Was developed by Eurostat Runs in Windows Provides interactive and batch support Allows input data to be prepared in text files or
spreadsheets Allows validation of results using graphical
tools Has option for Denton method
27
Benchmarking software for QNA
XLPBM
Was recently developed by IMF Statistics Department
Comprises a Microsoft Excel add-in function with options for the enhanced proportional Denton and Cholette-Dagum methods
Is particularly useful for QNA processing systems based on spreadsheets.
28
Benchmarking software for QNA
Benchmarking is needed to derive QNA series that are• Temporally consistent with the ANA benchmarks• Preserve as much as possible the movements in the
quarterly indicators• Provide accurate extrapolations for the current year
There are a number of methods to perform benchmarking
The pro rata distribution method is not recommended due to the step problem
The proportional Denton method is robust, simple and well suited for large-scale applications
A number of software are available to do benchmarking
29
Conclusions
What method is your country using to do benchmarking in the compilation of quarterly GDP estimates? Why?
Is the benchmarking method described in the national accounts methodological notes?
30
Questions
31
Thank you