83
Weighting issues Weighting issues Julian Chow Julian Chow Industrial and Energy Statistics Industrial and Energy Statistics Section Section United Nations Statistics United Nations Statistics Division (UNSD) Division (UNSD) Email: [email protected] Email: [email protected]

Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: [email protected]

Embed Size (px)

Citation preview

Page 1: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Weighting issuesWeighting issues

Julian ChowJulian ChowIndustrial and Energy Statistics SectionIndustrial and Energy Statistics Section

United Nations Statistics Division (UNSD)United Nations Statistics Division (UNSD)Email: [email protected]: [email protected]

Page 2: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

2

Overview – 1st Session

11stst Session – Weighting issues Session – Weighting issues The role of weights in an indexThe role of weights in an index Theory - weights in the Laspreyres Theory - weights in the Laspreyres

formulaformula Determining IIP weights in practiceDetermining IIP weights in practice Weight updatingWeight updating Fixed weight index vs. chained indexFixed weight index vs. chained index

Page 3: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

3

Overview – 2nd Session

22ndnd Session – Missing weights Session – Missing weights Missing weights for the most recent

periods Missing weights for the entire time span

of one component series Discussion

Page 4: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Question?How can the change of

the production level of coca-cola be reflected in IIP?

Page 5: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

5

Example Elementary observation (quantity/value produced by an establishment)

Coca-cola Product

Waters, with added sugar, other sweetening matter or flavoured, i.e. soft drinks Product group

Other non-alcoholic caloric beverages (CPC Ver.2 Sub-class 24990) 4-digit-Industry

Manufacture of soft drinks; production of mineral waters and other bottled waters (ISIC Rev.4 Class 1104)

3-digit industry Manufacture of beverages (ISIC Rev.4 Group 110)

2-digit industry Manufacturing of beverages (ISIC Rev.4 Division 11)

1-digit industry Manufacturing (ISIC Rev.4 Section C.)

Page 6: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

6

IIP StructureTotal IIP

1-digit ISIC

2-digit ISIC

4-digit ISIC

Product groups assigned to one 4 digit ISIC branch

3-digit ISIC

Stage 3: Weights for industry branches – Gross value added at basic prices

Stage 2: Product group weights – Value of output obtained via census/survey

Stage 1: Product weights –Value of output obtained via census/survey

Individual sampled products assigned to one product group

Page 7: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

7

The role of weights in the indexThe role of weights in the index

Weights are used to Weights are used to aggregate seriesaggregate series into higher level into higher level aggregatesaggregates Can be done at different levelsCan be done at different levels Weights have to be chosen accordinglyWeights have to be chosen accordingly

Weights have to reflect the Weights have to reflect the relative importancerelative importance of the of the individual components within the aggregateindividual components within the aggregate

Weights determine the impact that a particular volume Weights determine the impact that a particular volume change will have on the overall indexchange will have on the overall index

Page 8: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

8

More about weightsMore about weights

Over time, establishment production levels shift in Over time, establishment production levels shift in response to economic conditions.response to economic conditions.

Relative importance may changeRelative importance may change Products within a product group Products within a product group Product groups within an industryProduct groups within an industry Lower level industries within higher level aggregatesLower level industries within higher level aggregates

For the IIP to reflect the movements as good as possible, For the IIP to reflect the movements as good as possible, the weights have to reflect these changesthe weights have to reflect these changes

Page 9: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Recap the Laspreyres volume index

Page 10: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

10

Notations usedNotations used

NotationsNotations ppt : t : pricesprices at time at time tt

qqt :t : quantities quantities at time at time tt

Base periodBase period: t=0: t=0 i: units (i.e. products, product groups or

industries) to be aggregated n: number of units (i.e. products, product

groups or industries) to be aggregated.

Page 11: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

11

Laspeyres volume indexLaspeyres volume index

Fixed prices from the Fixed prices from the base base periodperiodHow much more would the value of basket be in How much more would the value of basket be in

the current period if the price in the current period the current period if the price in the current period is the same in the base period?"is the same in the base period?"

100

period in thebasket theof Value

price period base using

period in thebasket theof Value

1

00

1

0

n

iii

n

i

tii

qp

qp

base

current

Page 12: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

12

Laspeyres volume index – Laspeyres volume index – another formanother form

1001001

00

1

00

1

0

:0

n

ii

i

ti

n

iii

n

i

tii

tLaspeyres w

q

q

qp

qpI

n

iii

iii

qp

qpwwhere

1

00

000 The value share (weight) at period 0

prices and quantities for unit i

Volume index formulae may be rewritten so that indices Volume index formulae may be rewritten so that indices may be constructed using values instead of pricesmay be constructed using values instead of prices

Page 13: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

13

Weights in Laspreyres formulaWeights in Laspreyres formula

1001001

1

00

1

0

:0

n

i

tin

1i

0i

0i

0i

n

iii

n

i

tii

tLaspeyres q

qp

p

qp

qpI

n

i i

ti

n

iii

iin

iii

n

i

tii

tLaspeyres q

q

qp

qp

qp

qpI

10

1

00

00

1

00

1

0

:0 100100

Price weights

Value weights

quantities

Quantity relatives

Page 14: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Determining IIP weighting data in practice

Page 15: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

15

Example Elementary observation (quantity/value produced by an establishment)

Coca-cola Product

Waters, with added sugar, other sweetening matter or flavoured, i.e. soft drinks Product group

Other non-alcoholic caloric beverages (CPC Ver.2 Sub-class 24990) 4-digit-Industry

Manufacture of soft drinks; production of mineral waters and other bottled waters (ISIC Rev.4 Class 1104)

3-digit industry Manufacture of beverages (ISIC Rev.4 Group 110)

2-digit industry Manufacturing of beverages (ISIC Rev.4 Division 11)

1-digit industry Manufacturing (ISIC Rev.4 Section C.)

Page 16: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

16

Practical steps for selecting and determining weights

Determine sampling weights at the establishment and product level

Determine weights for individual sample products

Determine weights for the product groups Determine weights for the industry groups

Page 17: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

17

Sampling in the IIP: Example A random sample of establishments

Initially from the business register as part of the product survey goods (e.g. PRODCOM) (e.g. 25,000 establishments in total)

Subset of these sampled for the IIP (e.g. 7,000 establishment sampled)

A random sample of products from sampled establishment Again, using information provided in the product survey (e.g Results

in 9,000 product-establishment pairs)

A purposive sample of elementary observation from the sampled product-contributor pairs Undertaken using judgement of respondent but scrutinised by

subject expert

Page 18: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

18

Sampling weights at the establishment and product level

The associated weights at the establishment and product level to obtain the value/output of a particular products depend on the sampling scheme

If probability sampling techniques are used, the inverse of the sampling fractions are used as the weights

We are not going to discuss sampling weights in details since this is a topic of survey sampling

This leaves us with three fundamental level of weights in the IIP compilation Product weights Product group weights Weights for industry branches

Page 19: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

19

IIP StructureTotal IIP

1-digit ISIC

2-digit ISIC

4-digit ISIC

Product groups assigned to one 4 digit ISIC branch

3-digit ISIC

Stage 3: Weights for industry branches – Gross value added at basic prices

Stage 2: Product group weights – Value of output obtained via census/survey

Stage 1: Product weights –Value of output obtained via census/survey

Individual sampled products assigned to one product group

Page 20: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

20

Product weights

Reflect the relative importance of a particular product in the product group E.g relative importance of coca-cola in the product ‘soft drinks’

Share of value of output should be used to weight each product in the product group.

The product weights are generally obtained via the conduct of product censuses or surveys.

Page 21: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

21

Product weights

Product sales, though, are sometimes used in lieu of value of output as a weighting variable at this level of the index structure.

Value of output - work-in-progress

- output produced this period entered into inventory + inventory produced in the past sold in this period

= product sales (value of output sold)

Page 22: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

22

IIP StructureTotal IIP

1-digit ISIC

2-digit ISIC

4-digit ISIC

Product groups assigned to one 4 digit ISIC branch

3-digit ISIC

Stage 3: Weights for industry branches – Gross value added at basic prices

Stage 2: Product group weights – Value of output obtained via census/survey

Stage 1: Product weights –Value of output obtained via census/survey

Individual sampled products assigned to one product group

Page 23: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

23

Product group weights

Share of value of output (or proxies thereof) by product group within its ISIC class

These “values of output” allow product groups to be weighted together (combined) and reflect the relative importance of each product group within an ISIC class. E.g relative importance of soft drinks in ‘Other non-alcoholic

beverages (CPC Ver.2 24990)

Page 24: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

24

Product group weights

Each product group is assigned to just one ISIC 4-digit industry.

Sources The product group weights are generally

obtained via the conduct of product censuses or surveys.

Page 25: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

25

IIP StructureTotal IIP

1-digit ISIC

2-digit ISIC

4-digit ISIC

Product groups assigned to one 4 digit ISIC branch

3-digit ISIC

Stage 3: Weights for industry branches – Gross value added at basic prices

Stage 2: Product group weights – Value of output obtained via census/survey

Stage 1: Product weights –Value of output obtained via census/survey

Individual sampled products assigned to one product group

Page 26: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

26

Industry weights

Share of gross value added (GVA) at basic prices by industry of all industries in-scope of industrial production.

GVA at basic prices=Value of output – intermediate consumption + subsidy receivable on products

– tax payable on products

Page 27: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

27

Industry weights

Using value of output as weight is not suitable Introduce distortion by giving a higher

weight to any industry using intermediary goods and services

Double count intermediary goods and services in the final aggregate

Page 28: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

28

Industry weights

GVA vs NVA Net value added (NVA) = Gross value added (GVA) – consumption of fixed capital (depreciation)

Why select GVA, not NVA? Measure of consumption of fixed capital is quite difficult to

observe GVA refers more to supply side considerations to meet final

demand, including gross capital formation. Whereas NVA is more meaningful for an income approach in

measure welfare and living standards

Page 29: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

29

Industry weights

GVA should be used as weights starting from the 4-digit level of ISIC

Sources Such information is available as a result of

annual national accounts compilation. However, for some countries, it requires the use

of other comprehensive data sources, such as industry survey or economic census to obtain weights for lower levels of ISIC.

Page 30: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

30

Summary so farTotal IIP

1-digit ISIC

2-digit ISIC

4-digit ISIC

Product groups assigned to one 4 digit ISIC branch

3-digit ISIC

Stage 3: Weights for industry branches – Gross value added at basic prices

Stage 2: Product group weights – Value of output obtained via census/survey

Stage 1: Product weights –Value of output obtained via census/survey

Individual sampled products assigned to one product group

Page 31: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

31

Calculating weightsCalculating weights

VV : : Absolute weight Absolute weight (value)(value)

ww : : Relative weight Relative weight Base periodBase period: t=0: t=0 i: products, product

groups or industries to be aggregated

n: Set of all products, product groups or industries to be aggregated.

n

ii

ii

V

Vw

1

0

00

n

iiw

1

0 1

Weights formula

By consequence

Page 32: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

32

ExampleSuppose the product group “Other non-alcoholic caloric beverages (CPC Ver.2 24990) contains the following product with

Soft drinks (output value =70) Non-alcoholic beverages not containing (output value =20) Non-alcoholic beverages containing milk fat (output value =10)

Product weights within the product group are Soft drinks [weights = 70/(70+20+10)=0.7] Non-alcoholic beverages not containing [weights = 20/(70+20+10)=0.2] Non-alcoholic beverages containing milk fat

[weights = 10/(70+20+10)=0.1]

Page 33: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Weight updating

Page 34: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

34

Why updating the weights?

Reflect changing structure in the economyOver time production level shifts in response to

economic situationsExample

Smart phone Typewritters

Page 35: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

35

Key issues to consider when updating index weights

The frequency of weight updatesThe method used to incorporate new

weights into index structure

Page 36: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

36

Update frequency

Update frequency of IIP weights can be linked to The need to accurately reflect the current

relative importance of product groups and industries

Data availability The index type used to compile the index

• Laspreyres-type index provide some flexibility regarding update frequency as weights are not derived from the current period

Page 37: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

37

Update frequency - recommendation

Industry weights • Annual• The latest weights available are likely from t-2 or t-3• Frequent update of weights can alleviate the substitution

bias/changing weights problem

Product group weights• at least every 5 years• Less frequent than those for industry level due to resource and data

constraints

Product group • The weights of individual products are updated at the same time as The weights of individual products are updated at the same time as

product groupproduct group

Page 38: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

How to select reference period?

Page 39: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

39

Concepts of reference period

Quantity reference period the period whose volumes appear in the denominators

of the volume relatives used to calculate the index Weight reference period

The period, usually a year, whose values serve as weights for the index

the index reference period The period for which the index is set equal to 100.

The three types of base periods may coincide, but frequently do not.

Page 40: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

40

Weight reference period

Laspeyres-type volume index with weights updated annually The weight reference period will always

be the most recent period (year) for which weights are available

Page 41: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

41

Weight reference period

In circumstances of less frequent weight updates, the weight reference period should therefore possess the following characteristics:

(a) Reasonably normal/stable (i.e. typical of recent and likely future years);

(b) not too distant from the reference period;

(c) clearly identified when analyzing and comparing the index results.

Page 42: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

42

SummarySummary Industry level weightsIndustry level weights

• Annual update Annual update should be carried out.should be carried out.• Should ideally be Should ideally be National Accounts value added figures at National Accounts value added figures at

basic pricesbasic prices – adjustments necessary to make them timely – adjustments necessary to make them timely available.available.

Product group weightsProduct group weights• Should be updated frequently Should be updated frequently at least every 5 yearsat least every 5 years• Obtained by determining the share of value of output, via the Obtained by determining the share of value of output, via the

conduct of product census or surveysconduct of product census or surveys Product weightsProduct weights

• The weights of individual products are updated at the same The weights of individual products are updated at the same time as product grouptime as product group

• Obtained by determining the share of value of output, via the Obtained by determining the share of value of output, via the conduct of product census or surveysconduct of product census or surveys

Page 43: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Fixed weights vs chained index

- Concepts

Page 44: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

44

Fixed base volume indexFixed base volume index

Hold one period as the base period and compare Hold one period as the base period and compare all prices back to this periodall prices back to this period

Calculate movement back to the base period for Calculate movement back to the base period for each successive time pointeach successive time point

Each index in the time series is a comparison from Each index in the time series is a comparison from that period back to the base periodthat period back to the base period

Page 45: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

45

Fixed base volume indexFixed base volume index

Fixed base volume index, from time 0 to 4TABLE 2.12- PRICES AND QUANTITIES FOR SIX COMMODITIES, WITH DIRECT LASPEYRES VOLUME INDICES. Value of basket (£’s) fixed in period 0 prices

Commodity 0 1 2 3 4

A Agricultural commodity qAt pA

0 1.00 1.20 1.00 0.80 1.00

B Energy qBt pB

0 1.00 3.00 1.00 0.50 1.00

C Traditional manufacture qCt pC

0 2.00 2.60 3.00 3.20 3.20

D High-tech goods qDt pD

0 1.00 0.70 0.50 0.30 0.10

E Traditional services qEt pE

0 4.50 6.30 7.65 8.55 9.00

F High-tech services qFt pF

0 0.50 0.40 0.30 0.20 0.10

Total 10.00 14.20 13.45 13.55 14.40

Direct Laspeyres volume index (change from period 0)

100.0 142.0 134.5 135.5 144.0

Page 46: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

46

Direct (Fixed Base) Index

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=10

0.0)

Fixed base volume indexFixed base volume index

Page 47: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

47

Fixed base volume indexFixed base volume indexDirect (Fixed Base) Index

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=10

0.0)

Page 48: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

48

Chained volume indexChained volume index Calculate consecutive period volume index:Calculate consecutive period volume index:

Use a period 0 basket to look at period 0 to 1 changesUse a period 0 basket to look at period 0 to 1 changes

Use a period 1 basket to look at period 1 to 2 changesUse a period 1 basket to look at period 1 to 2 changes

Use a period 2 basket to look at period 2 to 3 changesUse a period 2 basket to look at period 2 to 3 changes

Use a period 3 basket to look at period 3 to 4 changesUse a period 3 basket to look at period 3 to 4 changes

Chain these results together to get a measure of price change from 0 to Chain these results together to get a measure of price change from 0 to 4 4

Page 49: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

49

Chained volume indexChained volume indexConsecutive period indices

80

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=10

0.0)

Page 50: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

5017 March 2010 ECLAC, Santiago

Slide 50 of 89Workshop on Manufacturing Statistics

for ECLAC member states

Chained volume indexChained volume index

CALCULATION OF INDIRECT (CHAINED) LASPEYRES VOLUME INDICES AND COMPARISON WITH DIRECT (FIXED BASE) LASPEYRES VOLUME INDICES

Period

Index 0 1 2 3 4

Direct volume index, period 0 to 1 100.0 142.0 - - -

Direct volume index, period 1 to 2 - 100.0 96.1 - -

Indirect (Chained) volume index, periods 0 to 2 100.0 142.0 136.5 - -

Direct volume index, period 2 to 3 - - 100.0 97.8 -

Indirect (Chained) volume index, periods 0 to 3 100.0 142.0 136.5 133.5 -

Direct volume index, period 3 to 4 - - - 100.0 99.7

Indirect (Chained) volume index, periods 0 to 4 100.0 142.0 136.5 133.5 133.1

Direct (Fixed base, period 0) volume index, periods 0 to 4

100.0 142.0 134.5 135.5 144.0

Page 51: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

51

Chained volume indexChained volume index

Consecutive period indices

80

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=10

0.0)

Page 52: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

52

Chained volume indexChained volume index

Chaining!Chaining!Indirect (Chained) Index

80

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=

100.0

)

Page 53: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

53

ComparisonComparisonComparison between Fixed basket and Chained Indices

80

90

100

110

120

130

140

150

0 1 2 3 4

Period

Ind

ex (

0=

100.0

)

Different resultDifferent result special case of equality is called a special case of equality is called a transitivetransitive index formula index formula

• fixed baskets with fixed baskets with differential weightsdifferential weights never transitive never transitive

Page 54: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

54

Fixed base vs chain indexFixed base vs chain index

Fixed base result more attractive Fixed base result more attractive operationallyoperationally Only one revaluing stepOnly one revaluing step One set of prices (weights at base period)One set of prices (weights at base period)

Why would we chain?Why would we chain? Updating the basket and weights!Updating the basket and weights!

Page 55: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Fixed weights vs chained index

- Recommendations

Page 56: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

56

Fixed weights vs chained index – on weights update

Fixed weight indices Weight structure fixed at particular point Compare volume in period t relative to some fixed base period When base year change, entire historical series are revised as value

for all periods are recalculated using the new base weights

Chain-linked indices Updating of weights and linking two index together to produce a

time series Unlike the fixed weight approach, the chain approach does not re-

calculate the entire historical series Therefore, the index is compiled for a succession of different

segments while keeping the original weights for each past segment fixed

Page 57: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

57

Old recommendationsOld recommendations Use fixed weights for the calculationUse fixed weights for the calculation Update weights every 5 yearsUpdate weights every 5 years Recalculate entire seriesRecalculate entire series

Problem:Problem: New weights may reflect better the movements in the New weights may reflect better the movements in the

current periods, but they are not applicable for past data current periods, but they are not applicable for past data (far from new weight period)(far from new weight period)

• Problem simply shifts to a different periodProblem simply shifts to a different period

Page 58: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

58

New recommendationsNew recommendations

Update weights more frequentlyUpdate weights more frequently Recommended: AnnuallyRecommended: Annually

Do not re-calculate entire seriesDo not re-calculate entire seriesUse chain linking to produce time Use chain linking to produce time

series for IIPseries for IIP

Page 59: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

59

New recommendationsNew recommendations

Chain-linking annually rebased series Chain-linking annually rebased series allows for better reflection of current allows for better reflection of current economic structure in the weights in each of economic structure in the weights in each of the sub-seriesthe sub-series Current period and weight base period Current period and weight base period

are not too far apartare not too far apart Alleviate substitution biasAlleviate substitution bias Provide opportunity to incorporate new Provide opportunity to incorporate new

productsproducts

Page 60: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

60

LinkingLinking

How to link the individual sub-series to obtain longer time How to link the individual sub-series to obtain longer time series?series?

A linking factor has to be determined to link the new series A linking factor has to be determined to link the new series to the existing historical seriesto the existing historical series This factor is then applied to the new (old) series to This factor is then applied to the new (old) series to

convert it to the old (new) base yearconvert it to the old (new) base year

Page 61: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

61

LinkingLinking

The long-term time series are The long-term time series are calculated from a succession of short-calculated from a succession of short-term series with updated weightsterm series with updated weights

• Note: Short-term series can span Note: Short-term series can span any number of periodsany number of periods

Page 62: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

62

Linking options Linking options

Annual overlap, linking factor based on Annual overlap, linking factor based on annual index for years t annual index for years t index of the same year using weights of year t-1 index of the same year using weights of year t-1

One-quarter overlap, linking factor based on One-quarter overlap, linking factor based on index of the first quarter of year t index of the first quarter of year t Index of the same quarter using weights of year t-1Index of the same quarter using weights of year t-1

Over-the-year techniqueOver-the-year technique Linking factor based on same periods for years t and t-1Linking factor based on same periods for years t and t-1

Page 63: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

63

Page 64: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

64

Recommended method Recommended method

Annual overlap techniqueAnnual overlap technique More practical for Laspeyres-type volume measuresMore practical for Laspeyres-type volume measures Monthly/quarterly data aggregate to annual dataMonthly/quarterly data aggregate to annual data

• However, there are no clear established rules for However, there are no clear established rules for choosing this approachchoosing this approach

• In most cases, the approaches will give similar In most cases, the approaches will give similar resultsresults

Page 65: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

65

Drawback of chainlinking

Lacks of additivity characteristic The lower level volume measures (e.g. ISIC 4-

digit class) do not sum to upper levels of the ISIC structures (e.g. 3 digit ISIC level)

When individual prices and quantities changes occurring in earlier periods are reverse in later period, chaining can lead to a worse result than a fixed base index

Page 66: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

66

SummarySummary Industry level weightsIndustry level weights

• Annual update Annual update should be carried out.should be carried out.• Should ideally be Should ideally be National Accounts value added figures at National Accounts value added figures at

basic pricesbasic prices – adjustments necessary to make them timely – adjustments necessary to make them timely available.available.

Product group weightsProduct group weights• Should be updated frequently Should be updated frequently at least every 5 yearsat least every 5 years• Obtained by determining the share of value of output, via the Obtained by determining the share of value of output, via the

conduct of product census or surveysconduct of product census or surveys Product weightsProduct weights

• The weights of individual products are updated at the same The weights of individual products are updated at the same time as product grouptime as product group

• Obtained by determining the share of value of output, via the Obtained by determining the share of value of output, via the conduct of product census or surveysconduct of product census or surveys

Page 67: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

67

SummarySummary

The The chained Laspreyres-type volume indexchained Laspreyres-type volume index is the is the recommended one for the compilation of the IIPrecommended one for the compilation of the IIP

When re-weighting occurWhen re-weighting occur Do not re-calculate the entire series Do not re-calculate the entire series The index is compiled with weights only for The index is compiled with weights only for

those period to which they relatethose period to which they relateFor monthly and quarterly data, advantage of For monthly and quarterly data, advantage of

chaining are less as price and quantity are subject chaining are less as price and quantity are subject to greater fluctuation.to greater fluctuation.

Page 68: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

68

Conceptual illustration of IIP annual chaining

80

85

90

95

100

105

110

115

120

Jan-07

Feb-07

Mar-07

Apr-07

May-07

Jun-07

Jul-07

Aug-07

Sep-07

Oct-07

Nov-07

Dec-07

Jan-08

Feb-08

Mar-08

Apr-08

May-08

Jun-08

Jul-08

Aug-08

Sep-08

Oct-08

Nov-08

Dec-08

Jan-09

Feb-09

Month

Ind

ex

nu

mb

er

Page 69: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Missing weights

Page 70: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

70

Missing weights

Missing weights for the most recent periodsMissing weights for the entire time span of

one component seriesNotice that there is no ‘recommended’

approach in this area.

Page 71: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

71

Estimation of the missing weights for the most recent period

In practice, the calculation of the IIP is likely to use industry weights from period t-2 i.e. the year 2005 index is likely to be compiled

using industry weights from 2003.This is because the necessary weighting data for

the industry level are not normally available until at least 18 months after the reference period.

Page 72: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

72

Missing weights for the most recent period

In some countries, for the first few months of a new year (i.e. year 2006 in this example) the index may need to be compiled using the ‘old’

weights (i.e. from 2003) because the ‘new’ weights (i.e. from 2004) are not yet available.

In these situations, the IIP should be recalculated (revised) on the basis of the new weights once they become available i.e. the January 2006 IIP should be calculated using the

weights from 2003 but be recalculated on the basis of 2004 weights when they become available.

Page 73: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

73

Estimation of the missing weights for the most recent period - others

Alternative source For example, use survey data (e.g. annual survey of

manufacturing) to impute for the missing GVA. Administrative sources

Subjective expert judgement Estimation

Time series method - ARIMA , state-space model, moving average, etc.

Regression model Imputation procedure.

Use equal weights Need proper quality check!

Page 74: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

More on estimations

ImputationRegression Exponential smoothingARIMA modelState space model

74

Page 75: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

75

Imputation Imputation Historic valueHistoric value

Use historic value such as last year valueUse historic value such as last year value

Historic value with trendHistoric value with trend Trend can be based on growth in another variable within the Trend can be based on growth in another variable within the

record, variables in other records, etc.record, variables in other records, etc.

Useful method when variables or growth rates are stable Useful method when variables or growth rates are stable over timeover time

Page 76: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

76

Regression modelRegression model

A regression model predicts a missing value using a A regression model predicts a missing value using a function of some auxiliary variables X.function of some auxiliary variables X.

Auxiliary variables can be from the current survey or Auxiliary variables can be from the current survey or other sources. E.g. historical information (previous other sources. E.g. historical information (previous period value)period value)

Regression coefficients (beta) can be determined Regression coefficients (beta) can be determined from historic datafrom historic data

ttt XY '

Page 77: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Exponential SmoothingExponential Smoothing

Smoothing parameter Smoothing parameter determined bydetermined by Subjective considerationSubjective consideration Minimizing sum of square of forecasting errorsMinimizing sum of square of forecasting errors

Relatively simple to useRelatively simple to use

77

1|1||1 ttttttt YYYY

Forecast of t+1 value at time t

Smoothing parameter

Forecast error at time t-1

Page 78: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

Autoregressive integrated Autoregressive integrated moving average (ARIMA)moving average (ARIMA)

ARIMA(p,d,q) model (assume d=0 in this case)ARIMA(p,d,q) model (assume d=0 in this case)

78

Identification of the model is necessary before proceeding to Identification of the model is necessary before proceeding to forecasting. forecasting. The AR and MA lag order (i.e. The AR and MA lag order (i.e. p p and and qq)) The AR and MA smoothing parameters (i.e. The AR and MA smoothing parameters (i.e. φφ and and θθ)) The integrated order, dThe integrated order, d

Complicated to use, but many statistical software, such as SAS Complicated to use, but many statistical software, such as SAS and R, has a built-in procedure for estimationand R, has a built-in procedure for estimation

Page 79: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

State Space ModelState Space Model State Space Model is a structural time series model that allows

Obtain unobserved component (unseen driving force) given observable series (something you can see)

Model each time series component (trend, seasonal, and also sampling error) within a structure

Update estimates at current time using ‘Kalman Filter’

Two equations in matrix formTwo equations in matrix form– Measurement (Observable) EquationMeasurement (Observable) Equation– State (Transition) EquationState (Transition) Equation

The magic is that once the system is specified in these two equations, the The magic is that once the system is specified in these two equations, the system can be updated through a certain set of algorithm.system can be updated through a certain set of algorithm.

Complicated to use, but unlike ARIMA, it does not require the series to be Complicated to use, but unlike ARIMA, it does not require the series to be stationary. In addition, it will cope with multivariate approach for further stationary. In addition, it will cope with multivariate approach for further extensionextension

Page 80: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

State Space Model Measurement Equation

State Equation

Notations Y: A Observable Series (e.g. Weights) α: State Vector (e.g. a vector of trend, sampling error) Z, T, R – matrix for computations I, η – Random Errors Subscript represent time point

ttt IZY

ttt RT 1

Page 81: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

81

Missing weights for the entire time span of component series

Use equal weights Expert judgementUse weights from other sourcesEstimationsProduct replacement

Page 82: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

SummarySummary

The calculation of the IIP is likely to use industry weights from period t-2

If period t-2 weights is not available, the index may be compiled using the t-3 weight

Several methods of estimating missing weights at the most recent periods are also proposed in this presentation, though there is no international recommendation in this area.

82

Page 83: Weighting issues Julian Chow Industrial and Energy Statistics Section United Nations Statistics Division (UNSD) Email: chowj@un.org

DiscussionDiscussion

83