Deciding between Alternative Approaches in Macroeconomics

DECIDING BETWEEN ALTERNATIVE APPROACHES IN MACROECONOMICS

David F. Hendry Institute for New Economic Thinking, Oxford Martin School, Oxford

Haavelmo Memorial Lecture, Oslo, November 2015 Research jointly with Jennifer Castle, Jurgen Doornik, Søren Johansen, Grayham Mizon

and Felix Pretis David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 1 / 70

Introduction

[1] All macroeconomic theories are incomplete, incorrect andchangeable;[2] all macroeconomic time-series data are aggregated,inaccurate and rarely match theoretical concepts;[3] all empirical macro-econometric models are non-constant,and mis-specified in numerous ways;[4] macroeconomic forecasts often go awry:[5] economic policy often has unexpected effects different fromprior analyses;so how to decide between alternative approaches?

Every decision about a theory formulation, its evidential base, itsempirical implementation and its evaluation involves selection, thoughoften such decisions are camouflaged.Selection is inevitable, unavoidable and ubiquitous.Key concept: empirical model discovery with theory evaluation.

David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 2 / 70

Introduction

[1] All macroeconomic theories are incomplete, incorrect andchangeable;[2] all macroeconomic time-series data are aggregated,inaccurate and rarely match theoretical concepts;[3] all empirical macro-econometric models are non-constant,and mis-specified in numerous ways;[4] macroeconomic forecasts often go awry:[5] economic policy often has unexpected effects different fromprior analyses;so how to decide between alternative approaches?

Every decision about a theory formulation, its evidential base, itsempirical implementation and its evaluation involves selection, thoughoften such decisions are camouflaged.Selection is inevitable, unavoidable and ubiquitous.Key concept: empirical model discovery with theory evaluation.


Key source


Credibility versus verisimilitude

Main justification of empirical macro-econometric evidence:conformity with conventionally-accepted economic theory:‘internal credibility’ as against verisimilitude.

Partly justified by manifest inadequacy of data:short, dependent, and heterogeneous time-series data, often subjectto extensive revision:if data are unreliable, better to trust the theory.

But theories have evolved greatly:most previous analyses have been abandoned.Almost self-contradictory to justify an empirical model by aninvalid theory that will soon be altered.

Is incorrect and mutable theory really more reliable thaninaccurate data evidence?














Possible explanations forwhy theory dominates

Constellation of data problems:non-stationarity, endogeneity, potential lack of identification, andcollinearity lead to belief that ‘data mining’ can produce almost anydesired result for economic data generation processes (DGPs).But so can theory choice by matching a model to non-existent‘stylized facts’, that are neither constant nor facts.

Conflating economic-theory models with the DGP:

huge gap between abstract theory and non-stationary datafinessed by asserting that the model is the mechanism–‘let’s take the model seriously’–means it is time to leave the talk.

Belief that data-based model selection is a subterfuge ofscoundrels is false–it is crucial to understanding complex macro-economies.Full story in Hendry and Doornik (2014).














New source

ii

“UGQE2book15” — 2015/5/20 — 9:10 — page i — #1 ii

ii

ii

Introductory Macro-econometrics:A New Approach

David F. Hendry

Published by Timberlake Consultants Ltdwww.timberlake.co.uk

www.timberlake-consultancy.com


Route map

(1) Trygve Haavelmo’s foundations for econometrics

(2) Unpredictability and its implications

(3) Multiple location shifts

(4) Modelling location shifts during model selection

(5) Can forecasting help distinguish between models?

(6) Empirical model discovery with theory evaluation

(7) Conclusion


Trygve Haavelmo’s foundationsfor econometrics

Above summary of present state of macroeconomics also prevailedwhen Trygve Haavelmo (1944) published his Probability Approach.

Haavelmo initially assumed that relevant macroeconomic theoryexisted, but viable econometric methods to quantify it did not, sohe developed a general framework for the latter.

That publication set in train many powerful advances in econometrics:see Morgan (1990), Qin (1993, 2013) and Hendry and Morgan (1995)for histories of econometrics.

But Haavelmo found there was little empirically-relevant theory,so switched to research on economics, as described in hisEconometric Society Presidential Address, Haavelmo (1958)—also see Anundsen, Nymoen, Krogh and Vislie (2012)—and in his Nobel Lecture, Haavelmo (1989):Bjerkholt (2005) provides valuable discussion of this switch.


Trygve Haavelmo’s foundationsfor econometrics

Above summary of present state of macroeconomics also prevailedwhen Trygve Haavelmo (1944) published his Probability Approach.

Haavelmo initially assumed that relevant macroeconomic theoryexisted, but viable econometric methods to quantify it did not, sohe developed a general framework for the latter.

That publication set in train many powerful advances in econometrics:see Morgan (1990), Qin (1993, 2013) and Hendry and Morgan (1995)for histories of econometrics.

But Haavelmo found there was little empirically-relevant theory,so switched to research on economics, as described in hisEconometric Society Presidential Address, Haavelmo (1958)—also see Anundsen et al. (2012)—and in his Nobel Lecture, Haavelmo (1989):Bjerkholt (2005) provides valuable discussion of this switch.


Trygve Haavelmo’s formalization

Haavelmo (1944) formalized analytical methods for econometrics in ageneral probabilistic framework, and clarified a range of specificstochastic models and their properties.

A key aspect was his notion of a design of experiments to matchthe data generation process, so valid inferences could beconducted.

‘Selection’ in Haavelmo (1944) refers to choosing the theory model,whereas ‘testing’ includes the notion of selection below—as well asthe usual senses of specification and mis-specification testing: seeSpanos (1989) and Juselius (1993).

Haavelmo (1944) also formalized the conditions for successfuleconomic forecasting: that the model was the DGP and the jointdistribution of the sample and forecast period was constant,(Bjerkholt, 2007 notes this chapter was added after the 1941 version),but was seemingly unaware of Smith (1929).


Trygve Haavelmo’s formalization

Haavelmo (1944) formalized analytical methods for econometrics in ageneral probabilistic framework, and clarified a range of specificstochastic models and their properties.

A key aspect was his notion of a design of experiments to matchthe data generation process, so valid inferences could beconducted.

‘Selection’ in Haavelmo (1944) refers to choosing the theory model,whereas ‘testing’ includes the notion of selection below—as well asthe usual senses of specification and mis-specification testing: seeSpanos (1989) and Juselius (1993).

Haavelmo (1944) also formalized the conditions for successfuleconomic forecasting: that the model was the DGP and the jointdistribution of the sample and forecast period was constant,(Bjerkholt, 2007 notes this chapter was added after the 1941 version),but was seemingly unaware of Smith (1929).


Changes since Trygve Haavelmo’sformalization

Major changes have occurred in the conceptualization of economictime series since Haavelmo wrote, especially their non-stationarity.

Economies change continually, sometimes abruptly, forcing acareful consideration of stochastic trends from unit roots andstructural shifts in DGPs.

These features further complicate empirical analyses: developing toolsfor valid inferences in such settings has been a major & successfulenterprise.

However, less attention has been paid to the perniciousimplications of unanticipated location shifts for theoreticalmacroeconomics, economic policy and forecasting.

Another key development has been in viable methods of selectingempirical models despite all the complications of aggregate economicdata and the need to allow for more variables (N) than observations(T ) in the candidate set of determinants to be considered.


Changes since Trygve Haavelmo’sformalization

Major changes have occurred in the conceptualization of economictime series since Haavelmo wrote, especially their non-stationarity.

Economies change continually, sometimes abruptly, forcing acareful consideration of stochastic trends from unit roots andstructural shifts in DGPs.

These features further complicate empirical analyses: developing toolsfor valid inferences in such settings has been a major & successfulenterprise.

However, less attention has been paid to the perniciousimplications of unanticipated location shifts for theoreticalmacroeconomics, economic policy and forecasting.

Another key development has been in viable methods of selectingempirical models despite all the complications of aggregate economicdata and the need to allow for more variables (N) than observations(T ) in the candidate set of determinants to be considered.


Empirical econometrics

To establish ‘truth’ requires at least these 12 assumptions:1. correct, comprehensive, & immutable economic theory;2. correct, complete choice of all relevant variables & lags;3. validity & relevance of all regressors & instruments;4. precise functional forms for all variables;5. absence of hidden dependencies;6. all expectations formulations correct;7. all parameters identified, constant over time, & invariant;8. exact data measurements on every variable;9. errors are ‘independent’ & homoscedastic;10. error distributions are constant over time;11. appropriate estimator at relevant sample sizes;12. valid and non-distortionary method of model selection.

If ‘truth’ is not on offer–what is?


The role of economic theory inempirical econometrics

Many features of empirical models are not derivable from abstracttheory, so must be based on the available data sample to discoverwhat actually matters empirically.

Since ceteris paribus is inevitably inappropriate in macroeconomics, atbest macroeconomic theory provides an object within modelling, notthe target , although it is often imposed as the latter.

We now consider what the target might be, and how that matchesHaavelmo’s concept of ‘design of experiments’.


Design of experimentsand the theory of reduction

Haavelmo (1944, p. 14) states: “We try to choose a theory and adesign of experiments to go with it, in such a way that the resultingdata would be those which we get by passive observation of reality.”

At first sight, it is not obvious how to choose a ‘design of experiments’in macroeconomics to match aggregate observed data.

But the theory of reduction provides a possible interpretation.

To understand how an economy actually functions, the appropriatetarget for model selection must be its data generation process.

The DGP is the joint density DW1T(w1 . . .wT |ψ

1T ,W0) whereW1

T

denotes the complete set of variables over a time period 1, . . . , T ,conditional on the past,W0, but D(·) and the ‘parameters’ ψ1

T ∈ Ψ ofthe economic agents’ decision processes may be time varying.

But as the DGP is the joint density of all the variables in the economyunder analysis, it is too high dimensional and non-stationary todevelop complete theories about, or to model empirically.


Design of experimentsand the theory of reduction

Haavelmo (1944, p. 14) states: “We try to choose a theory and adesign of experiments to go with it, in such a way that the resultingdata would be those which we get by passive observation of reality.”

At first sight, it is not obvious how to choose a ‘design of experiments’in macroeconomics to match aggregate observed data.

But the theory of reduction provides a possible interpretation.

To understand how an economy actually functions, the appropriatetarget for model selection must be its data generation process.

The DGP is the joint density DW1T(w1 . . .wT |ψ

1T ,W0) whereW1

T

denotes the complete set of variables over a time period 1, . . . , T ,conditional on the past,W0, but D(·) and the ‘parameters’ ψ1

T ∈ Ψ ofthe economic agents’ decision processes may be time varying.

But as the DGP is the joint density of all the variables in the economyunder analysis, it is too high dimensional and non-stationary todevelop complete theories about, or to model empirically.


Local DGP

The local DGP (denoted LDGP) is the DGP for the r variables {xt}which an investigator has chosen to model, usually based on a priorsubject-matter theory with entailed ‘parameters’ θ1

T ∈Θ.

The theory of reduction explains the derivation of the LDGP DX1T(·)

from DW1T(·) (see e.g., Cook and Hendry, 1993, and Hendry, 2009).

Following Doob (1953), the LDGP DX1T(·) can always be written by

sequential factorization with a martingale difference error, orinnovation, that is unpredictable from the past of the process:

DX1T

(X1

T | X0,θ1T

)= DxT

(xT |X

1T−1,X0,θT

)DX1

T−1

(X1

T−1|X0,θ1T−1

)... (1)

=

T∏t=1

Dxt

(xt | X

1t−1,X0,θt

).

Thus, the joint density is the product of the individual densities.


Properties of the LDGP

Defining εt = xt − EXt−1

[xt|X

1t−1

], then EXt−1

[εt|X

1t−1

]= 0,

confirming that {εt} is an innovation error process, and providing asuitable basis for laws of large numbers and central limit theorems.

Consequently, the LDGP provides an appropriate ‘design ofexperiments’ whereby the passively observed data are described towithin the smallest possible innovation errors given the choice of {xt}.

The LDGP innovation error {εt} is designed, or created, by thereductions entailed in moving from the DGP to the LDGP, and isnot an ‘autonomous’ process, but a reflection of our ignorance.

A ‘better’ choice of variables than {xt}, namely one where the LDGP iscloser to the actual DGP, would deliver yet smaller innovation errors.

Nevertheless, once {xt} is chosen, one cannot do better than knowDX1

T(·), which encompasses all models thereof on the same data (or

subsets thereof) (see Bontemps and Mizon, 2008). Thus, given {xt},the LDGP is the appropriate target for model selection.


Properties of the LDGP

Defining εt = xt − EXt−1

[xt|X

1t−1

], then EXt−1

[εt|X

1t−1

]= 0,

confirming that {εt} is an innovation error process, and providing asuitable basis for laws of large numbers and central limit theorems.

Consequently, the LDGP provides an appropriate ‘design ofexperiments’ whereby the passively observed data are described towithin the smallest possible innovation errors given the choice of {xt}.

The LDGP innovation error {εt} is designed, or created, by thereductions entailed in moving from the DGP to the LDGP, and isnot an ‘autonomous’ process, but a reflection of our ignorance.

A ‘better’ choice of variables than {xt}, namely one where the LDGP iscloser to the actual DGP, would deliver yet smaller innovation errors.

Nevertheless, once {xt} is chosen, one cannot do better than knowDX1

T(·), which encompasses all models thereof on the same data (or

subsets thereof) (see Bontemps and Mizon, 2008). Thus, given {xt},the LDGP is the appropriate target for model selection.


The LDGP as the target formodel selection

The set of variables {xt} chosen for analysis will depend on thesubject-matter theory, institutional knowledge, and previous evidence,so any theory-model object is directly related to the target LDGP.

While the LDGP provides the appropriate ‘design of experiments’ tomatch the ‘passive observations of reality’, it is always unknown inpractice, which is why Hendry and Doornik (2014) emphasize the needto discover the LDGP from the available evidence.

Doing so requires nesting that LDGP in a suitably general unrestrictedmodel (denoted GUM) while also embedding the theory model in thatGUM, then searching for the simplest acceptable representation,stringently evaluating that selection for congruence & encompassing.

Unfortunately, (1) provides cold comfort for empirical modelling:sequential factorization delivers an innovation error only by using thecorrect Dxt(·) at each point in time, the difficulty we now address.


The LDGP as the target formodel selection

The set of variables {xt} chosen for analysis will depend on thesubject-matter theory, institutional knowledge, and previous evidence,so any theory-model object is directly related to the target LDGP.

While the LDGP provides the appropriate ‘design of experiments’ tomatch the ‘passive observations of reality’, it is always unknown inpractice, which is why Hendry and Doornik (2014) emphasize the needto discover the LDGP from the available evidence.

Doing so requires nesting that LDGP in a suitably general unrestrictedmodel (denoted GUM) while also embedding the theory model in thatGUM, then searching for the simplest acceptable representation,stringently evaluating that selection for congruence & encompassing.

Unfortunately, (1) provides cold comfort for empirical modelling:sequential factorization delivers an innovation error only by using thecorrect Dxt(·) at each point in time, the difficulty we now address.


Route map







(7) Conclusion


Large unanticipated changes occur

Current crisis latest example of large unanticipated changes ineconomics. Distributions can shift abruptly, called location shifts.

2000 2001 2002 2003 2004 2005 2006 2007 2008-20

-10

0

10

20Annual percentage change in Japan Exports

Year on year % change in Japanese exports: looks ‘well behaved’.David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 18 / 70

Large unanticipated changes occur

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011-70

-50

-30

-10

10

30

Annual percentage change in Japan Exports

Then a drop of 70% in mid 2008–2009!David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 18 / 70

Unpredictability comes in threevarieties: (a) intrinsic unpredictability

A random variable X is unpredictable with respect to some informationI if knowing that information does not change our knowledge about X.

Intrinsic unpredictability in a known distribution:unknown knowns from chance distribution sampling, ‘randomerrors’... But which draw matters: bet on Red & get Black at Roulette.


Unpredictability comes in threevarieties: (a) intrinsic unpredictability




Illustrating intrinsic unpredictability

original distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

original distribution

Normal distribution often the basis for probabilities, and randomsampling from a known distribution basis for statistical inference:example of intrinsic unpredictability.


Unpredictability comes in threevarieties (b) instance unpredictability



Instance unpredictability:known unknowns as in outliers from ‘fat-tailed’ distributions atunanticipated times –see Taleb (2009)


Illustrating intrinsic andinstance unpredictability

original distribution fat-tailed distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40


Sometimes observe what are called ‘black swan events’:instance unpredictability–unknown magnitude, sign and timing.Unlikely to get several such draws with independent sampling.


Unpredictability comes in threevarieties (c) extrinsic unpredictability




Extrinsic unpredictability:unknown unknowns from unanticipated shifts of distributions.Unknown numbers, signs, magnitudes & timings of outliers and shifts.

Most pernicious form of extrinsic unpredictability is due to locationshifts: changes from previous ‘level’ of X at unanticipated times byunknown magnitudes.Let me illustrate that.


Unpredictability comes in threevarieties (c) extrinsic unpredictability




Extrinsic unpredictability:unknown unknowns from unanticipated shifts of distributions.Unknown numbers, signs, magnitudes & timings of outliers and shifts.

Most pernicious form of extrinsic unpredictability is due to locationshifts: changes from previous ‘level’ of X at unanticipated times byunknown magnitudes.Let me illustrate that.David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 23 / 70

Illustrating unpredictability andlocation shifts


-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40


Often see flocks of apparent ‘black swans’.Could never get such draws with independent sampling.


Illustrating unpredictability andlocation shifts

original distribution fat-tailed distribution shift in distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

original distribution fat-tailed distribution shift in distribution

Location shifts make new ordinary seem unusual relative to past.Example of extrinsic unpredictability: many potential causes forsuch shifts, and many shifts have occurred historically.


Irrational to hold ‘rational expectations’when shifts occur

When unanticipated shifts occur, today’s conditional expectation ofevents tomorrow can be biased and dominated by other predictors.Today’s expectation can be poor estimate of tomorrow’s outcome.

original distribution shift in distribution

-10 -5 0 5 10

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

← initial expectationtomorrow’s mean →

original distribution shift in distribution

Extrinsic unpredictability wrecks economic agents’ ability to planinter-temporally: if shifts not anticipated, agents need‘error-correction mechanisms’ after their occurrence.


Law of iterated expectations failswhen distributions shift

When variables, say (xt+1, xt), at different dates are drawn from thesame distribution fx(·), law of iterated expectations holds:

Efx [Efx [xt+1 | xt]] = Efx [xt+1] . (2)

But when distributions shift:

Efxt

[Efxt+1

[xt+1 | xt]]6= Efxt+1

[xt+1] (3)

as:

fxt+1(xt+1 | xt) fxt

(xt) 6= fxt+1(xt+1 | xt) fxt+1

(xt) (4)

Hendry and Mizon (2014) provide formal derivations.

Invalidates inter-temporal derivations facing unanticipated shifts:or theory requires no location shifts, so is empirically irrelevant.DSGEs are intrinsically non-structural as their very mathematicalbasis fails when distributions alter.Facing location shifts, economists cannot rely on theory-basedselection alone.


Law of iterated expectations failswhen distributions shift

When variables, say (xt+1, xt), at different dates are drawn from thesame distribution fx(·), law of iterated expectations holds:

Efx [Efx [xt+1 | xt]] = Efx [xt+1] . (2)

But when distributions shift:

Efxt

[Efxt+1

[xt+1 | xt]]6= Efxt+1

[xt+1] (3)

as:

fxt+1(xt+1 | xt) fxt

(xt) 6= fxt+1(xt+1 | xt) fxt+1

(xt) (4)

Hendry and Mizon (2014) provide formal derivations.

Invalidates inter-temporal derivations facing unanticipated shifts:or theory requires no location shifts, so is empirically irrelevant.DSGEs are intrinsically non-structural as their very mathematicalbasis fails when distributions alter.Facing location shifts, economists cannot rely on theory-basedselection alone.


Economic agents cannot tell what hasshifted till long afterwards

30 35 40 45 50

20

30

40

50a

Shifts in dynamics and causal links

30 35 40 45 50

20

30

40

50b

Shifts in intercepts and dynamics

30 35 40 45 50

20

30

40

50c

Shifts in intercepts, dynamics and causal links

2006 2007 2008 2009 2010-10.0

-7.5

-5.0

-2.5

0.0

2.5

5.0d

Annual percentage change in UK GDP

Can create essentially same ν shape by changing many combinationsof parameters in artificial data—yet panel d is annualized % changein UK GDP. Learning new parameters will take agents and economiststime, but both could use forecasting devices robust after shifts.


Route map







(7) Conclusion


UK inflation and its location shifts

1850 1875 1900

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 mean ∆w

(4)

(5)(6)

(7)

(8)

WWII→

Units−rate

(1)

(3)

(4)

(5)

(6)

(7)

(8)

WWII→

Units−rate

annual change in wagesannual price inflation

(9)

UK ‘great depression’ ↓

Annual changes in wages and prices. UK ‘great depression’noticeable, then all quiet before the Western Front.



1850 1875 1900

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(3)

(4)

(5)

(6)

(7)

(8)

WWI →

WWII→

Units−rate



(9)(2)

Both wage and price inflation leap in World War I.



1850 1875 1900

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

WWI →

(3) postwar crash →

Units−rate



(9)

Hugh crash in both at the end of WWI.



1850 1875 1900 1925

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(6)

(7)

(8)

↑US Great Depression

WWI →

postwarcrash →

Units−rate



(9)

Both stagnate in the interwar period, then.....



1850 1875 1900 1925

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(6)

(7)

(8)


WWI →

←postwar crash

Units−rate



(9)

WWII→ (5)

Jump in WWII, especially wages, with price controls duringrationing.



1850 1875 1900 1925 1950

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)


WWI →WWII→

←postwar crash

Units−rate

post-warreconstruction

↓



(9)

Steady and low through the post-war reconstruction, but:



1850 1875 1900 1925 1950 1975

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)


WWI →

WWII→

Oil crisis →

←postwar crash

Units−rate


↓



←‘Barber boom’

(9)

(8)

Wrecked by the oil crises.



1850 1875 1900 1925 1950 1975 2000

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25 shifts in ∆w

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)


WWI →

WWII→

← Oil crisis

←postwar crash

leave ←ERM

Units−rate


↓



←‘Barber boom’

(9)

(10) ↑GreatRecession

Little effect from leaving ERM, but some from ‘Great Recession’.Changes highlight major shifts & breaks–ten distinct epochs.


Unemployment and its location shifts

1860 1865 1870 1875 1880 1885 1890 1895 1900 1905 1910

0.025

0.050

0.075

0.100

0.125

0.150

← WWII

Boer war →

Units−rate

financial ↑crisis


Clear business cycle before World War I, with UK ‘greatdepression’ effects extended till 1913.



1860 1870 1880 1890 1900 1910 1920 1930

0.025

0.050

0.075

0.100

0.125

0.150

←Postwar crash

Boer war →

US crash →

Units−rate

Leave gold standard →

Financial ↑crisis


WWI →

Units−rate

financial ↑crisis

Unemployment leaps after WWI, and again after US crash.



1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970

0.025

0.050

0.075

0.100

0.125

0.150

Oil crisis→ ←Postwar crash

Boer war →

US crash →

↑ Post-war reconstruction

Units−rate

←Leave gold standard

Financial ↑crisis


WWI →

Units−rate

financial ↑crisis

← WWII

Units−rate

financial ↑crisis

Rapid drop at WWII, then steady through the post-warreconstruction, but:



1860 1880 1900 1920 1940 1960 1980 2000 2020

0.025

0.050

0.075

0.100

0.125

0.150

WWI →

← WWII

← Oil crisis ←Postwar crash

←Mrs T

Boer war →

US crash →

↑ Post-war reconstruction

←Leave ERM

Units−rate

←Leave gold standard


Units−rateUnits−rate

Financial ↑crisis

Units−rateUnits−rate

Wrecked by the oil crisis and Mrs Thatcher–shows 8 main epochs, but many do not match inflation shifts.


Major location shifts inunemployment and inflation

1860 1880 1900 1920 1940 1960 1980 2000 2020

-0.10

-0.05

0.00

0.05

0.10

0.15Units−rateUnits−rate

Unemployment shifts ↓

↑ inflation shifts

(a)

(b)

(c)

(b)

(a)

(a)

(b)

(c)

(a): inflation changes; unemployment does not

(b): shifts in opposite directions

(c): unemployment shifts; inflation does not

Unemployment shifts often do not match inflation shifts.


Explains why ‘Phillips Curves’ shift

0.01 0.03 0.05 0.07 0.09 0.11 0.13 0.15

-0.2

-0.1

0.0

0.1

0.2

186118621863

18641865 1866

186718681869

18701871

18721873

1874

18751876 1877

1878 1879

18801881188218831884 1885 1886

188718881889

1890

18911892

189318941895

18961897189818991900

19011902 1903 190419051906

190719081909

1910191119121913

1914

1915

1916

1917

1918

19191920

1921

1922

1923

1924 1925

1926

1927

1928

19291930

1931 19321933

1934193519361937

1938

1939

1940

1941

1942

1943

1944

1945

1946

19471948

19491950

19511952

19531954

19551956

1957

19581959

19601961

19621963

196419651966

1967

19681969

19701971

19721973

1974

1975

1976

1977

1978

1979

1980

∆w

Ur

1860-1913

1914-1945

1946-1980

1981-20111981

19821983

1984

198519861987198819891990

19911992

19931994

199519961997199819992000

20012002

200320042005

200620072008

2009

20102011

Wars and oil crises↓

↑Pricecontrols

← Price indexation

Must model shifts to understand underlying economic behaviour.


Route map







(7) Conclusion


Detecting multiple breaks

Numbers, timings and magnitudes of breaks in models usuallyunknown: obviously so for unknowingly omitted variables.‘Portmanteau’ approach required to detect location shifts anywhere insample, while also selecting over many candidate variables.To check the null of no outliers or location shifts in a model,impulse-indicator saturation (IIS)creates complete set of impulse indicator variables:{1{j=t}

}= 1 when j = t and 0 otherwise for j = 1, . . . , T :

add T indicators to set of candidate variables when T observations.

Feasible ‘split-sample’ IIS algorithm in Hendry, Johansen andSantos (2008), generalized by Johansen and Nielsen (2009) whoextend IIS to both stationary and unit-root autoregressions, andimplemented in Autometrics by Doornik (2009).Castle, Doornik and Hendry (2012) show potency of IIS under varietyof alternatives, including fat tails.Potential for major gain under alternatives of breaks and/oroutliers, yet can be done jointly with all other selections.


Detecting multiple breaks

Numbers, timings and magnitudes of breaks in models usuallyunknown: obviously so for unknowingly omitted variables.‘Portmanteau’ approach required to detect location shifts anywhere insample, while also selecting over many candidate variables.To check the null of no outliers or location shifts in a model,impulse-indicator saturation (IIS)creates complete set of impulse indicator variables:{1{j=t}

}= 1 when j = t and 0 otherwise for j = 1, . . . , T :

add T indicators to set of candidate variables when T observations.Feasible ‘split-sample’ IIS algorithm in Hendry et al. (2008),generalized by Johansen and Nielsen (2009) who extend IIS to bothstationary and unit-root autoregressions, and implemented inAutometrics by Doornik (2009).Castle et al. (2012) show potency of IIS under variety of alternatives,including fat tails.Potential for major gain under alternatives of breaks and/oroutliers, yet can be done jointly with all other selections.


IIS and SIS

Extension of IIS to step-indicator saturation (SIS): adding a completeset of step indicators S1 =

{1{t6j}, j = 1, . . . , T

}, where 1{t6j} = 1

for observations up to j, and zero otherwise.Step indicators are the cumulation of impulse indicators up to eachnext observation.

IIS: Impulses SIS: Step Shifts1 0 0 00 1 0 00 0 1 0

0 0 0. . .

1 1 1 10 1 1 10 0 1 10 0 0 1

SIS has correct null retention frequency (gauge) in constantconditional models for a nominal test size of α.Alternative retention-frequency function (potency) has appropriateprobabilities of retaining location shifts.


Illustrating SIS when no location shifts

‘Split-sample’ search by SIS at 1% on white noise.

0 50 100

0.5

1.0Indicators included initially Selected model: actual and fitted

Bloc

k 1

0 50 100

0.5

1.0Indicators retainedIndicators retained

actual fitted

0 50 100

10.0

12.5actual fitted



‘Split-sample’ search by SIS at 1% on white noise.

0 50 100

0.5


Bloc

k 1

0 50 100

0.5

1.0Indicators retained

actual fitted

0 50 100

10.0

12.5actual fitted

0 50 100

0.5

1.0

Bloc

k 2

0.0 0.5 1.0

0.5

1.0

0 50 100

10.0

12.5



0 50 100

0.5


Bloc

k 1

0 50 100

0.5


actual fitted

0 50 100

10.0

12.5actual fitted

0 50 100

0.5

1.0

Bloc

k 2

0.0 0.5 1.0

0.5

1.0

0 50 100

10.0

12.5

0 50 100

0.5

1.0

Fina

l

Indicators retained

0 50 100

0.5

1.0

Indicators retained

0 50 100

10.0

12.5

T = 100, and no shifts, retains 2 significant steps, so lose 2degrees of freedom–but could be combined to one dummy.


Illustrating ‘split-half’ SIS for asingle location shift

Add half indicators and select ones significant at 1%.

0 50 100

0.5


Bloc

k 1

0 50 100

0.5


actual fitted

0 50 100

0

5

10

15actual fitted



Drop, add other half indicators and again select at 1%.

0 50 100

0.5


Bloc

k 1

0 50 100

0.5


actual fitted

0 50 100

0

5

10

15actual fitted

0 50 100

0.5

1.0

Bloc

k 2

0 50 100

0.5

1.0

0 50 100

0

5

10

15



Combine retained indicators and re-select at 1%.

0 50 100

0.5


Bloc

k 1

0 50 100

0.5


actual fitted

0 50 100

0

5

10

15actual fitted

0 50 100

0.5

1.0

Bloc

k 2

0 50 100

0.5

1.0

0 50 100

0

5

10

15

0 50 100

0.5

1.0

Fina

l

0 50 100

0.5

1.0

0 50 100

0

5

10

15

Initially retains last step as mean shifts down, then finds locationshift, so eliminates redundant indicator: just one step needed.SIS is applicable during multi-path model selection.


Route map







(7) Conclusion


Can forecasting help distinguish?

Almost no economic theories allow for unanticipated location shifts:yet empirically occur intermittently.Analogy: rocket to moon dueto land on 4th July, but hit bymeteor and knocked off course,so never arrives there.

Forecast is badly wrong

Outcome is not due to poor forecasting models;and does not refute underlying (Newtonian gravitation) theory.Failure due to an unanticipated location shift.

But some methods robust to location shiftsInsure against systematic forecast failure–so forecasting ‘success’ need not entail a good model.

Cannot rely on forecasting to distinguish between alternativeapproaches.


Can forecasting help distinguish?

Almost no economic theories allow for unanticipated location shifts:yet empirically occur intermittently.Analogy: rocket to moon dueto land on 4th July, but hit bymeteor and knocked off course,so never arrives there.

Forecast is badly wrong

Outcome is not due to poor forecasting models;and does not refute underlying (Newtonian gravitation) theory.Failure due to an unanticipated location shift.

But some methods robust to location shiftsInsure against systematic forecast failure–so forecasting ‘success’ need not entail a good model.

Cannot rely on forecasting to distinguish between alternativeapproaches.


Bank of England forecasts

Chart shows Bank of England quarterly forecasts for annual changesin UK GDP at February 2008 through end 2010:

Distinct slowdown envisaged–but nothing like the unanticipated 6% fall that materialized.


‘Mimic’ forecasts and outcomes

forecasts ∆4log(GDP)

2004 2005 2006 2007 2008 2009 2010 2011

-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0.00

0.01

0.02

0.03

0.04

forecasts ∆4log(GDP)

Massive forecast failure: forecasts up, while data down, thenexcellent (!), as predicted by our theory of forecasting.


Understanding forecast errors

Let DGP over t = 1, . . . , T be:∆xt = (ρ− 1) (xt−1 − θ) + γ(zt − κ) + εt (5)

Problem if shift in long-run mean E[xt] = (µ+ γκ)/(1− ρ) = θ,where µ is the intercept.

No forecast failure if E[xt] = θ before and after shift.

Forecast failure if E[xT+h] = θ∗ 6= θ changes in:

∆xT+h = (ρ∗ − 1) (xT+h−1 − θ∗) + γ∗(zT+h − κ∗) + εT+h (6)

All models in equilibrium correction class fail systematically whenE[·] changes to θ∗, as forecasts converge back to θ,irrespective of new parameter values in DGP.

Huge class of equilibrium-correction models (EqCMS):regressions; dynamic systems; VARs; DSGEs;ARCH; GARCH; some other volatility models.

Pervasive and pernicious problem affecting all members.


Forecast-error sources

Problems exacerbated by model mis-specification:

∆xt = (ρ− 1)(xt−1 − θ

)+ εt (7)

All main forecast errors occur using (7) when (6) is DGP:

εT+1|T = θ∗−θ+ρ∗ (xT − θ∗)−ρ(xT − θ

)+γ∗ (zT+1 − κ

∗)+εT+1

(8)(ia) deterministic shifts: (θ, κ) to (θ∗, κ∗);(ib) stochastic breaks: (ρ, γ) to (ρ∗, γ∗);(iia,b) inconsistent parameter estimates: θe 6= θ, ρe 6= ρ;(iii) forecast origin uncertainty: xT ;(iva,b) estimation uncertainty: V[ρ, θ];(v) omitted variables: zT+1;(vi) innovation errors: εT+1.

Hazardous basis for forecasting even if start with ‘goodin-sample model’: including the in-sample DGP.


Dramatic difference in 1-step forecastsof level of GDP by robust device

Log(GDP) ‘Conventional’ forecast ‘Robust’ forecast

2007 2008 2009 2010 2011

12.66

12.68

12.70

12.72

12.74

12.76Forecasts of the (log) level of UK GDP

Log(GDP) ‘Conventional’ forecast ‘Robust’ forecast

Robust device has much smaller RMSFE,but not a justification for selecting it as an economic model.


Route map







(7) Conclusion


Discovery in economics

Discoveries in economics mainly from theory.But all economic theories are:(a) incomplete; (b) incorrect; and (c) mutable.(a) Need strong ceteris paribus assumptions:inappropriate in a non-stationary, evolving world.

(b) Consider an economic analysis which suggests:y = f (z) (9)

where the k variables y depend on n ‘explanatory’ variables z withm > n instruments x.Form of f (·) in (9) depends on:utility or loss functions of agents,constraints they face, & information they possess.Analyses arbitrarily assume: forms for f (·), that f (·) is constant, thatonly z matters, & that the xs are ‘exogenous’.Yet must aggregate across heterogeneous individualswhose endowments shift over time, often abruptly.



Discoveries in economics mainly from theory.But all economic theories are:(a) incomplete; (b) incorrect; and (c) mutable.(a) Need strong ceteris paribus assumptions:inappropriate in a non-stationary, evolving world.(b) Consider an economic analysis which suggests:

y = f (z) (9)where the k variables y depend on n ‘explanatory’ variables z withm > n instruments x.Form of f (·) in (9) depends on:utility or loss functions of agents,constraints they face, & information they possess.

Analyses arbitrarily assume: forms for f (·), that f (·) is constant, thatonly z matters, & that the xs are ‘exogenous’.Yet must aggregate across heterogeneous individualswhose endowments shift over time, often abruptly.



Discoveries in economics mainly from theory.But all economic theories are:(a) incomplete; (b) incorrect; and (c) mutable.(a) Need strong ceteris paribus assumptions:inappropriate in a non-stationary, evolving world.(b) Consider an economic analysis which suggests:

y = f (z) (9)where the k variables y depend on n ‘explanatory’ variables z withm > n instruments x.Form of f (·) in (9) depends on:utility or loss functions of agents,constraints they face, & information they possess.Analyses arbitrarily assume: forms for f (·), that f (·) is constant, thatonly z matters, & that the xs are ‘exogenous’.Yet must aggregate across heterogeneous individualswhose endowments shift over time, often abruptly.


Theory evolves

(c) Economic analyses have not only changed our understanding, theyhave changed the world: from the ‘invisible hand’ in Adam Smith’sTheory of Moral Sentiments (1759, p.350) onwards, theory hasprogressed dramatically–key insights into option pricing, auctions and contracts,principal-agent and game theories, trust and moral hazard,asymmetric information, institutions:major impacts on market functioning, industrial,and even political, organization.

But imagine imposing 1900’s economic theory in empiricalresearch today.

Much past applied econometrics research is forgotten:discard the economic theory that it ‘quantified’ andyou discard the associated empirical evidence.

Hence fads & fashions, ‘cycles’ and ‘schools’ in economics.


Theory evolves

(c) Economic analyses have not only changed our understanding, theyhave changed the world: from the ‘invisible hand’ in Adam Smith’sTheory of Moral Sentiments (1759, p.350) onwards, theory hasprogressed dramatically–key insights into option pricing, auctions and contracts,principal-agent and game theories, trust and moral hazard,asymmetric information, institutions:major impacts on market functioning, industrial,and even political, organization.

But imagine imposing 1900’s economic theory in empiricalresearch today.

Much past applied econometrics research is forgotten:discard the economic theory that it ‘quantified’ andyou discard the associated empirical evidence.

Hence fads & fashions, ‘cycles’ and ‘schools’ in economics.


Discovery: learning somethingpreviously unknown

Cannot know how to discover what is not known.So unlikely there is a ‘best’ way of doing so.

Both empirical and theoretical discoveries important.

Science is inductive and deductive.Must distinguish between:context of discovery—where ‘anything goes’, andcontext of evaluation—rigorous attempts to refute.

Howsoever a discovery is made, needs a warrant that it is ‘real’.Methods of evaluation are subject-specific:economics requires a theoretical interpretation consistent with‘mainstream theory’.

Accumulation and consolidation of evidence crucial:

data reduction a key attribute of science (think E = mc2).


























Common aspects of discovery

Aspects in common to historical empirical and theoretical discoveries:1 theoretical context, or framework of ideas.2 going outside existing state of knowledge.3 searching for something.4 recognition of significance of what is found.5 quantification of what is found.6 evaluating discovery to ascertain its ‘reality’.7 parsimoniously summarize information acquired.

But science perforce is simple to general–a slow and uncertain route to new knowledge.Globally, learning must be simple to general; locally, need not be.Econometrics discovery from observational data can circumvent suchlimits....


How to proceed?

Embed economic analysis y = f(z) in an initial much moregeneral unrestricted model (GUM): y = g(z,η).

Nest ‘theory-driven’ and ‘data-driven’ approaches to retain theoryinsights unaffected by selection, but select over additional rivalcandidate variables, lags, functional forms, and shifts (η above).

Need to tackle all these complications jointly.Re-frame empirical modelling as discovery processcombined with theory evaluation.

If theory correct and complete, distributions of parameterestimators of z in g(·) identical to directly fitting y = f(z) to data:if incorrect, discover better empirical model.

Retain theory formulation in a congruent, parsimoniousencompassing model, seeking parameters invariant to relevantpolicy changes.


How to proceed?

Embed economic analysis y = f(z) in an initial much moregeneral unrestricted model (GUM): y = g(z,η).

Nest ‘theory-driven’ and ‘data-driven’ approaches to retain theoryinsights unaffected by selection, but select over additional rivalcandidate variables, lags, functional forms, and shifts (η above).

Need to tackle all these complications jointly.Re-frame empirical modelling as discovery processcombined with theory evaluation.

If theory correct and complete, distributions of parameterestimators of z in g(·) identical to directly fitting y = f(z) to data:if incorrect, discover better empirical model.

Retain theory formulation in a congruent, parsimoniousencompassing model, seeking parameters invariant to relevantpolicy changes.


Retaining economic theory insights

Approach is not atheoretic.

Theory formulations should be embedded in GUM,to be retained without selection,but does not guarantee parameters will be significant.

Can also ensure theory-derived signs of long-run relation maintained,if not significantly rejected by the evidence.

However, much observed data variability in economics is due tofeatures absent from most economic theories:which empirical models must handle.

Extension of DGP candidates, zt, in GUM allows theory formulationas special case, yet protects against contaminating influences (likeoutliers) absent from theory.

‘Extras’ can be selected at tight significance levels.


Retaining economic theory insights

Approach is not atheoretic.

Theory formulations should be embedded in GUM,to be retained without selection,but does not guarantee parameters will be significant.

Can also ensure theory-derived signs of long-run relation maintained,if not significantly rejected by the evidence.

However, much observed data variability in economics is due tofeatures absent from most economic theories:which empirical models must handle.

Extension of DGP candidates, zt, in GUM allows theory formulationas special case, yet protects against contaminating influences (likeoutliers) absent from theory.

‘Extras’ can be selected at tight significance levels.


When economic theory isonly part of the explanation

Two distinct forms of under-specification:

a] omitting relevant shifts, functions or lags of variables in DGP withcoefficients γ 6= 0: avoided by sufficiently general initial model;b] omitting relevant variables, ηt, from the DGP;induces less useful local DGP–but hard to avoid if ηt unknown.

In a], when γ 6= 0, but zt and ηt orthogonal, estimated coefficients offormer same as if model with just zt is simply fitted to the data:but may be insignificant with wrong signs.

In b], when the GUM nests DGP, but ηt omitted from theory,selection with SIS could substantively improve the final model:see Castle and Hendry (2014a).

Win-win situation—theory kept if valid and complete, yet learnif it is incorrect–empirical model discovery with theory evaluation.

Can automatic model selection still work when N > T?








Can automatic model selection still work when N > T?








Can automatic model selection still work when N > T?David F. Hendry (Oxford) Deciding between Alternative Approaches Haavelmo Memorial Lecture 2015 62 / 70

More candidate variablesthan observations

If also have relevant variables to be retained, orthogonalize them withrespect to the rest.

When N > T , divide in sub-blocks, setting α = 1/N.

Retain desired sub-set of n variables at every stage, and only selectover putative irrelevant variables at stringent significance level α:under the null, selection has no impact on estimated coefficientsof relevant variables, or their distributions.

Thus, almost costless to check large numbers of rival variables:huge benefits if initial specification incorrect but enlarged GUMnests DGP.

In effect, answer every likely seminar question in advance;but controlling the null retention frequency.


Evaluating policy models

Parameter invariance essential in policy models:else mis-predict under regime shifts.

Automatic test of super exogeneity in Hendry and Santos (2010):apply IIS in marginal models, retain all significant outcomes andtest their relevance in conditional model.

No ex ante knowledge of timing or magnitudes of breaks:need not know DGP of marginal variables.

Test has correct size under null of super exogeneityfor a range of sizes of marginal-model saturation tests.

Power to detect failures of super exogeneity when location shiftsin marginal models:advance warning of likely policy invalidity.


Route map







(7) Conclusion


Conclusions on developmentssince Haavelmo

Design of experimentsTheory of reduction provides a possible interpretation.

Local data generation process (LDGP) designed to match the‘passive observations of reality’.

LDGP is then the target for model selection.

LDGP always unknown in practice

Need to discover the LDGP from the available evidence.

Seek to nest LDGP in a general unrestricted model (GUM)embedding the theory model object, often with N > T .

Search for simplest acceptable representation thereof, stringentlyevaluating it for congruence and encompassing.

Must allow for non-stationarities, especially location shifts.


Conclusions on developmentssince Haavelmo

Design of experimentsTheory of reduction provides a possible interpretation.

Local data generation process (LDGP) designed to match the‘passive observations of reality’.

LDGP is then the target for model selection.

LDGP always unknown in practice

Need to discover the LDGP from the available evidence.

Seek to nest LDGP in a general unrestricted model (GUM)embedding the theory model object, often with N > T .

Search for simplest acceptable representation thereof, stringentlyevaluating it for congruence and encompassing.

Must allow for non-stationarities, especially location shifts.


Conclusions on deciding betweenalternative approaches

Theory onlyLocation shifts invalidate law of iterated expectations;

conditional expectations not unbiased for next period;

economic agents could not quickly learn what had changed.

Forecast onlyForecast failure primarily due to location shifts;

systematic mis-forecasting by all equilibrium-correctionmodels: regressions, VARs, DSGEs, EqCMs, GARCH, etc.;

yet every DGP parameter can shift without noticeable effect.

Difficult to predict shifts: but mitigate failure by robustdevices.


Conclusions on deciding betweenalternative approaches

Theory onlyLocation shifts invalidate law of iterated expectations;

conditional expectations not unbiased for next period;

economic agents could not quickly learn what had changed.

Forecast onlyForecast failure primarily due to location shifts;

systematic mis-forecasting by all equilibrium-correctionmodels: regressions, VARs, DSGEs, EqCMs, GARCH, etc.;

yet every DGP parameter can shift without noticeable effect.

Difficult to predict shifts: but mitigate failure by robustdevices.


Conclusions on empirical modeldiscovery with theory evaluation

All essential steps feasible once target LDGP defined:1. automatically create general model from investigator’s xt:extra variables, longer lags, non-linearities, & shift indicators–ensures congruent GUM & avoids costs of under-specification;2. embed theory-model, orthogonalizing other variables–ensures baseline specification is retained unaltered;3. select most parsimonious, congruent, encompassing model–ensures undominated representation;4. compute near-unbiased parameter estimates–ensures appropriate quantification; and5. stringently evaluate results, especially super exogeneity–ensures selected model valid, and usable.

Generalizes to N > T with expanding and contracting searches.Surprising, but can empirically model if not theorize or forecast.

Empirical model discovery with theory evaluation is a way ahead.


References I

Anundsen, A. K., Nymoen, R., Krogh, T. S., and Vislie, J. (2012). The macroeconomics of Trygve Haavelmo. NordicJournal of Political Economy, 37, 1–26.

Bjerkholt, O. (2005). Frisch’s econometric laboratory and the rise of Trygve Haavelmo’s Probability Approach.Econometric Theory, 21, 491–533.

Bjerkholt, O. (2007). Writing the probability approach with nowhere to go: Haavelmo in the United States, 1939-1944.Econometric Theory, 23, 775–837.

Bontemps, C., and Mizon, G. E. (2008). Encompassing: Concepts and implementation. Oxford Bulletin of Economicsand Statistics, 70, 721–750.

Castle, J. L., Doornik, J. A., and Hendry, D. F. (2012). Model selection when there are multiple breaks. Journal ofEconometrics, 169, 239–246.

Castle, J. L., Doornik, J. A., Hendry, D. F., and Nymoen, R. (2014). Mis-specification testing: Non-invariance ofexpectations models of inflation. Econometric Reviews, 33, 553–574.

Castle, J. L., Fawcett, N. W. P., and Hendry, D. F. (2011). Forecasting breaks and during breaks. In Clements, M. P., andHendry, D. F. (eds.), Oxford Handbook of Economic Forecasting, pp. 315–353. Oxford University Press.

Castle, J. L., and Hendry, D. F. (2009). The long-run determinants of UK wages, 1860–2004. Journal of Macroeconomics,31, 5–28.

Castle, J. L., and Hendry, D. F. (2014a). Model selection in under-specified equations with breaks. Journal ofEconometrics, 178, 286–293.

Castle, J. L., and Hendry, D. F. (2014b). Semi-automatic non-linear model selection. In Haldrup, N., Meitz, M., andSaikkonen, P. (eds.), Essays in Nonlinear Time Series Econometrics, pp. 163–197. Oxford: OxfordUniversity Press.

Castle, J. L., and Shephard, N. (eds.)(2009). The Methodology and Practice of Econometrics. Oxford University Press.Cook, S., and Hendry, D. F. (1993). The theory of reduction in econometrics. Poznan Studies in the Philosophy of the

Sciences and the Humanities, 38, 71–100.Doob, J. L. (1953). Stochastic Processes. New York: John Wiley Classics Library. 1990 edition.Doornik, J. A. (2009). Autometrics. In Castle, and Shephard (2009), pp. 88–121.Galı, J., and Gertler, M. (1999). Inflation dynamics: A structural econometric analysis. Journal of Monetary Economics,

44, 195–222.Haavelmo, T. (1944). The probability approach in econometrics. Econometrica, 12, 1–118. Supplement.


References II

Haavelmo, T. (1958). The role of the econometrician in the advancement of economic theory. Econometrica, 26, 351–357.Haavelmo, T. (1989). Prize Lecture. Sveriges Riksbank: Prize in Economic Sciences in Memory of Alfred Nobel.Hendry, D. F. (2009). The methodology of empirical econometric modeling: Applied econometrics through the

looking-glass. In Mills, T. C., and Patterson, K. D. (eds.), Palgrave Handbook of Econometrics, pp. 3–67.Palgrave MacMillan.

Hendry, D. F., and Doornik, J. A. (2014). Empirical Model Discovery and Theory Evaluation. MIT Press.Hendry, D. F., Johansen, S., and Santos, C. (2008). Automatic selection of indicators in a fully saturated regression.

Computational Statistics, 33, 317–335. Erratum, 337–339.Hendry, D. F., and Krolzig, H.-M. (2005). The properties of automatic Gets modelling. Economic Journal, 115, C32–C61.Hendry, D. F., and Mizon, G. E. (2014). Unpredictability in economic analysis, econometric modeling and forecasting.

Journal of Econometrics, 182, 186–195.Hendry, D. F., and Morgan, M. S. (eds.)(1995). The Foundations of Econometric Analysis. Cambridge University Press.Hendry, D. F., and Santos, C. (2010). An automatic test of super exogeneity. In Watson, M. W., Bollerslev, T., and Russell,

J. (eds.), Volatility and Time Series Econometrics, pp. 164–193. Oxford University Press.Johansen, S., and Nielsen, B. (2009). An analysis of the indicator saturation estimator as a robust regression estimator.

In Castle, and Shephard (2009), pp. 1–36.Juselius, K. (1993). VAR modelling and Haavelmo’s probability approach to econometrics. Empirical Economics, 18,

595–622.Morgan, M. S. (1990). The History of Econometric Ideas. Cambridge: Cambridge University Press.Qin, D. (1993). The Formation of Econometrics: A Historical Perspective. Oxford: Clarendon Press.Qin, D. (2013). A History of Econometrics: The Reformation from the 1970s. Oxford: Clarendon Press.Smith, B. B. (1929). Judging the forecast for 1929. Journal of the American Statistical Association, 24, 94–98.Soros, G. (2008). The New Paradigm for Financial Markets. London: Perseus Books.Spanos, A. (1989). On re-reading Haavelmo: A retrospective view of econometric modeling. Econometric Theory, 5,

405–429.Taleb, N. N. (2009). Errors, robustness, and the fourth quadrant. International Journal of Forecasting, 25, 744–759.


Documents

Deciding between Alternative Approaches in Macroeconomics