Early Signs of Financial Crisescheongsa/Talks/soms.pdfSegmentation-clustering analysis of high-frequency DJI time series: – Discovery of phases with straightforward macroeconomic

Early Signs of Financial Crises

CHEONG Siew Ann

Division of Physics and Applied Physics,School of Physical and Mathematical Sciences,

Nanyang Technological University

10 June 2009

International Workshop on Coping with Crises in Complex Socio-Economic Systems,ETH Zurich, Switzerland

Acknowledgements

WONG Jian Cheng/SPMS LIAN Heng/SPMS

GOO Yik Wen/MAE LIAN Tong Wei/EEE

ONG Wei Guang/SPMS CHOI Wen Ting/SPMS

SOMS Workshop 2009, ETH Zurich, Switzerland 1

Motivation

• A financial crisis is a “diseased state” of the market.

• In medicine, early intervention is more effective, and less costly than a late cure.

– For current crisis, trillions of dollars in relief and stimulus, just starting to seelittle positive results.

• Early economic and financial intervention requires:

– sensitive detection of structural changes in market; and

– robust classification of structural changes as onset of financial crisis.

• Identify structural changepoints enroute to economic recovery:

– learn which relief and stimulus measures are effective; and

– presumably, same measures should be effective as interventions.


Data & Models

• Dow Jones Industrial Average (DJI): representative spectrum of industries.

• Time series between 1 Jan 1997 to 31 Aug 2008: M segments from P phases.

• Half-hourly frequency: statistically detect segments as short as 1 day.

• Normal index movement (NIM) model:

– index movements within segment m normally distributed with mean µm andvariance σ2

m;– actual changes in index.

• Log-normal index movement (LIM) model:

– log-index movements within segment m normally distributed with mean µ′mand variance σ′m

2;– percentage changes in index;– popular in finance literature.

• Maximum likelihood estimates µm, σm, µ′m, and σ′m.


Jensen-Shannon Divergence

• Recursive entropic segmentation scheme to determine M. [Bernaola-Galvan et al,Phys. Rev. E 53, 5181 (1996); Roman-Roldan et al, Phys. Rev. Lett. 80, 1344(1998)];

• If x = (x1, . . . , xi, xi+1, . . . , xN) single segment with mean µ and variance σ2, likeli-hood

L1 = P(x|µ, σ2) =

N∏j=1

1√

2πσ2exp[−

(x j − µ)2

2σ2

].

• If x two segments, xL = (x1, . . . , xi) with mean µL and variance σ2L, and xR =

(xi+1, . . . , xN) with mean µR and variance σ2R, likelihood

L2(i) = P(xL|µL, σ2L)P(xR|µR, σ

2R)

=

i∏j=1

1√2πσ2

L

exp−(x j − µL)2

2σ2L

N∏j=i+1

1√2πσ2

R

exp−(x j − µR)2

2σ2R

.• Define Jensen-Shannon divergence to be

∆(i) = logL2(i)L1≥ 0. (1)


Recursive Segmentation

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

0

500

1000

1500

2000

2500Je

nsen

-Sha

nnon

div

erge

nce



1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

0

500

1000

1500

2000

2500Je

nsen

-Sha

nnon

div

erge

nce



1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

0

500

1000

1500

2000

2500Je

nsen

-Sha

nnon

div

erge

nce



1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

0

500

1000

1500

2000

2500Je

nsen

-Sha

nnon

div

erge

nce


Segmentation Optimization

2003 2004 2005 2006 2007

0

500

1000

1500

2000

2500

3000

3500

4000Je

nsen

-Sha

nnon

div

erge

nce


Overview of Segmentation Algorithm

• STEP 1a (Segmentation):

– Given segment dx = (dx1, . . . , dxN), compute Jensen-Shannon divergence ∆(i)as function of cursor position i.

– Find i∗ such that ∆(i∗) = maxi ∆(i). Best 2-segment model for dx is dxL =

(dx1, . . . , dxi∗) and dxR = (dxi∗+1, . . . , dxN).

• STEP 1b (Optimization).

• STEP 2 (Recursion): Repeat STEP 1 for dxL and dxR.

• STEP 3 (Termination): 1-segment model selected over 2-segment model if:

– Hypothesis Testing: probability of obtaining divergence beyond ∆max greaterthan prescribed tolerance ε; or

– Model Selection: information criterion (e.g. AIC, BIC) for 2-segment modelgreater than 1-segment model; or

– Signal-to-Noise Ratio: ∆(i) contains more noise than signal.


Segmentation Results

• Number of segments:

NIM LIM116 119

• 85 boundaries in common: segment boundaries statistically robust.

• Disagreement intervals bound by very robust segment boundaries:

start date end date number of segments commonNIM LIM boundaries

Nov 3, 1997 Mar 31, 1998 4 5 0Aug 26, 1998 Oct 20, 1998 3 2 0Jan 13, 1999 Nov 5, 1999 3 7 0Mar 9, 2001 Jun 3, 2002 18 10 6Oct 16, 2002 Aug 6, 2003 9 6 2Mar 10, 2004 Oct 18, 2005 3 8 1Jul 28, 2006 Aug 15, 2006 1 2 0Sep 5, 2006 Dec 27, 2006 4 1 0Jul 25, 2007 Mar 10, 2008 7 14 4


Statistical Clustering

• Statistical clustering of segments to determine P.

– Jensen-Shannon divergence as statistical distance between segments;

– agglomerative hierarchical clustering;

– complete link algorithm.

• Early works assume small P. [Goldfeld & Quandt, J. Econometrics 1, 3 (1973);Hamilton, Econometrica 57, 357 (1989); Sims & Zha, Am. Econ. Rev. 96, 54(2006)]

• Textbook macroeconomic phases:

– expansion;

– contraction;

– correction;

– crash.


Clustering Results939598

10390

31.3

1104

141646

88792

52082

3100

97179489228478

739.1

4112

301040441580194986646668273288

10281

10583759196

249.3

42.7

6859955396552547729

107115

267617411574723503341313845

726217379435663133418596269

114116

71109111

102.2

34.4

95325

110486012355824

1013642

1065172287637

10870

113

(NIM)

889499909296

36.3

21100107

50110

93105

8387238591777981

63.1

12.9

353891561

8118

788095

112643686

109104

183244489875

101

8.8

15

1720436245105684

10627407630747033

57.5

139.3

20.7

46524386963162972

108102113

478260

649

1191454

117115

3542375851

4.7

2341319

1162226733167

75755461197

111

53.2

357.0

964.6

7.3

952

11468

1032841662571123959

(LIM)


Temporal Distribution of Clustered Segments

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

4000

6000

8000

10000

12000

14000

16000D

JI1 2 3 4 5 6 7 8 9 10


Story Told by Temporal Features

• Two high-volatility phases:

– mid-1998 to mid-2003.– mid-2007 to present.

• 1-year series of precursor shocks prior to low-to-high transitions, and 1-year seriesof inverted shocks prior to high-to-low transition.

• First low-to-high transition triggered by Asian Financial Crisis.

• Second low-to-high transition triggered by Chinese Correction.

– Strange coincidence between US housing market correction and Chinese mar-ket correction in May 2006.

• Detection:

– look out for precursor shocks, and discount isolated shocks;– if two consecutive shocks observed, then 3 months into precursor shock, 6 to

9 months early warning.


Intervention Measures

• Can detect, can prevent?

– Must understand causal links in order to break them.

• Two lines of inquiry:

– macroeconomic: segmentation/clustering study of US economic sectors.

∗ Sequence of sectors into decline?∗ Effective intervention measures?∗ In progress. . . .

– microeconomic: what really happened at the start of a financial crisis?

∗ Short time scale study of the Feb 2007 Chinese Correction.∗ Whole-market analyses.


Effective Variables and Effective Dynamics

• Market crash is a cooperative phenomenon.

– Study not individual stocks, but collections of stocks.

– But what collections?

• Previous study on extracting effective variables from financial markets: [Goo et al.,q-fin/0903.2099]

– whole-market analyses of daily price movements for 2006–2007.

– hierarchical organization of effective variables:

∗ financial atoms: collections of strongly-correlated stocks.∗ financial molecules: collections of strongly-correlated financial atoms.

– One financial molecule each in SGX and HKSE:

∗ comprises roughly half local stocks, half Chinese stocks.∗ No apparent reason for this structure apart from Chinese Correction.


Financial Molecules

3 8

6

11

10

4

7

9

1

5

SGX

10

146

8

1 9

3

16

15

11

7

5

2

HKSE


Financial Atoms & Atomic Correlation Levels

Feb

06

Mar

06

Apr

06

May

06

Jun

06

Jul 0

6

Aug

06

Sep

06

Oct

06

Nov

06

Dec

06

Jan

07

Feb

07

Mar

07

Apr

07

May

07

Jun

07

Jul 0

7

Aug

07

Sep

07

Oct

07

Nov

07

Dec

07

0

50

100

150

200

250

300

corr

elat

ion

leve

l

upper atomic correlation levellower atomic correlation level

0

5

10

15

20nu

mbe

r of

ato

ms

strong financial atomsweak financial atoms

US housingmarket correction/Chinese market

correction CorrectionChinese

CrisisSubprime


Financial Molecules & Molecular Sizes

1

2

3

4

5

num

ber

of m

olec

ules

Feb

06

Mar

06

Apr

06

May

06

Jun

06

Jul 0

6

Aug

06

Sep

06

Oct

06

Nov

06

Dec

06

Jan

07

Feb

07

Mar

07

Apr

07

May

07

Jun

07

Jul 0

7

Aug

07

Sep

07

Oct

07

Nov

07

Dec

07

0

100

200

300

400

500

num

ber

of s

tock

s

US housingmarket correction/Chinese market

correctionChinese

CorrectionSubprime

Crisis


Summary & Conclusions

• Segmentation-clustering analysis of high-frequency DJI time series:

– Discovery of phases with straightforward macroeconomic interpretations.

– High-volatility phase: mid-1998 to mid-2003, and mid-2007 to present.

– Year-long precursor shocks prior to low-to-high transitions, and year-long in-verted shocks prior to high-to-low transition.

– Mid-1998 low-to-high transition triggered by Asian Financial Crisis.

– Mid-2007 low-to-high transition triggered by Chinese Correction.

• High-frequency, whole-market correlational analysis of SGX:

– Market-level correlations peak around May 06 US/Chinese market correc-tions, Feb 07 Chinese Correction, and Aug 07 Subprime Crisis.

– Giant financial molecule whose size increases up to 1–2 month before majormarket events.

– Clear statistical signatures of giant financial molecule breaking up after marketcrashes.


Summary & Conclusions

• Applications to forecasting:

– 6–9 months lead time based on empirical precursor shock patterns.

– 1–2 months lead time based on growth of giant financial molecule to criticalsize.

• Applications to intervention & stimulus:

– Detailed compositional analysis currently underway.

– Understand detailed dynamics of giant financial molecule formation and dis-sociation:

∗ Force early dissociation? Soft landing?∗ Conscious restoration of pre-crash correlational structure?


Thank You!

Mean-Variance Scatter Plot

−40 −20 0 20 40µ

0

50

100

150

200σ

−20 −10 0 10 20µ′ (× 104)

0

10

20

30

40

50

60

σ′ (×

104 )

(NIM) (LIM)


Temporal Distribution of Clustered Segments

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

−400

−300

−200

−100

0

100

200

300

400in

dex

mov

emen

ts


Summary of Main Features

• Two dominant phases:

– low-volatility phase (economic expansion).

– high-volatility phase (contains economic contraction).

• Interrupted by:

– moderate-volatility phase (market correction).

∗ consistent 20-point NIM standard deviations;∗ short (1–2)-week segments (mostly in low-volatility phase);∗ long (1.5–2)-month segments (mostly in high-volatility phase).

– extremely-high-volatility phase (market crash).

∗ NIM standard deviations from 50 to 150 index points;∗ short (1–3)-day segments;∗ intermediate 1-week segments;∗ long (2–3)-week segments.


Sliding Window Analysis

• Repeat whole-market analysis of SGX at higher temporal resolution: half-hourlyprice movements within 2-month window.

• Slide 2-month window in 1-month steps to get 23 overlapping 2-month windowsbetween 2006 and 2007.

• Find financial atoms and molecules in each 2-month window.

• See how these evolve over time.

– Correlation level analysis;

– Giant component analysis;

– Compositional analysis.


Documents

Early Signs of Financial Crisescheongsa/Talks/soms.pdfSegmentation-clustering analysis of high-frequency DJI time series: – Discovery of phases with straightforward macroeconomic