Upload
rob-hyndman
View
622
Download
5
Tags:
Embed Size (px)
DESCRIPTION
Demographic forecasting using functional data analysis
Citation preview
Demographic forecasting
using functional data
analysis
Rob J Hyndman
Joint work with: Heather Booth, Han Lin Shang,Shahid Ullah, Farah Yasmeen.
Demographic forecasting using functional data analysis 1
Mortality rates
Demographic forecasting using functional data analysis 2
Fertility rates
Demographic forecasting using functional data analysis 3
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis 4
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis A functional linear model 5
Some notation
Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis A functional linear model 6
Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.
εt ,xiid∼ N(0,1) and et(x)
iid∼ N(0,v(x)).
Some notation
Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis A functional linear model 6
Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.
εt ,xiid∼ N(0,1) and et(x)
iid∼ N(0,v(x)).
Some notation
Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis A functional linear model 6
Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.
εt ,xiid∼ N(0,1) and et(x)
iid∼ N(0,v(x)).
Some notation
Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis A functional linear model 6
Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.
εt ,xiid∼ N(0,1) and et(x)
iid∼ N(0,v(x)).
Some notation
Let yt ,x be the observed (smoothed) data in period tat age x , t = 1, . . . ,n .
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis A functional linear model 6
Estimate ft(x) using penalized regression splines.Estimate µ(x) as me(di)an ft(x) across years.Estimate βt ,k and φk(x) using (robust) functionalprincipal components.
εt ,xiid∼ N(0,1) and et(x)
iid∼ N(0,v(x)).
French mortality components
Demographic forecasting using functional data analysis A functional linear model 7
0 20 40 60 80 100
−6
−5
−4
−3
−2
−1
Age
µ(x)
0 20 40 60 80 100
0.00
0.05
0.10
0.15
0.20
Ageφ 1
(x)
t
β t1
1850 1900 1950 2000
−15
−10
−5
05
10
0 20 40 60 80 100
−0.
2−
0.1
0.0
0.1
0.2
Age
φ 2(x
)t
β t2
1850 1900 1950 2000
−2
02
46
8
French mortality components
Demographic forecasting using functional data analysis A functional linear model 7
1850 1900 1950 2000
020
4060
8010
0 Residuals
Year
Age
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 8
French male mortality rates
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
0 20 40 60 80 100
−8
−6
−4
−2
0France: male death rates (1900−2009)
Age
Log
deat
h ra
te
French male mortality rates
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
0 20 40 60 80 100
−8
−6
−4
−2
0France: male death rates (1900−2009)
Age
Log
deat
h ra
te
War years
French male mortality rates
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9
0 20 40 60 80 100
−8
−6
−4
−2
0France: male death rates (1900−2009)
Age
Log
deat
h ra
te
War years
Aims
1 “Boxplots” for functional data2 Tools for detecting outliers in
functional data
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Let {ft(x)}, t = 1, . . . ,n , be a set of curves.
å Each point in scatterplot represents one curve.å Outliers show up in bivariate score space.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10
1 Apply a robust principal component algorithm
ft(x) = µ(x)+n−1∑k=1
βt ,kφk(x)
µ(x) is median curve{φk (x)} are principal components{βt ,k } are PC scores
2 Plot βi ,2 vs βi ,1
Robust principal components
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11
●●●●●●●●●●
●●
●●
● ●
●
●
●
●
●●●●●●●●●
●●●●●●●●●●
●
●
●
●
●
●
●
●●●
●●●●●
●●●●
●●●
●●●
●●●●●●●●●●●●
●●●
●●●●●●●●●
●●●
●●●●●●●●●●●●
●●●●●●●
−10 −5 0 5 10 15
−2
02
46
Scatterplot of first two PC scores
PC score 1
PC
sco
re 2
Robust principal components
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11
●●●●●●●●●●
●●
●●
● ●
●
●
●
●
●●●●●●●●●
●●●●●●●●●●
●
●
●
●
●
●
●
●●●
●●●●●
●●●●
●●●
●●●
●●●●●●●●●●●●
●●●
●●●●●●●●●
●●●
●●●●●●●●●●●●
●●●●●●●
−10 −5 0 5 10 15
−2
02
46
Scatterplot of first two PC scores
PC score 1
PC
sco
re 2
● ●
●
●
●
●
●
●
●
●
●
●
19141915
1916
1917
1918
1919
1940
1941
1942
1943
1944
1945
Functional bagplot
Bivariate bagplot due to Rousseeuw et al. (1999).Rank points by halfspace location depth.Display median, 50% convex hull and outerconvex hull (with 99% coverage if bivariatenormal).
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12
−10 −5 0 5 10 15
−2
02
46
PC score 1
PC
sco
re 2
● ●
●
●
●
●
●
●
●
●
●●●●●●●●
●●
●
●
●●
● ●
●
●
●
●
●●
●●●
●●●●
●
●●●●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●●
●
●
●●
●●
●●●●
●●
●●●●
●●●
●●●
●●●
●●●●
●●●●●
●●
●●●●
●●●
●●●
●●●●
● ●
●
●
●
●
●
●
●
●
19141915
1916
1917
1918
1919
19401943
1944
1945
Functional bagplot
Bivariate bagplot due to Rousseeuw et al. (1999).Rank points by halfspace location depth.Display median, 50% convex hull and outerconvex hull (with 99% coverage if bivariatenormal).Boundaries contain all curves inside bags.95% CI for median curve also shown.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12
−10 −5 0 5 10 15
−2
02
46
PC score 1
PC
sco
re 2
● ●
●
●
●
●
●
●
●
●
●●●●●●●●
●●
●
●
●●
● ●
●
●
●
●
●●
●●●
●●●●
●
●●●●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●●
●
●
●●
●●
●●●●
●●
●●●●
●●●
●●●
●●●
●●●●
●●●●●
●●
●●●●
●●●
●●●
●●●●
● ●
●
●
●
●
●
●
●
●
19141915
1916
1917
1918
1919
19401943
1944
1945
0 20 40 60 80 100
−8
−6
−4
−2
0
Age
Log
deat
h ra
te
1914191519161917
1918194019431944
Functional bagplot
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 13
0 20 40 60 80 100
−8
−6
−4
−2
0
Age
Log
deat
h ra
te
1914191519161917
1918194019431944
Functional HDR boxplot
Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
−10 −5 0 5 10 15
−2
02
46
PC score 1
PC
sco
re 2
●
●
o●
●●●●●●●●●
●
●
●●
● ●
●
●
●
●
●●
●●●
●●●●
●
●●●●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●●
●
●
●●
●●
●●●●
●●
●●●●
●●●
●●●
●●●
●●●●
●●●●●
●●
●●●●
●●●
●●●
●●●●
●
●
1919
1943
99% outer region
Functional HDR boxplot
Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
−10 −5 0 5 10 15
−2
02
46
PC score 1
PC
sco
re 2
● ●
●
●
●
●
●
●
●
●
●
o●
●●●●●●●●●
●
●
●●
● ●
●
●
●
●
●●
●●●
●●●●
●
●●●●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●●
●
●
●●
●●
●●●●
●●
●●●●
●●●
●●●
●●●
●●●●
●●●●●
●●
●●●●
●●●
●●●
●●●●
● ●
●
●
●
●
●
●
●
●
●
19141915
1916
1917
1918
1919
1940
1942
1943
1944
1945
90% outer region
Functional HDR boxplot
Bivariate HDR boxplot due to Hyndman (1996).Rank points by value of kernel density estimate.Display mode, 50% and (usually) 99% highestdensity regions (HDRs) and mode.Boundaries contain all curves inside HDRs.
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14
−10 −5 0 5 10 15
−2
02
46
PC score 1
PC
sco
re 2
● ●
●
●
●
●
●
●
●
●
●
o●
●●●●●●●●●
●
●
●●
● ●
●
●
●
●
●●
●●●
●●●●
●
●●●●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●●
●
●
●●
●●
●●●●
●●
●●●●
●●●
●●●
●●●
●●●●
●●●●●
●●
●●●●
●●●
●●●
●●●●
● ●
●
●
●
●
●
●
●
●
●
19141915
1916
1917
1918
1919
1940
1942
1943
1944
1945
90% outer region
0 20 40 60 80 100
−8
−6
−4
−2
0
Age
Log
deat
h ra
te
191419151916191719181919
19401943194419451948
Functional HDR boxplot
Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 15
0 20 40 60 80 100
−8
−6
−4
−2
0
Age
Log
deat
h ra
te
191419151916191719181919
19401943194419451948
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis Functional forecasting 16
Functional time series model
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.
Demographic forecasting using functional data analysis Functional forecasting 17
Functional time series model
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.
Demographic forecasting using functional data analysis Functional forecasting 17
Functional time series model
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.
Demographic forecasting using functional data analysis Functional forecasting 17
Functional time series model
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.
Demographic forecasting using functional data analysis Functional forecasting 17
Functional time series model
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
The eigenfunctions φk(x) show the mainregions of variation.The scores {βt ,k } are uncorrelated byconstruction. So we can forecast each βt ,kusing a univariate time series model.Outliers are treated as missing values.Univariate ARIMA models are used forforecasting.
Demographic forecasting using functional data analysis Functional forecasting 17
Forecasts
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
Demographic forecasting using functional data analysis Functional forecasting 18
Forecasts
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
where vn+h ,k = Var(βn+h ,k | β1,k , . . . ,βn ,k)and y = [y1,x , . . . ,yn ,x ].Demographic forecasting using functional data analysis Functional forecasting 18
E[yn+h ,x | y] = µ̂(x)+K∑
k=1
β̂n+h ,k φ̂k(x)
Var[yn+h ,x | y] = σ̂2µ (x)+
K∑k=1
vn+h ,k φ̂2k(x)+ σ
2t (x)+ v(x)
Forecasting the PC scores
Demographic forecasting using functional data analysis Functional forecasting 19
0 20 40 60 80 100
−6
−4
−2
0
Main effects
Age
Mea
n
0 20 40 60 80 100
0.05
0.10
0.15
0.20
AgeB
asis
func
tion
1
Interaction
Year
Coe
ffici
ent 1
1900 1940 1980 2020
−20
−10
05
1015
●
●
●
●
●
●
●
●
0 20 40 60 80 100
−0.
2−
0.1
0.0
0.1
0.2
Age
Bas
is fu
nctio
n 2
YearC
oeffi
cien
t 2
1900 1940 1980 2020
−6
−4
−2
02
●
●
●
●
●
●
●
●
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 20
0 20 40 60 80 100
−10
−8
−6
−4
−2
0France: male death rates (1900−2009)
Age
Log
deat
h ra
te
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 20
0 20 40 60 80 100
−10
−8
−6
−4
−2
0France: male death rates (1900−2009)
Age
Log
deat
h ra
te
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 20
0 20 40 60 80 100
−10
−8
−6
−4
−2
0France: male death forecasts (2010−2029)
Age
Log
deat
h ra
te
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 20
0 20 40 60 80 100
−10
−8
−6
−4
−2
0France: male death forecasts (2010 & 2029)
Age
Log
deat
h ra
te
80% prediction intervals
Fertility application
Demographic forecasting using functional data analysis Functional forecasting 21
15 20 25 30 35 40 45 50
050
100
150
200
250
Australia fertility rates (1921−2009)
Age
Fer
tility
rat
e
Fertility model
Demographic forecasting using functional data analysis Functional forecasting 22
15 20 25 30 35 40 45 50
05
1015
Age
Μ
15 20 25 30 35 40 45 50
−0.
3−
0.1
0.0
0.1
0.2
AgeΦ
1(x)
t
Βt1
1970 1980 1990 2000 2010
−10
−5
05
10
15 20 25 30 35 40 45 50
0.05
0.15
0.25
Age
Φ2(
x)t
Βt2
1970 1980 1990 2000 2010
−4
−2
02
46
8
Fertility model
Demographic forecasting using functional data analysis Functional forecasting 23
1970 1980 1990 2000
1520
2530
3540
45Residuals
Year
Age
Fertility model
Demographic forecasting using functional data analysis Functional forecasting 24
15 20 25 30 35 40 45 50
05
1015
Main effects
Age
Mea
n
15 20 25 30 35 40 45 50
−0.
3−
0.1
0.0
0.1
0.2
AgeB
asis
func
tion
1
Interaction
Year
Coe
ffici
ent 1
1970 1990 2010 2030
−10
05
1015
2025
15 20 25 30 35 40 45 50
0.05
0.15
0.25
Age
Bas
is fu
nctio
n 2
YearC
oeffi
cien
t 2
1970 1990 2010 2030
−4
−2
02
46
8
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 25
15 20 25 30 35 40 45 50
050
100
150
200
250
Australia fertility rates (1921−2009)
Age
Fer
tility
rat
e
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 25
15 20 25 30 35 40 45 50
050
100
150
200
250
Australia fertility rates (1921−2009)
Age
Fer
tility
rat
e
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 25
15 20 25 30 35 40 45 50
050
100
150
200
250
Australia fertility rates: 2010−2029
Age
Fer
tility
rat
e
Forecasts of ft(x)
Demographic forecasting using functional data analysis Functional forecasting 25
15 20 25 30 35 40 45 50
050
100
150
200
250
Australia fertility rates: 2010 and 2029
Age
Fer
tility
rat
e
80% prediction intervals
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis Forecasting groups 26
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
The problem
Let ft ,j(x) be the smoothed mortality rate for age x ingroup j in year t .
Groups may be males and females.Groups may be states within a country.Expected that groups will behave similarly.Coherent forecasts do not diverge over time.Existing functional models do not imposecoherence.
Demographic forecasting using functional data analysis Forecasting groups 27
Forecasting the coefficients
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis Forecasting groups 28
Forecasting the coefficients
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis Forecasting groups 28
Forecasting the coefficients
yt ,x = ft(x)+ σt(x)εt ,x
ft(x) = µ(x)+K∑
k=1
βt ,k φk(x)+ et(x)
We use ARIMA models for each coefficient{β1,j ,k , . . . ,βn ,j ,k }.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.
Demographic forecasting using functional data analysis Forecasting groups 28
Male fts model
Demographic forecasting using functional data analysis Forecasting groups 29
0 20 40 60 80
−8
−6
−4
−2
Age
µ M(x
)
0 20 40 60 80
0.05
0.10
0.15
Age
φ 1(x
)
Year
β t1
1960 2000
−10
−5
05
0 20 40 60 80
−0.
3−
0.1
0.0
0.1
0.2
Age
φ 2(x
)
Year
β t2
1960 2000
−2.
5−
1.5
−0.
50.
5
0 20 40 60 80
−0.
10.
00.
10.
2
Age
φ 3(x
)
Year
β t3
1960 2000
−1.
0−
0.5
0.0
0.5
Australian male death rates
Female fts model
Demographic forecasting using functional data analysis Forecasting groups 30
0 20 40 60 80
−8
−6
−4
−2
Age
µ F(x
)
0 20 40 60 80
0.05
0.10
0.15
Age
φ 1(x
)
Year
β t1
1960 2000
−10
−5
05
0 20 40 60 80
−0.
10.
00.
10.
2
Age
φ 2(x
)
Year
β t2
1960 2000
−1.
0−
0.5
0.0
0.5
0 20 40 60 80
−0.
2−
0.1
0.0
0.1
Age
φ 3(x
)
Year
β t3
1960 2000
−0.
8−
0.4
0.0
0.4
Australian female death rates
Australian mortality forecasts
Demographic forecasting using functional data analysis Forecasting groups 31
0 20 40 60 80 100
−10
−8
−6
−4
−2
(a) Males
Age
Log
deat
h ra
te
0 20 40 60 80 100
−10
−8
−6
−4
−2
(b) Females
Age
Log
deat
h ra
te
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
Product and ratio are approximatelyindependent
Ratio should be stationary (for coherence) butproduct can be non-stationary.
Demographic forecasting using functional data analysis Forecasting groups 32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
Product and ratio are approximatelyindependent
Ratio should be stationary (for coherence) butproduct can be non-stationary.
Demographic forecasting using functional data analysis Forecasting groups 32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
Product and ratio are approximatelyindependent
Ratio should be stationary (for coherence) butproduct can be non-stationary.
Demographic forecasting using functional data analysis Forecasting groups 32
Mortality product and ratios
Key idea
Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
Product and ratio are approximatelyindependent
Ratio should be stationary (for coherence) butproduct can be non-stationary.
Demographic forecasting using functional data analysis Forecasting groups 32
Mortality rates
Demographic forecasting using functional data analysis Forecasting groups 33
Mortality rates
Demographic forecasting using functional data analysis Forecasting groups 34
Model product and ratios
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt(x)] = µr(x)+L∑`=1
γt ,`ψ`(x)+wt(x).
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)
fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).
Demographic forecasting using functional data analysis Forecasting groups 35
Model product and ratios
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt(x)] = µr(x)+L∑`=1
γt ,`ψ`(x)+wt(x).
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)
fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).
Demographic forecasting using functional data analysis Forecasting groups 35
Model product and ratios
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt(x)] = µr(x)+L∑`=1
γt ,`ψ`(x)+wt(x).
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)
fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).
Demographic forecasting using functional data analysis Forecasting groups 35
Model product and ratios
pt(x) =√ft ,M(x)ft ,F(x) and rt(x) =
√ft ,M(x)
/ft ,F(x).
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt(x)] = µr(x)+L∑`=1
γt ,`ψ`(x)+wt(x).
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).No restrictions for βt ,1, . . . ,βt ,K .Forecasts: fn+h |n ,M(x) = pn+h |n(x)rn+h |n(x)
fn+h |n ,F(x) = pn+h |n(x)/rn+h |n(x).
Demographic forecasting using functional data analysis Forecasting groups 35
Product model
Demographic forecasting using functional data analysis Forecasting groups 36
0 20 40 60 80
−8
−6
−4
−2
Age
µ P(x
)
0 20 40 60 80
0.05
0.10
0.15
Age
φ 1(x
)
Year
β t1
1960 2000
−10
−5
05
0 20 40 60 80
−0.
2−
0.1
0.0
0.1
0.2
Age
φ 2(x
)
Year
β t2
1960 2000−2.
5−
1.5
−0.
50.
5
0 20 40 60 80
−0.
2−
0.1
0.0
0.1
Age
φ 3(x
)
Year
β t3
1960 2000
−0.
6−
0.2
0.2
0.6
Ratio model
Demographic forecasting using functional data analysis Forecasting groups 37
0 20 40 60 80
0.1
0.2
0.3
0.4
0.5
Age
µ R(x
)
0 20 40 60 80
−0.
100.
000.
100.
20
Age
φ 1(x
)
Year
β t1
1960 2000
−0.
50.
00.
5
0 20 40 60 80
−0.
050.
050.
150.
25
Age
φ 2(x
)
Year
β t2
1960 2000
−0.
6−
0.2
0.0
0.2
0.4
0 20 40 60 80
−0.
20.
00.
10.
20.
30.
4
Age
φ 3(x
)
Year
β t3
1960 2000
−0.
3−
0.1
0.1
0.3
Product forecasts
Demographic forecasting using functional data analysis Forecasting groups 38
0 20 40 60 80 100
−10
−8
−6
−4
−2
Age
Log
of g
eom
etric
mea
n de
ath
rate
Ratio forecasts
Demographic forecasting using functional data analysis Forecasting groups 39
0 20 40 60 80 100
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Age
Sex
rat
io: M
/F
Coherent forecasts
Demographic forecasting using functional data analysis Forecasting groups 40
0 20 40 60 80 100
−10
−8
−6
−4
−2
(a) Males
Age
Log
deat
h ra
te
0 20 40 60 80 100
−10
−8
−6
−4
−2
(b) Females
Age
Log
deat
h ra
te
Ratio forecasts
Demographic forecasting using functional data analysis Forecasting groups 41
0 20 40 60 80 100
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Independent forecasts
Age
Sex
rat
io o
f rat
es: M
/F
0 20 40 60 80 100
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Coherent forecasts
Age
Sex
rat
io o
f rat
es: M
/F
Life expectancy forecasts
Demographic forecasting using functional data analysis Forecasting groups 42
Life expectancy forecasts
Year
Age
1960 1980 2000 2020
7075
8085
1960 1980 2000 2020
7075
8085Life expectancy difference: F−M
Year
Num
ber
of y
ears
1960 1980 2000 2020
24
68
Coherent forecasts for J groups
pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J
and rt ,j(x) = ft ,j(x)/pt(x),
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt ,j(x)] = µr ,j(x)+L∑
l=1
γt ,l ,jψl ,j(x)+wt ,j(x).
Demographic forecasting using functional data analysis Forecasting groups 43
pt(x) and all rt ,j(x)are approximatelyindependent.
Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.
log[ft ,j(x)] = log[pt(x)rt ,j(x)]
Coherent forecasts for J groups
pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J
and rt ,j(x) = ft ,j(x)/pt(x),
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt ,j(x)] = µr ,j(x)+L∑
l=1
γt ,l ,jψl ,j(x)+wt ,j(x).
Demographic forecasting using functional data analysis Forecasting groups 43
pt(x) and all rt ,j(x)are approximatelyindependent.
Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.
log[ft ,j(x)] = log[pt(x)rt ,j(x)]
Coherent forecasts for J groups
pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J
and rt ,j(x) = ft ,j(x)/pt(x),
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt ,j(x)] = µr ,j(x)+L∑
l=1
γt ,l ,jψl ,j(x)+wt ,j(x).
Demographic forecasting using functional data analysis Forecasting groups 43
pt(x) and all rt ,j(x)are approximatelyindependent.
Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.
log[ft ,j(x)] = log[pt(x)rt ,j(x)]
Coherent forecasts for J groups
pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J
and rt ,j(x) = ft ,j(x)/pt(x),
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt ,j(x)] = µr ,j(x)+L∑
l=1
γt ,l ,jψl ,j(x)+wt ,j(x).
Demographic forecasting using functional data analysis Forecasting groups 43
pt(x) and all rt ,j(x)are approximatelyindependent.
Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.
log[ft ,j(x)] = log[pt(x)rt ,j(x)]
Coherent forecasts for J groups
pt(x) = [ft ,1(x)ft ,2(x) · · · ft ,J(x)]1/J
and rt ,j(x) = ft ,j(x)/pt(x),
log[pt(x)] = µp(x)+K∑
k=1
βt ,kφk(x)+ et(x)
log[rt ,j(x)] = µr ,j(x)+L∑
l=1
γt ,l ,jψl ,j(x)+wt ,j(x).
Demographic forecasting using functional data analysis Forecasting groups 43
pt(x) and all rt ,j(x)are approximatelyindependent.
Ratios satisfy constraintrt ,1(x)rt ,2(x) · · ·rt ,J(x) = 1.
log[ft ,j(x)] = log[pt(x)rt ,j(x)]
Coherent forecasts for J groups
µj(x) = µp(x)+µr ,j(x) is group mean
zt ,j(x) = et(x)+wt ,j(x) is error term.
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).
No restrictions for βt ,1, . . . ,βt ,K .
Demographic forecasting using functional data analysis Forecasting groups 44
log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]
= µj(x)+K∑
k=1
βt ,kφk(x)+L∑`=1
γt ,`,jψ`,j(x)+ zt ,j(x)
Coherent forecasts for J groups
µj(x) = µp(x)+µr ,j(x) is group mean
zt ,j(x) = et(x)+wt ,j(x) is error term.
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).
No restrictions for βt ,1, . . . ,βt ,K .
Demographic forecasting using functional data analysis Forecasting groups 44
log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]
= µj(x)+K∑
k=1
βt ,kφk(x)+L∑`=1
γt ,`,jψ`,j(x)+ zt ,j(x)
Coherent forecasts for J groups
µj(x) = µp(x)+µr ,j(x) is group mean
zt ,j(x) = et(x)+wt ,j(x) is error term.
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).
No restrictions for βt ,1, . . . ,βt ,K .
Demographic forecasting using functional data analysis Forecasting groups 44
log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]
= µj(x)+K∑
k=1
βt ,kφk(x)+L∑`=1
γt ,`,jψ`,j(x)+ zt ,j(x)
Coherent forecasts for J groups
µj(x) = µp(x)+µr ,j(x) is group mean
zt ,j(x) = et(x)+wt ,j(x) is error term.
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).
No restrictions for βt ,1, . . . ,βt ,K .
Demographic forecasting using functional data analysis Forecasting groups 44
log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]
= µj(x)+K∑
k=1
βt ,kφk(x)+L∑`=1
γt ,`,jψ`,j(x)+ zt ,j(x)
Coherent forecasts for J groups
µj(x) = µp(x)+µr ,j(x) is group mean
zt ,j(x) = et(x)+wt ,j(x) is error term.
{γt ,`} restricted to be stationary processes:either ARMA(p ,q) or ARFIMA(p ,d ,q).
No restrictions for βt ,1, . . . ,βt ,K .
Demographic forecasting using functional data analysis Forecasting groups 44
log[ft ,j(x)] = log[pt(x)rt ,j(x)] = log[pt(x)] + log[rt ,j ]
= µj(x)+K∑
k=1
βt ,kφk(x)+L∑`=1
γt ,`,jψ`,j(x)+ zt ,j(x)
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis Population forecasting 45
Demographic growth-balance equation
Demographic growth-balance equation
Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)
x = 0,1,2, . . . .
Pt(x) = population of age x at 1 January, year tBt = births in calendar year t
Dt(x ,x +1) = deaths in calendar year t of persons aged x atthe beginning of year t
Dt(B ,0) = infant deaths in calendar year tGt(x ,x +1) = net migrants in calendar year t of persons aged
x at the beginning of year tGt(B ,0) = net migrants of infants born in calendar year t
Demographic forecasting using functional data analysis Population forecasting 46
Demographic growth-balance equation
Demographic growth-balance equation
Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)
x = 0,1,2, . . . .
Pt(x) = population of age x at 1 January, year tBt = births in calendar year t
Dt(x ,x +1) = deaths in calendar year t of persons aged x atthe beginning of year t
Dt(B ,0) = infant deaths in calendar year tGt(x ,x +1) = net migrants in calendar year t of persons aged
x at the beginning of year tGt(B ,0) = net migrants of infants born in calendar year t
Demographic forecasting using functional data analysis Population forecasting 46
Key ideas
Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample
paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.
Demographic forecasting using functional data analysis Population forecasting 47
Key ideas
Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample
paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.
Demographic forecasting using functional data analysis Population forecasting 47
Key ideas
Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample
paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.
Demographic forecasting using functional data analysis Population forecasting 47
Key ideas
Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample
paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.
Demographic forecasting using functional data analysis Population forecasting 47
Key ideas
Build a stochastic functional model for eachof mortality, fertility and net migration.Treat all observed data as functional (i.e.,smooth curves of age) rather than discretevalues.Use the models to simulate future sample
paths of all components giving the entire agedistribution at every year into the future.Compute future births, deaths, net migrants.and populations from simulated rates.Combine the results to get age-specificstochastic population forecasts.
Demographic forecasting using functional data analysis Population forecasting 47
The available data
In most countries, the following data are
available:
Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x
From these, we can estimate:
mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F
t (x) = fertility rate for females ofage x in calendar year t .
Demographic forecasting using functional data analysis Population forecasting 48
The available data
In most countries, the following data are
available:
Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x
From these, we can estimate:
mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F
t (x) = fertility rate for females ofage x in calendar year t .
Demographic forecasting using functional data analysis Population forecasting 48
The available data
In most countries, the following data are
available:
Pt(x) = population of age x at 1 January, year tEt(x) = population of age x at 30 June, year tBt(x) = births in calendar year t to females of age xDt(x) = deaths in calendar year t of persons of age x
From these, we can estimate:
mt(x) = Dt(x)/Et(x) = central death rate incalendar year t ;ft(x) = Bt(x)/E F
t (x) = fertility rate for females ofage x in calendar year t .
Demographic forecasting using functional data analysis Population forecasting 48
Net migration
We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.
Demographic growth-balance equation
Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)
x = 0,1,2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis Population forecasting 49
Net migration
We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.
Demographic growth-balance equation
Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)
x = 0,1,2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis Population forecasting 49
Net migration
We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.
Demographic growth-balance equation
Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)
x = 0,1,2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis Population forecasting 49
Net migration
We need to estimatemigration data based ondifference in population numbers afteradjusting for births and deaths.
Demographic growth-balance equation
Gt(x ,x +1) = Pt+1(x +1)− Pt(x)+Dt(x ,x +1)Gt(B ,0) = Pt+1(0) −Bt +Dt(B ,0)
x = 0,1,2, . . . .
Note: “net migration” numbers also include errors
associated with all estimates. i.e., a “residual”.
Demographic forecasting using functional data analysis Population forecasting 49
Net migration
Demographic forecasting using functional data analysis Population forecasting 50
0 20 40 60 80 100
−20
000
2000
4000
6000
8000 Australia: male net migration (1973−2008)
Age
Net
mig
ratio
n
Net migration
Demographic forecasting using functional data analysis Population forecasting 50
0 20 40 60 80 100
−20
000
2000
4000
6000
8000 Australia: female net migration (1973−2008)
Age
Net
mig
ratio
n
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM
t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis Population forecasting 51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM
t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis Population forecasting 51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM
t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis Population forecasting 51
Stochastic population forecasts
Component models
Data: age/sex-specific mortality rates, fertilityrates and net migration.Models: Functional time series models formortality (M/F), fertility and net migration (M/F)assuming independence between componentsand coherence between sexes.Generate random sample paths of eachcomponent conditional on observed data.Use simulated rates to generate Bt(x),D Ft (x ,x +1), DM
t (x ,x +1) for t = n +1, . . . ,n +h ,assuming deaths and births are Poisson.
Demographic forecasting using functional data analysis Population forecasting 51
Simulation
Demographic growth-balance equation used to getpopulation sample paths.
Demographic growth-balance equation
Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)
x = 0,1,2, . . . .
10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.
Demographic forecasting using functional data analysis Population forecasting 52
Simulation
Demographic growth-balance equation used to getpopulation sample paths.
Demographic growth-balance equation
Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)
x = 0,1,2, . . . .
10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.
Demographic forecasting using functional data analysis Population forecasting 52
Simulation
Demographic growth-balance equation used to getpopulation sample paths.
Demographic growth-balance equation
Pt+1(x +1) = Pt(x)−Dt(x ,x +1)+Gt(x ,x +1)Pt+1(0) = Bt −Dt(B ,0) +Gt(B ,0)
x = 0,1,2, . . . .
10000 sample paths of population Pt(x), deathsDt(x) and births Bt(x) generated fort = 2004, . . . ,2023 and x = 0,1,2, . . . ,.This allows the computation of the empiricalforecast distribution of any demographicquantity that is based on births, deaths andpopulation numbers.
Demographic forecasting using functional data analysis Population forecasting 52
Forecasts of TFR
Demographic forecasting using functional data analysis Population forecasting 53
Forecast Total Fertility Rate
Year
TF
R
1920 1940 1960 1980 2000 2020
2000
2500
3000
3500
Population forecastsForecast population: 2028
Population ('000)
Age
Male Female
150 100 50 0 50 100 150
010
3050
7090
010
3050
7090
020
4060
8010
0
020
4060
8010
0
Age
20092009
Forecast population pyramid for 2028, along with80% prediction intervals. Dashed: actual populationpyramid for 2009.
Demographic forecasting using functional data analysis Population forecasting 54
Population forecasts
Demographic forecasting using functional data analysis Population forecasting 55
● ● ●●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●●
●●
●
1980 1990 2000 2010 2020 2030
810
1214
16Total females
Year
Pop
ulat
ion
(mill
ions
)
Population forecasts
Demographic forecasting using functional data analysis Population forecasting 55
● ● ●●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●●
●●
●
1980 1990 2000 2010 2020 2030
810
1214
16Total males
Year
Pop
ulat
ion
(mill
ions
)
Old-age dependency ratio
Demographic forecasting using functional data analysis Population forecasting 56
Old−age dependency ratio forecasts
Year
ratio
1920 1940 1960 1980 2000 2020
0.10
0.15
0.20
0.25
0.30
Advantages of stochastic simulation approach
Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)
Demographic forecasting using functional data analysis Population forecasting 57
Advantages of stochastic simulation approach
Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)
Demographic forecasting using functional data analysis Population forecasting 57
Advantages of stochastic simulation approach
Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)
Demographic forecasting using functional data analysis Population forecasting 57
Advantages of stochastic simulation approach
Functional data analysis provides a way offorecasting age-specific mortality, fertility andnet migration.Stochastic age-specific cohort-componentsimulation provides a way of forecasting manydemographic quantities with predictionintervals.No need to select combinations of assumedrates.True prediction intervals with specifiedcoverage for population and all derivedvariables (TFR, life expectancy, old-agedependencies, etc.)
Demographic forecasting using functional data analysis Population forecasting 57
Outline
1 A functional linear model
2 Bagplots, boxplots and outliers
3 Functional forecasting
4 Forecasting groups
5 Population forecasting
6 References
Demographic forecasting using functional data analysis References 58
Selected referencesHyndman, Shang (2010). “Rainbow plots, bagplots and boxplots forfunctional data”. Journal of Computational and Graphical Statistics19(1), 29–45Hyndman, Ullah (2007). “Robust forecasting of mortality and fertilityrates: A functional data approach”. Computational Statistics andData Analysis 51(10), 4942–4956Hyndman, Shang (2009). “Forecasting functional time series (withdiscussion)”. Journal of the Korean Statistical Society 38(3),199–221Shang, Booth, Hyndman (2011). “Point and interval forecasts ofmortality rates and life expectancy : a comparison of ten principalcomponent methods”. Demographic Research 25(5), 173–214Hyndman, Booth (2008). “Stochastic population forecasts usingfunctional data models for mortality, fertility and migration”.International Journal of Forecasting 24(3), 323–342Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:the product-ratio method with functional time series models”.Demography, to appearHyndman (2012). demography: Forecasting mortality, fertility,migration and population data.cran.r-project.org/package=demography
Demographic forecasting using functional data analysis References 59
Selected referencesHyndman, Shang (2010). “Rainbow plots, bagplots and boxplots forfunctional data”. Journal of Computational and Graphical Statistics19(1), 29–45Hyndman, Ullah (2007). “Robust forecasting of mortality and fertilityrates: A functional data approach”. Computational Statistics andData Analysis 51(10), 4942–4956Hyndman, Shang (2009). “Forecasting functional time series (withdiscussion)”. Journal of the Korean Statistical Society 38(3),199–221Shang, Booth, Hyndman (2011). “Point and interval forecasts ofmortality rates and life expectancy : a comparison of ten principalcomponent methods”. Demographic Research 25(5), 173–214Hyndman, Booth (2008). “Stochastic population forecasts usingfunctional data models for mortality, fertility and migration”.International Journal of Forecasting 24(3), 323–342Hyndman, Booth, Yasmeen (2012). “Coherent mortality forecasting:the product-ratio method with functional time series models”.Demography, to appearHyndman (2012). demography: Forecasting mortality, fertility,migration and population data.cran.r-project.org/package=demography
Demographic forecasting using functional data analysis References 59
å Papers and R code:robjhyndman.com
å Email: [email protected]