25
Functional linear models Functional linear models 0 50 100 150 200 250 300 350 -30 -20 -10 0 10 20 D ay D eg C M ean Tem perature 0 50 100 150 200 250 300 350 0 2 4 6 8 10 12 D ay D eg C M ean Precipitation

Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Embed Size (px)

Citation preview

Page 1: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Functional linear modelsFunctional linear models

0 50 100 150 200 250 300 350

-30

-20

-10

0

10

20

Day

Deg

C

Mean Temperature

0 50 100 150 200 250 300 3500

2

4

6

8

10

12

Day

Deg

C

Mean Precipitation

Page 2: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Three types of linear model to Three types of linear model to consider:consider:

1.1. Response is a function; covariates Response is a function; covariates are multivariate.are multivariate.

2.2. Response is scalar or multivariate; Response is scalar or multivariate; covariates are functional.covariates are functional.

3.3. Both response and covariates are Both response and covariates are functional.functional.

Page 3: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Functional response with Functional response with multivariate covariatesmultivariate covariates

Response: Response: yyii(t), i=1,…,N(t), i=1,…,N Covariate: Covariate: xxi1i1,…, x,…, xipip

Model:Model:

1 1 ...i i p ip iy t t x t x t

Page 4: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

How does daily temperature How does daily temperature depend on climate zone?depend on climate zone?

35 Canadian temperature stations, 35 Canadian temperature stations, divided into four zones: Atlantic, divided into four zones: Atlantic, Pacific, Continental, and Arctic.Pacific, Continental, and Arctic.

Response is 30-year average daily Response is 30-year average daily temperature.temperature.

A functional one-way analysis of A functional one-way analysis of variance, set up to have a main variance, set up to have a main effect, and zone effects summing to effect, and zone effects summing to zero. zero.

Page 5: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Analyzing the dataAnalyzing the data

This is straightforward. This is straightforward. If If Y(t)Y(t) is the N-vector of response is the N-vector of response

functions, functions, ββ(t)(t) is the 5-vector of is the 5-vector of regression functions (main effect + regression functions (main effect + zone effects), then the LS estimate iszone effects), then the LS estimate is

ββ(t) = (X’X)(t) = (X’X)-1-1X’ Y(t)X’ Y(t) . .

Page 6: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Main and zone effect Main and zone effect functionsfunctions

0 50 100 150 200 250 300 350-25

-20

-15

-10

-5

0

5

10

15

20

Day

Regression Functions

InterceptAtlanticPacificContinentalArctic

Page 7: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Assessing effectsAssessing effects

We probably want to assess effects We probably want to assess effects pointwisepointwise: For what times : For what times tt is an is an effect substantial?effect substantial?

This can be done using F-ratios This can be done using F-ratios conditional on conditional on tt, pointwise , pointwise confidence bands, etc.confidence bands, etc.

The multiple comparison problem is The multiple comparison problem is especially challenging here. especially challenging here.

Page 8: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Response is scalar, Covariate is Response is scalar, Covariate is a single functional variablea single functional variable

Response: Response: yyii , i=1,…,N , i=1,…,N Covariate: Covariate: xxi i (t)(t) Model:Model:

0

S

i i iy s x s ds e

Page 9: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

We have to smooth!We have to smooth!

The technical and conceptual issues The technical and conceptual issues become much more interesting when the become much more interesting when the covariate is functional.covariate is functional.

A functional covariate is effectively an A functional covariate is effectively an infinite-dimensional predictor for a finite set infinite-dimensional predictor for a finite set of N responses. We can fit the data exactly!of N responses. We can fit the data exactly!

Smoothing becomes essential; without it, Smoothing becomes essential; without it, ββ(t)(t) will be unacceptably rough, and we will be unacceptably rough, and we won’t learn anything useful. won’t learn anything useful.

Page 10: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Predicting log annual Predicting log annual precipitation from the precipitation from the temperature profilestemperature profiles

Can we determine how much precipitation Can we determine how much precipitation a weather station will receive from the a weather station will receive from the shape of the temperature profile?shape of the temperature profile?

What roughness penalty should we use to What roughness penalty should we use to smooth smooth ββ(t)(t) ? ?

We penalize the size of We penalize the size of (2(2ππ/365)/365)22DDββ+D+D33ββ,, the the harmonic accelerationharmonic acceleration of of ββ(t)(t) . This . This

smooths towards a shifted sinusoid. smooths towards a shifted sinusoid.

Page 11: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The smoothed regression The smoothed regression functionfunction

Annual Annual precipitation is precipitation is determined by: (1) determined by: (1) spring spring temperature, and temperature, and (2) by the contrast (2) by the contrast between late between late summer and fall summer and fall temperatures. temperatures.

0 50 100 150 200 250 300 350-1.5

-1

-0.5

0

0.5

1

1.5

x 10-3

Day

Re

gre

ssio

n F

un

ctio

n

Page 12: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The fit to the dataThe fit to the data

The fit is The fit is good. good.

We see We see clusters of hi-clusters of hi-precip. marine precip. marine stations, and stations, and of continential of continential stations. stations.

Arctic stations Arctic stations have the least have the least precip. precip.

2 2.5 32

2.5

3

3.5

Predicted Annual Precipitation

Act

ua

l An

nu

al P

reci

pita

tion

R-sqrd = 0.82

Page 13: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

What about both the response What about both the response and covariate being functional?and covariate being functional? Response Response y(t)y(t), covariate , covariate x(s) x(s) or or x(s,t).x(s,t).Here we have a lot of possibilities. We can Here we have a lot of possibilities. We can

predict predict y(t) y(t) using the shape of using the shape of x(s,t)x(s,t) over: over:

all of all of ss, especially for periodic data,, especially for periodic data, only at only at s = ts = t, concurrent influence only, , concurrent influence only,

or for some delay or for some delay s = t – s = t – δδ,, s s t, t, no feed forward,no feed forward, some region some region ΩΩtt depending on depending on tt..

Page 14: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

Predicting the precipitation Predicting the precipitation profile from the temperature profile from the temperature

profileprofile The model is:The model is:

365

0( ) ,i i iy t t s t x s ds t

In this case we have to smooth In this case we have to smooth ββ(s,t)(s,t) with respect to both with respect to both ss and and tt..

Page 15: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The regression functionThe regression function

Page 16: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 4000.3

0.35

0.4

0.45

0.5

0.55

Day

Lo

g P

reci

pita

tion

montreal R2 = 0.95848

Page 17: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 400-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Day

Lo

g P

reci

pita

tion

vancouvr R2 = 0.79446

Page 18: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 400-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Day

Lo

g P

reci

pita

tion

winnipeg R2 = 0.84043

Page 19: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The concurrent modelThe concurrent model

This time, we’ll only use temperature at This time, we’ll only use temperature at time time tt to predict precipitation at time to predict precipitation at time tt::

i i iy t t t x t t

Page 20: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The regression functionsThe regression functions

The influence of The influence of temperature temperature is nearly is nearly constant over constant over the year.the year.

Let’s see how Let’s see how the two fits the two fits compare.compare. 0 50 100 150 200 250 300 350 400

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Day

Re

gre

ssio

n F

un

ctio

n

InterceptTemperature

Page 21: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 4000.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Day

Lo

g P

reci

pita

tion

montreal R2 = 0.79645

DataConcurrentFull

Page 22: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 400-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Day

Lo

g P

reci

pita

tion

vancouvr R2 = 0.82636

DataConcurrentFull

Page 23: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

0 50 100 150 200 250 300 350 400-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Day

Lo

g P

reci

pita

tion

winnipeg R2 = 0.35819

DataConcurrentFull

Page 24: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The historical linear modelThe historical linear model

When the functions are not periodic, When the functions are not periodic, it may not be reasonable to assume it may not be reasonable to assume that that x(s)x(s) can influence can influence y(t)y(t) when when s > s > tt..

The historical linear model is The historical linear model is described in described in Applied Functional Data Applied Functional Data AnalysisAnalysis, and in talk at this , and in talk at this conference by Nicole Malfait. conference by Nicole Malfait.

Page 25: Functional linear models. Three types of linear model to consider: 1. Response is a function; covariates are multivariate. 2. Response is scalar or multivariate;

The concurrent model and The concurrent model and differential equationsdifferential equations

One important extension of the One important extension of the concurrent model is to the fitting of concurrent model is to the fitting of data by a differential equation.data by a differential equation.

A simple example isA simple example is

i i i iDy t t y t t x t t