Lecture 9
Interpolation and Splines
Lingo
Interpolation – filling in gaps in data
Find a function f(x) that
1) goes through all your data points
2) does something sensible in between
Lingo
Splines – a broad class of ways of performing interpolation
(we’ll get to the details, eventually)
Find a function f(x) that
1) goes through all your data points
(observations)
2) does something sensible in between
(prior information)
Why not just use least-squares?
Remember this?
mest = mA + M [ dobs – GmA]
where M = [GTCd-1G + Cm
-1]-1 GT Cd-1
m – a vector of all the points at which you want to estimate the function, including the points for which you have observations
d – a vector of just those points where you have observations
So the equation Gm=d is very simple, a model parameter equals the data when the corresponding observation is available:
…0 … 0 1 0 … 0…
…mi
…
…dj
… =
Just a single “1” per row
You then implement a smoothness constraint through minimizing |Dm|2, where D is some measure of the non-smoothness of m
Thus mA= 0 and Cm-1 = 2DTD and Cd=I
…0 … 1 -2 1 … 0…
D =
One possibility is to use the finite-difference approximation of the second derivative
First derivative
[dm/dx]i (1/x) mi – mi-1
mi – mi-1
Second derivative
[d2m/dx2]i [dm/dx]i+1 - [dm/dx]i
= mi+1 – mi – mi + mi-1
= mi+1 – 2mi + mi-1
example101 equally spaced along the x-axis
So 101 values of the function f(x)
40 of these values measured (the data, d)the rest are unknown
Two prior informationminimize 2nd derivative for interior 99 x’sminimize 1st derivative at left and right x’s
(nice to have the same numberof priors as unknowns, but notrequired)
= 10-6
data
result
f(x)
x
can be chosen by trail and error
but usually the result fairly insensitive to , as long as its
small
varying over six orders of magnitude
log 10
(Tot
al E
rror
)log10()x
f(x)
A purist might say that this is not really interpolation, because the curve goes through the data
only in the limit
but for small ’sthe error is extremely small
an aside
Construct an equation F m = h as follows:
G
D
d
m =
then note [FTF]-1FT h = [GTG+2 DTD]-1GT d
so if you want, you can just append D to the bottom of G and solve by simple least-squares
solved via solved via[FTF]-1FT h [GTG+2 DTD]-1GT d
exactly the same!
another reason to work with
F m = h G
D
d
m =
both G and D, and therefore F, too, are mostly zero
(that is, they’re sparse matrices)
very efficient algorithms are available for solving Fm=h
in the least-squares sense when F is a sparse matrix
(note GTG and DTD are not as sparse as G or D
and [GTG and DTD]-1 is not sparse at all)
2D Example
(here a sparse solver would really be useful, for the number of unknowns is very large)
21 unknowns
21 u
nk
now
ns
2121=441 unknowns
44 observed data
Prior information:
2f = d2f/dx2 + d2f/dy2 = 0 in interior of the box
nf = 0 on edges of box
… a generalization of the 1D case
results
comparison
one limitation of this method is that it is discrete
it only gives the unknown function at specific, prescribed values of xi
one might prefer to have an analytic formula for the value of the function at
any x
LINGO
an analytic formula that gives the value of the function at any x is
called an interpolant
high order polynomial
something that sound like a good idea but isn’t
even though an N-1 polynomal can computed to pass through any N points
example: 10th order polynomial fit to 11 points
Big swings not what we hoped for
solution
simple functione.g. a low order polynomial
that is valid in some interval near xi
obviously, we need many such polynomials to over the whole x-axis
This approach is called a spline
we’ve all used one already -linear splines
xxi xi+1
y iy i+
1y
in this intervaly(x) = yi + (yi+1-yi)(x-xi)/(xi+1-xi)
cubic splines – somewhat more complicatedbut a lot nicer …
xxi xi+1
y iy i+
1
y
cubic a+bx+cx2+dx3 in this interval
a different cubic in this interval
counting up unknowns …
xxi xi+1
y iy i+
1y
four coefficients a, b, c, d in every interval
unknownsN dataN-1 intervals4 coefficients per interval4(N-1)=4N-4 coefficientstotal=4N-4 unknowns
constraintscurve goes thru point at end of its interval
2(N-1)=2N-2dy/dx match at interior points
N-2 constraintsd2y/dx2 match at interior points
N-2 constraintsd2y/dx2 =0 at end points
2 constraintstotal: 4N-4 constraints
formulating the cubic spline problem in an efficient manner
f(x) with N observations (xi, fi)
let hi = xi = xi+1- xi
and
fi = fi+1-fi
Si(x) are cubic polynomials, one for each interval
Let the 2nd derivatives have values
d2Si/dx2=y”i at the left hand end of its interval
But since the second derivative is presumed continuous across intervals, d2Si/dx2=y”i+1 on the right hand side of its interval too.
since Si(x) is a cubic, its second derivative is a linear function
So within an interval d2Si/dx2 varies linearly
d2Si/dx2 = y”i (xj+1-x)/hj + y”i+1 (x-xj)/hj
we’ll wind up solving for these y”i’s
now integrate twice to get Si(x)
d2Si/dx2 = y”i (xj+1-x)/hj + y”i+1 (x-xj)/hj
Si(x) = y”i (xj+1-x)3/(6hj) + y”i+1 (x-xj)3/(6hj) +
ci(x-xi) + di(xi+1-x)
where ci and di are integration constants
now choose the integration constants ci and di such that the the cubic goes through the data points. That is, Si(xi)=fi and Si(xi+1)=fi+1
this leads to
ci = fi+1/hi – y”i+1hi/6
di = fi/hi – y”ihi/6
so
Si(x) = y”i (xj+1-x)3/(6hj)
+ y”i+1 (x-xj)3/(6hj)
+ {fj+1/hi – y”i+1hi/6}(x-xi)
+ {fi/hi – y”ihi/6}(xi+1-x)
where ci and di are integration constants
but we still haven’t implemented the continuity of dS/dx condition …
so we compute the derivative
S’i(x) = dSi/dx =
y”i (xi+1-x)2/(2hi)
+ y”i+1 (x-xi)2/(2hi)
+ {fi+1/hi – y”i+1hi/6}
- {fi/hi – y”ihi/6}
= ½y”i (xi+1-x)2hi + ½y”i+1 (x-xi)2/hi
+ fi+1/hi – (y”i+1-y”i)hi/6
now require the first derivative match across intervals: Si-1’(xi)=Si’(xi)
this leads to an equation for the unknown y”i
hi-1y”i-1 + 2(hi+hi-1)y”i + hiy”i+1 = bj
with bj = 6fi/hi – 6fi-1/hi-1
the equation for the unknown y”i
hi-1y”i-1 + 2(hi+hi-1)y”i + hiy”i+1 = bi
is just a matrix equationi=1: h0y”0 + 2(h1+h0)y”1 + h1y”2 = b1
i=2: h1y”1 + 2(h2+h1)y”2 + h2y”2 = b2
i=2: h2y”2 + 2(h3+h2)y”3 + h3y”3 = b3
…i=N-1 hN-2y”N-2 + 2(hN-1+hN-2)y”N-1 + hN-1y”N = bN-1
i=N hN-1y”N-1 + 2(hN+hN-1)y”N + hNy”N+1 = bN
= we’ll discuss the issue raise by y”0 and y”N+1 in a moment
the matrix equation, with gi=2(hi+hi-1), is
h0 g1 h1
h1 g2 h2
h2 g3 h3
… hN-2 gN-1 hN-1
hN-1 gN hN
y”0
y”1
y”2
y”3
…y”N
y”N+1
b0
b1
b2
b3
…bN
bN+1
=
A Tridiagonal Matrix, by the way. Very fast solvers are available …
the matrix equation, with gi=2(hi+hi-1), is
h0 g1 h1
h1 g2 h2
h2 g3 h3
… hN-2 gN-1 hN-1
hN-1 gN hN
y”0
y”1
y”2
y”3
…y”N
y”N+1
b0
b1
b2
b3
…bN
bN+1
=
I’ve written this as if there were two extra points, one to the left of the first point and one to the right of the last point. Of course, there aren’t. The way to handle this is to prescribe y”0 and y”N+1 and move them to the r.h.s. of the equation.
N
N+2
moving over these two now-specified unknowns
g1 h1
h1 g2 h2
h2 g3 h3
… hN-1 gN hN
hN gN
y”1
y”2
y”3
…y”N
b1 – h0y”0
b2
b3
…bN – hN+1y”N+1
=
we can set y”0 and y”N to whatever we want. A simple choice is zero, in which case the splines are called natural cubic splines
N
N
example of cubic spline interpolation
very easy in MatLab
new_y = spline(x,y,new_x);