12/8/2011

1

Titus Beu 2011

Titus BeuUniversity “Babes-Bolyai”Department of Theoretical and Computational PhysicsCluj-Napoca, Romania

10. Approximation of tabulated functions10. Approximation of tabulated functions

Titus Beu 2011

Bibliography

Introduction

Interpolation and regression

Lagrange interpolating polynomial

Spline interpolation

Linear regression

12/8/2011

2

Titus Beu 2011

D.W. Marquardt, Journal of the Society for Industrial and Applied Mathematics, vol. 11, p. 431-441 (1963).

C.L. Lawson şi R. Hanson, Solving Least Squares Problems (Prentice-Hall, Englewood Cliffs, NJ, 1974).

B.P. Demidovich şi I.A. Maron, Computational Mathematics (MIR Publishers, Moskow, 1981).

C. Iacob, D. Homentcovschi, N. Marcov şi A. Nicolau, Matematici clasice şi moderne, vol. 4 (Editura tehnică, Bucureşti, 1983).

R.L. Burden şi J.D. Faires, Numerical Analysis, Third Edition (Prindle, Weber & Schmidt, Boston, 1985).

W.H. Press, S.A. Teukolsky, W.T. Vetterling şi B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, Second Edition (Cambridge University Press, Cambridge, 1992).

Beu, T. A., Numerical Calculus in C, Third Edition(MicroInformatica Publishing House, Cluj-Napoca, 2004).

Titus Beu 2011

The problem of approximating tabulated functions

Let f be a real function of a real variable:

specified by its values on a mesh of points, xi ∈ [α,β]:

Main goal: constructing new data points within the range of the known data points

Dependence typically represented by a continuous model function depending on a finite number of parameters:

Model function F and parameters aj are chosen such as to “optimally” approximate the original function f.

: [ , ] , [ , ]f α β α β→ ℜ ⊂ ℜ

( ) , 1,2, ,i if x y i n= = …

1( ; , , )mF F x a a= …

12/8/2011

3

Titus Beu 2011

Choice of the model function:

From an analytically convenient class of functions (polynomials, Gaussians etc.), which can be easily manipulated

From a theory in which the parameters have a well-established significance

From a compromise between the constraints and advantages of the first options

Distance between the original and the model function:

Can be used to characterize the quality of the approximation

Root mean square (RMS) or quadratic mean approximation – based on distance

Method of least squares – used in interpolation and regression

a

1/2

2

1

( , ) ( ) ( ; )

n

i i

i

d f F f x F x

=

= − ∑

Titus Beu 2011

Intepolation

Values yi are considered to be exact

Model function must satisfy the interpolation conditions:

F(x;a) – interpolant

xi – interpolation points

Model parameters aj – solving the system of interpolation conditions

Popular polynomial interpolants: Lagrange, spline, Akima

( , ) 0d f F =

a( ; ) , 1,2, ,i iF x y i n= = …

12/8/2011

4

Titus Beu 2011

Regression

Tabulated values yi originate from observations (measurements)

Associated measurement errors –

standard deviations σi

Merit functions used practically –Sum of squared residuals:

Chi-Square:

a

22

21

1( ; )

n

i i

i i

y F xχσ=

= − ∑

( , ) minja

d f F =

a

2

1

( ; )

n

i i

i

S y F x

=

= − ∑

Titus Beu 2011

Regression

Minimum of χ2 with respect to the model parameters:

Model parameters – solution of system

Approach – regression or curve fittingby the method of least squares

2

0, 1,2, ,j

j ma

χ∂= =

∂…

12/8/2011

5

Titus Beu 2011

One of the oldest and most general interpolation schemes

Basis for classes of numerical integration and differentiation formulas

Assume the known values:

Define a polynomial satisfying the interpolation conditions:

Family of auxiliary polynomials vanishing at all tabulation points except for xi:

( ) , 1,2, ,i if x y i n= = …

( ) , 1,2, ,m i iP x y i n= = …

( )

( ) ( )( )

jj i

i j ij ii j

j i

x x

p x p xx x

δ≠

≠

∏ −

= → =∏ −

Titus Beu 2011

Polynomial

satisfies the interpolation conditions:

Lagrange interpolating polynomial:

1

1

( ) ( )

n

n i i

i

P x p x y−

=

= ∑

1

1 1

( ) ( ) , 1,2, ,

n n

n j i j i ij i j

i i

P x p x y y y j nδ−

= =

= = = =∑ ∑ …

1

1

( )

( )

( )

n

n jj i

n ini

i jj i

x x

P x y

x x

≠−

=

≠

∏ −

=

∏ −

∑

12/8/2011

6

Titus Beu 2011

Uniqueness

Assume there exists another polynomial of order n − 1 satisfying the same interpolation conditions:

At the tabulation points:

Being a polynomial of order n having n zeroes it must be identically null:

1( ) , 1,2, ,n i iP x y i n− = =� …

1 1( ) ( ) 0, 1,2, ,n i n iP x P x i n− −− = =� …

1 1( ) ( )n nP x P x− −≡�

Titus Beu 2011

Particular cases

Linear interpolation, n = 2:

Parabolic interpolation, n = 3:

2 11 1 2

1 2 2 1

( )x x x x

P x y yx x x x

− −= +

− −

2 3 1 3 1 22 1 2 3

1 2 1 3 2 1 2 3 3 1 3 2

( )( ) ( )( ) ( )( )( )

( )( ) ( )( ) ( )( )

x x x x x x x x x x x xP x y y y

x x x x x x x x x x x x

− − − − − −= + +

− − − − − −

12/8/2011

7

Titus Beu 2011

//===========================================================================

float Lagrange(float xi[], float yi[], int ni, float x)//---------------------------------------------------------------------------

// Evaluates the Lagrange interpolating polynomial at point x//

// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points

// ni - no. of interpolation points// x - argument

//---------------------------------------------------------------------------{

float p, y;int i, j;

y = 0.0;

for (i=1; i<=ni; i++) {p = 1.0;

for (j=1; j<=ni; j++)if (j != i) p *= (x-xi[j])/(xi[i]-xi[j]);

y += p*yi[i];}

return y;}

//===========================================================================

float Lagrange(float xi[], float yi[], int ni, float x)//---------------------------------------------------------------------------

// Evaluates the Lagrange interpolating polynomial at point x//

// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points

// ni - no. of interpolation points// x - argument

//---------------------------------------------------------------------------{

float p, y;int i, j;

y = 0.0;

for (i=1; i<=ni; i++) {p = 1.0;

for (j=1; j<=ni; j++)if (j != i) p *= (x-xi[j])/(xi[i]-xi[j]);

y += p*yi[i];}

return y;}

Titus Beu 2011

Inverse interpolation

Finding the argument for which the

Lagrange interpolant has given value y:

Application – approximate solutions of algebraic equations f(x) = 0

Calculate yi for n arguments xi close to the solution and take y = 0

Pitfalls of Lagrange interpolation

Oscillations for large no. of tabulation points due to large order of polynomial

1

( )

( )

n jj i

ii ji

j i

y y

x xy y

≠

=≠

∏ −

=∏ −∑

12/8/2011

8

Titus Beu 2011

Spline – instrument used in old times to draw smooth curves through given points

Division of interval [α,β]:

Function f : [α,β] → ℜ satisfying the interpolation conditions:

Spline function of order m relative to division ∆ of interval [α,β]:function S : [α,β] → ℜ of class Cm−1[α,β], whose restrictions Si(x) on subintervals [xi,xi+1] are polynomials of order m

Piece-wise smooth function – first (m−1) derivatives are continuous on [α,β]

Cubic spline function ∈ C2[α,β] – restrictions Si(x) are 3rd order polynomials

1 2: nx x xα β∆ = < < < =…

( ), 1,2, ,i iy f x i n= = …

3 21( ) [ , ], 1,2, , 1i i i i i i iS x Ax B x C x D x x x i n+= + + + = −…∈

Titus Beu 2011

2nd order derivative – linear function ( – 2nd order derivatives in mesh points):

or:

By integration:

1

1

( ) i ii i

i i i

D DS x D

x x x x

′′+

+

−−=

− −

� ��

1 11

( ) ( )( ) , , 1,2, , 1i i i ii i i i

i

D x x D x xS x h x x i n

h

+ +′′+

− + −= = − = −� �

…

2 21 1

3 31 1

( ) ( )( )

2

( ) ( )( )

6

i i i ii i

i

i i i ii i i

i

D x x D x xS x C

h

D x x D x xS x C x C

h

+ +′ ′

+ + ′ ′′

− − −= +

− + −= + +

� �

� �

iD�

12/8/2011

9

Titus Beu 2011

Interpolation conditions – integration constants:

Finally:

2

21 1 1

1 1

( ) 6( )

6

i ii i i ii i i

i i i i ii i i i

D hC x C yS x y

S x y D hC x C y

′ ′′

+ + + ′ ′′+ +

+ + = = ⇒ = + + =

�

�

1 1

1 1 1 1

( )

6

( )

6

i i i i ii

i

i i i i i i i i ii

i

y y D D hC

h

x y x y x D x D hC

h

+ +′

+ + + +′′

− −= −

− −= +

� �

� �

Titus Beu 2011

Restriction coefficients:

1

1 1

2 21 1 1 2

3 3 21 1 1 1

6

2

2

6 3

i ii

i

i i i ii

i

i i i i i ii i i

i i

i i i i i i i i i ii

i i

D DA

h

D x D xB

h

D x D x y yC Ah

h h

D x D x y x y x B hD

h h

+

+ +

+ + +

+ + + +

− = − = − − = + − − − = + −

� �

� �

� �

� �

12/8/2011

10

Titus Beu 2011

Continuity of 1st order derivatives:

System for 2nd order derivatives:

Supplementary conditions for end-points x1 and xn:

Hence:

1( ) ( ), 2, 3, , 1i i i iS x S x i n′ ′− = = −…

11 1 11 1

16 3 6

2,3, , 1

i ii i i i i ii i i

i i

y yh h h h y yD D D

h h

i n

+− − −− +

−

−+ −+ + = −

= −

� � �

…

1 1 1 1( ) , ( )n n nS x y S x y′ ′ ′ ′−= =

1 1 2 11 2 1

1

1 1 11

1

3 6

6 3n n n n

n n nn

h h y yD D y

h

h h y yD D y

h

′

′− − −−

−

−+ = −

−+ = −

� �

� �

Titus Beu 2011

Second order derivatives in mesh points from tridiagonal system:

Coefficients:

1 1 1 2 1

1 1

1

, 2, 3, , 1i i i i i i i

n n n n n

b D c D d

a D b D c D d i n

a D b D d

− +

−

+ = + + = = − + =

� �

� � � …

� �

2 11 1 1 1 1 1 1

1

1 11 1

1

11 1

1

0, 2 , , 6

, 2( ), , 6

, 2 , 0, 6

i i i ii i i i i i i i

i i

n nn n n n n n n

n

y ya b h c h d y

h

y y y ya h b h h c h d

h h

y ya h b h c d y

h

′

+ −− −

−

′ −− −

−

− = = = = − − − = = + = = − − = = = = −

12/8/2011

11

Titus Beu 2011

Calculation steps:

1. Second order derivatives in mesh points from tridiagonal system

2. Restriction coefficients Ai, Bi, Ci, Di

3. Evaluation of the intepolant for particular arguments

Advantages:

Reduced oscillations – better approximation than the Lagrange polynomial

Order of the restrictions is independent on the no. of interpolation points.

Pittfall – too few tabulation points and large variations of observations

iD�

Titus Beu 2011

//===========================================================================

void Spline(float xi[], float yi[], int ni, float x[], float y[], int n,float a[], float b[], float c[], float d[],

float d1, float dn, int iopt)//---------------------------------------------------------------------------

// Calculates the coefficients of the restrictions of he cubic spline// function relative to an interpolation point mesh and evaluates the

// polynomial on an evaluation point mesh//

// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points

// ni - no. of interpolation points// x[] - abscissas of the evaluation points

// y[] - evaluated polynomial values (output)// n - no. of evaluation points

// a..d[]- coefficients of restrictions// d1,dn - 1st derivatives for end-points

// iopt - iopt == 0, calculates the 1st order derivatives for end-points// iopt /= 0, takes input values d1 and dn

//---------------------------------------------------------------------------{

float di, dm, hi, hm, xx, xp;int i, ip;

//===========================================================================

void Spline(float xi[], float yi[], int ni, float x[], float y[], int n,float a[], float b[], float c[], float d[],

float d1, float dn, int iopt)//---------------------------------------------------------------------------

// Calculates the coefficients of the restrictions of he cubic spline// function relative to an interpolation point mesh and evaluates the

// polynomial on an evaluation point mesh//

// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points

// ni - no. of interpolation points// x[] - abscissas of the evaluation points

// y[] - evaluated polynomial values (output)// n - no. of evaluation points

// a..d[]- coefficients of restrictions// d1,dn - 1st derivatives for end-points

// iopt - iopt == 0, calculates the 1st order derivatives for end-points// iopt /= 0, takes input values d1 and dn

//---------------------------------------------------------------------------{

float di, dm, hi, hm, xx, xp;int i, ip;

12/8/2011

12

Titus Beu 2011

// parabolic approximation for 1st derivative at end-points

if (iopt==0) {d1 = (yi[2] - yi[1])/(xi[2] - xi[1])

- (yi[3] - yi[2])/(xi[3] - xi[2])+ (yi[3] - yi[1])/(xi[3] - xi[1]);

dn = (yi[ni ] - yi[ni-1])/(xi[ni ] - xi[ni-1])- (yi[ni-1] - yi[ni-2])/(xi[ni-1] - xi[ni-2])

+ (yi[ni ] - yi[ni-2])/(xi[ni ] - xi[ni-2]);}

// coefficients of the system of 2nd order derivativehi = 0.0; di = d1;

for (i=1; i<=(ni-1); i++) {hm = hi; hi = xi[i+1] - xi[i];

dm = di; di = (yi[i+1] - yi[i])/hi;a[i] = hm; b[i] = 2*(hm + hi); c[i] = hi; d[i] = 6*(di - dm);

}a[ni] = hi; b[ni] = 2*hi; c[ni] = 0.0; d[ni] = 6*(dn - di);

TriDiag(a,b,c,d,ni); // system solution => 2nd derivatives in d

for (i=1; i<=(ni-1); i++) { // coefficients of restrictions

ip = i+1;xx = xi[i]; xp = xi[ip]; hi = xp - xx;

a[i] = (d[ip] - d[i])/(6*hi);b[i] = (d[i]*xp - d[ip]*xx)/(2*hi);

c[i] = (d[ip]*xx*xx - d[i]*xp*xp)/(2*hi)+ (yi[ip] - yi[i])/hi - a[i]*hi*hi;

d[i] = (d[i]*xp*xp*xp - d[ip]*xx*xx*xx)/(6*hi)+ (yi[i]*xp - yi[ip]*xx)/hi - b[i]*hi*hi/3;

}

// parabolic approximation for 1st derivative at end-points

if (iopt==0) {d1 = (yi[2] - yi[1])/(xi[2] - xi[1])

- (yi[3] - yi[2])/(xi[3] - xi[2])+ (yi[3] - yi[1])/(xi[3] - xi[1]);

dn = (yi[ni ] - yi[ni-1])/(xi[ni ] - xi[ni-1])- (yi[ni-1] - yi[ni-2])/(xi[ni-1] - xi[ni-2])

+ (yi[ni ] - yi[ni-2])/(xi[ni ] - xi[ni-2]);}

// coefficients of the system of 2nd order derivativehi = 0.0; di = d1;

for (i=1; i<=(ni-1); i++) {hm = hi; hi = xi[i+1] - xi[i];

dm = di; di = (yi[i+1] - yi[i])/hi;a[i] = hm; b[i] = 2*(hm + hi); c[i] = hi; d[i] = 6*(di - dm);

}a[ni] = hi; b[ni] = 2*hi; c[ni] = 0.0; d[ni] = 6*(dn - di);

TriDiag(a,b,c,d,ni); // system solution => 2nd derivatives in d

for (i=1; i<=(ni-1); i++) { // coefficients of restrictions

ip = i+1;xx = xi[i]; xp = xi[ip]; hi = xp - xx;

a[i] = (d[ip] - d[i])/(6*hi);b[i] = (d[i]*xp - d[ip]*xx)/(2*hi);

c[i] = (d[ip]*xx*xx - d[i]*xp*xp)/(2*hi)+ (yi[ip] - yi[i])/hi - a[i]*hi*hi;

d[i] = (d[i]*xp*xp*xp - d[ip]*xx*xx*xx)/(6*hi)+ (yi[i]*xp - yi[ip]*xx)/hi - b[i]*hi*hi/3;

}

Titus Beu 2011

// evaluates interpolant

for (i=1; i<=n; i++) { // loop over argumentsxx = x[i];

ip = 1; while (xx > xi[ip+1]) ip++; // index of restrictiony[i] = ((a[ip]*xx + b[ip])*xx + c[ip])*xx + d[ip];

}}

// evaluates interpolant

for (i=1; i<=n; i++) { // loop over argumentsxx = x[i];

ip = 1; while (xx > xi[ip+1]) ip++; // index of restrictiony[i] = ((a[ip]*xx + b[ip])*xx + c[ip])*xx + d[ip];

}}

12/8/2011

13

Titus Beu 2011

Simplest curve fitting by the method of least squares to a straight line

Model function:

Merit function:

Minimum condition:

( ; , )F x a b ax b= +

2 2

21

1( , ) ( )

n

i i

i i

a b y ax bχσ=

= − −∑

2

21

2

21

12 ( ) 0

12 ( ) 0

n

i i i

i in

i i

i i

y ax b xa

y ax bb

χ

σ

χ

σ

=

=

∂ ≡ − − − = ∂∂ ≡ − − − = ∂

∑

∑

Titus Beu 2011

Defining

Linear system for model parameters a and b:

Optimized model parameters:

2

2 2 21 1 1

21 1

1, ,

,

n n n

i ix xx

i i ii i in n

iy xy i i

i ii

x xs s s

ys s x y

σ σ σ

σ

= = =

= =

= = = = =

∑ ∑ ∑

∑ ∑

xx x xy

x y

s a s b s

s a s b s

+ = + =

2

,

,

xy x y

y xx x xy

xx x

s s s sa

s s s sb s s s

−=

∆−

= ∆ = −∆

12/8/2011

14

Titus Beu 2011

Variance (probable uncertainty) of the model parameters (assuming independent observations):

Derivatives of model parameters:

In particular:

Probable uncertainties (variances) of model parameters:

2 2

2 2 2 2

1 1

,

n n

a i b ii ii i

a b

y yσ σ σ σ

= =

∂ ∂ = = ∂ ∂ ∑ ∑

2 2,

i x xx x i

i ii i

s x s s s xa b

y yσ σ

− −∂ ∂= =

∂ ∂∆ ∆

2 2 2 2 2 2 2

2

2 2 2 21

2 2 ( )ni i x x xx x x xx x

a

i i

s x s x s s s s s s s s s s s s sσ

σ=

− + − + −= = = =

∆∆ ∆ ∆∑

, xxa b

ssσ σ= =

∆ ∆

Titus Beu 2011

Example of linear fitting for exponential dependence:

By taking the logarithm:

a and b′ are adjusted relative to the data set (xi,ln yi)

Merit functional S:a = 0.9100, b = 0.0900

Merit functional χ2 :a = 0.9983, b = −0.1681

fitted line goes through all error bars!

axy be=

ln

ln

y ax b

y y

b b

′ ′= +

′ =

′ =

12/8/2011

15

Titus Beu 2011

//===========================================================================

void LinFit(float x[], float y[], float sigmy[], int ndat, int iopt,float &a, float &b, float &sigma, float &sigmb, float &chi2)

//---------------------------------------------------------------------------// Determines by Chi-square regression the model coefficients a and b and the

// associated probable uncertainties, sigma and sigmb, for a linear model,// F(x;a,b) = a x + b, relative to a set of observed data

// x[] - abscissas of observed data// y[] - ordinates of observed data

// sigmy[] - standard deviations associated with y[]// ndat - no. of observations

// iopt - iopt == 0 Sum of squares criterion is used// iopt /= 0 Chi-square criterion is used

// a, b - model parameters (output)// sigma - probable uncertainties of model parameters (output)

// sigmb// chi2 - Chi-square sum for optimized model parameters (output)

//---------------------------------------------------------------------------{

float f, s, sx, sy, sxx, sxy;int i;

if (iopt == 0) for (i=1; i<=ndat; i++) sigmy[i] = 1.0;

s = sx = sy = sxx = sxy = 0.0;

for (i=1; i<=ndat; i++) {f = 1.0/(sigmy[i]*sigmy[i]);

s += f;sx += x[i]*f; sxx += x[i]*x[i]*f;

sy += y[i]*f; sxy += x[i]*y[i]*f; }

f = 1.0/(s*sxx - sx*sx);

a = (s *sxy - sx*sy )*f; sigma = sqrt(s*f);b = (sy*sxx - sx*sxy)*f; sigmb = sqrt(sxx*f);

chi2 = 0.0;

for (i=1; i<=ndat; i++)chi2 += pow((y[i] - a*x[i] - b)/sigmy[i],2);

}

//===========================================================================

void LinFit(float x[], float y[], float sigmy[], int ndat, int iopt,float &a, float &b, float &sigma, float &sigmb, float &chi2)

//---------------------------------------------------------------------------// Determines by Chi-square regression the model coefficients a and b and the

// associated probable uncertainties, sigma and sigmb, for a linear model,// F(x;a,b) = a x + b, relative to a set of observed data

// x[] - abscissas of observed data// y[] - ordinates of observed data

// sigmy[] - standard deviations associated with y[]// ndat - no. of observations

// iopt - iopt == 0 Sum of squares criterion is used// iopt /= 0 Chi-square criterion is used

// a, b - model parameters (output)// sigma - probable uncertainties of model parameters (output)

// sigmb// chi2 - Chi-square sum for optimized model parameters (output)

//---------------------------------------------------------------------------{

float f, s, sx, sy, sxx, sxy;int i;

if (iopt == 0) for (i=1; i<=ndat; i++) sigmy[i] = 1.0;

s = sx = sy = sxx = sxy = 0.0;

for (i=1; i<=ndat; i++) {f = 1.0/(sigmy[i]*sigmy[i]);

s += f;sx += x[i]*f; sxx += x[i]*x[i]*f;

sy += y[i]*f; sxy += x[i]*y[i]*f; }

f = 1.0/(s*sxx - sx*sx);

a = (s *sxy - sx*sy )*f; sigma = sqrt(s*f);b = (sy*sxx - sx*sxy)*f; sigmb = sqrt(sxx*f);

chi2 = 0.0;

for (i=1; i<=ndat; i++)chi2 += pow((y[i] - a*x[i] - b)/sigmy[i],2);

}