pade and prony technique

8/13/2019 pade and prony technique

1/12

1

SIGNAL MODELING (1)

INTRODUCTION

Signal modeling:

The central issues in signal modeling:

Modeling a signal as the output of a linear shift-invariant filter:

Techniques for determining model parameters:

Optimization Theory:

PART I. DETERMINISTIC SIGNAL MODELINGThe signal model :

THE LEAST SQUARES (DIRECT) METHOD

The modeling error:

The error function to be minimized:

Determining the filter coefficients:

THE PAD APPROXIMATION

Modeling error:Determining the filter coefficients:

THE PRONYS METHOD

Alternative modeling error:

The square error to be minimized:

Determining the filter coefficients )(kap :

Minimum error:

Determining the filter coefficients )(kbq :

THE SHANKS METHOD

Some useful MATLAB functions


2/12

2

SIGNAL MODELING

INTRODUCTION

Signal modeling:

Signal modeling is concerned with the representation of signals in an efficient manner. It has been widely

applied in speech and audio coding, imaging compression, data compression, and spectrum estimation.

Example 1.Consider a signal x(n) consisting of 2000 data values, x(0), x(1), , x(1999). The plot of the

signalx(n) is shown in Fig. 1. Now we want to model this signal. What should we do?

Fig. 1.(a) A given signal1

x(n) to be modeled.

(b) Difference between

the given and modeled

signals.

Step 1.From our knowledge, the signal in Fig. 1 is sinusoidal. Therefore, we choose a parametric modelof

sinusoid to model the signal. A sinusoid model may be of the form )sin()( 0 += nAnx with parameters,

A, 0 and .

Step 2. After having chosen a parametric model, we shall determine the parameters,A, 0 and , for the

model. The parameters should be found based on a certain criterion (e.g., least squares method) so that the

parameters provides the best approximation to the given signal. In the present case, we may use the direct

observation method to estimate the parameters. From the figure we can estimate that A=2,

57.12/ == , and 0314.0200/20 == , which makes )57.10314.0sin(2)( = nnx . With the model

parameters found, the model is established.

Step 3.After having established the model, we should evaluate the model and see how accurate the model

will be, by looking into the error (difference) between the given signal and the model signal (see Fig. 1(b)).

Now consider to send the given signal x(n) in Fig. 1(a) across a communication channel with two

schemes: (1) directly send these 2000 signal values; or (2) only send four parameter values: the amplitude A,

the phase , the frequency 0 and data length N=2000, and then reconstruct the signal using the model

established. Obviously, the latter scheme of sending the signal is much more efficient than the formal one.

With the established model, we can easily extrapolate the signal beyond n= 01999, e.g.,x(2000),x(2001),

, and interpolate if some values (e.g.,x(1400), ,x(1500)) are missing or severely distorted.

1Fig. 1(a) originates from ( )2/799.199/2sin2)( = nnx


3/12

3

The central issues in signal modeling:

From the above discussion, we may summarize the following central issues in Signal Modeling:

(1) Choosing an appropriate parametric modelfor modeling a given signal, deterministic or random;

(2) Determining the model parametersthat provides the bestapproximation to the given signal;

(3) Evaluating the model performanceby looking into an error between the model and the given signal.

Modeling a signal as the output of a linear shift-invariant filter:

Fig. 2.Modeling a signalx(n) as the response )( nx of an LSI

filter to an input v(n).

The model that is chosen and will be studied in this course is one that models a signal as the output of a

causal linear shift-invariant filterthat has a rational system function of the form

=

=

+

==p

k

kp

q

k

kq

p

q

zka

zkb

zA

zBzH

1

0

)(1

)(

)(

)()( (1)

This is shown in Fig. 2. A causal LSI filter means that h(n)=0 for n


4/12

4

ways to define the best approximation, and using different definitions, we will have different solutions to

the modeling problem along with different techniques for finding the model parameters.

The techniques for finding )(kap and )(kbq for deterministic signals are the least squares method, the

Pad approximation, the Prony's method and the Shank's method; and the techniques for finding )(kap and

)(kbq for random signals are modified Yule-Walker equations and extend Yule-Walker equations. Each

method is established based on a certain best approximation, or a certain error function.

We start with developing the signal modeling techniques for deterministic signals, and then extend the

techniques to the case of random signals.

Optimization Theory:

To find model parameters for the best approximation is equivalent to find model parameters for the minimal

error between the given signalx(n) and the signal model )( nx , i.e., e(n)=x(n) )( nx . To minimize the error,

we shall employ optimization theory.

Theorem.If ),( *zzf is a real-valued function of zandz* and if ),( *zzf is analytic with respect to bothz

andz*, then the stationary points of ),( *zzf may be found by setting 0),( * = zzzf or 0),( ** = zzzf

and solving forz.

At the stationary pointsf(x) are local and global minima!

Example. The stationary point and the minima

Consider that 2|)(|)( zezf = is a real-valued function of a complex number z, where e(z)=x(n)z+ b(n)is a

linear function of z and x(n) and b(n) are complex. The function2|)(|)( zezf = can be written as

)(*)(*),( zezezzf = . The stationary points off(z) can be in two ways.

(i) If )(*)(*),( zezezzf = is treated as a function ofzwithz* being constant, then

[ ])(**)(*)()(*|)(| 2 nbznxzedz

dzeze

dz

d+== x(n) (5)

(ii) If )(*)(*),( zezezzf = is treated as a function ofz* withzbeing constant, then

[ ] )(*)()()(**

)(|)(|*

2 nxnbznxzedz

dzeze

dz

d+== (6)

Setting both derivatives in Eqs. (5) and (6) to be zero and solving for z, we will get the same solution,

)(/)( nxnbz = , at which (the stationary point)f(x) is minimum since 0|)(|)( 2= zezf . In this course, the

second method, i.e., setting ( ) [ ] )(*)()(*|)(|2

nxnbznxdzzed += =0, will be used since it is more convenient

due to the autocorrelation of )(nx defined as )}(*)({)( knxnxEkrx = .

PART I. DETERMINISTIC SIGNAL MODELING

The signal model:

The problem to be considered is to model a deterministic signal,x(n), as the unit sample response of a LSI

filter with a system function in Eq. (2), namely, )(nx =h(n). Thus, the input v(n) is the unit sample )(n . It is

assumed that x(n)=0 for n< 0 and that the filter h(n) is causal. There are different methods available for

finding the filter's coefficients )(kap and )(kbq when different error functions are used to get the best

approximation ofx(n) from )(nx .


5/12

5

THE LEAST SQUARES (DIRECT) METHOD

The modeling error:

In this method, the modeling error

)()()(' nhnxne = (7)

is used to find the filter coefficients, )(kap and )(kbq , that give best approximation ofx(n).

The error function to be minimized:

The error measure that is to be minimized is the squared error

LS =

=0

2)('

n

ne (8)

Note that e'(n) = 0 for n< 0 sincex(n) and h(n) are both assumed to be zero for n< 0 (i.e., causal).


To find the filter coefficients )(kap and )(kbq that give the minimized squared error, we apply the

optimization theory, setting

=

)(* kap

LS 0 ; k= 1, 2, ,p (9)

=

)(* kbq

LS 0 ; k= 0, 1, , q (10)

Using Parseval's theorem, the least square error may be written in the frequency domain as follows

LS =

deE j 2|)('|2

1 (11)

where )()()(' jjj eHeXeE = is the Fourier transform of )()()(' nhnxne = .

Substituting Eq. (11) into Eq. (9), we have

[ ]

=

=

d

ka

eEeEdeEeE

kaka p

jjjj

pp

LS

)(

)(')('

2

1)(')('

)(2

1

)( *

**

**

[ ]

=

deeA

eB

eA

eB

eX

jk

jq

jq

jp

jqj

2*

*

)(

)(

)(

)(

)(2

1

=0, k= 1, 2, ,p (12)

Similarly, we have

=

de

eAeA

eBeX

kb

jk

jq

jp

jqj

q

LS

)(

1

)(

)()(

2

1

)(**

=0, k= 0, 1, , q (13)

In Eq. (12) there arepequations, and in Eq. (13) there are q+ 1 equations. Thus, the optimum set of model

parameters, )(kap and )(kbq , are defined implicitly in terms of a set of p+q+1 equations that are nonlinear

because )(zBq / )(zAp and )(zAp )(zBq create nonlinear equations of )(kap and )(kbq , where

=

+=

p

k

k

pp zkazA1

)(1)( and =

=

q

k

k

qq zkbzB0

)()( . Although iterative techniques such as the method of

steepest descent, Newtons method, or iterative pre-filtering could be used to solve these equations, the least


6/12

6

squares approach is not mathematically tractable and not amenable to real-time signal processing

applications. It is for this reason that we shall find some indirect, simple methods of signal modeling. In

these methods, the modeling problem is changed slightly so that the model parameters may be found more

easily.

THE PAD APPROXIMATION

The model error:

The Pad method sets h(n)=x(n) for n= 0, 1, ,p+ q, and thus, the modeling error becomes

+=

==otherwiseunknown,

...,2,1,0,for,0)()()('

qpnnhnxne


Unlike the least squares solution, the Pad approximation only requires solving a set of linear equations to

find the coefficients )(kap and )(kbq . Specifically, the Pad method sets h(n)=x(n) for n= 0, 1, ,p+ qin

Eq. (3). This leads to the following set ofp+ q+1 linear equations inp+ q+1 unknowns,

++=

==+

= pqqn

qnnbknxkanx

qp

k

p...,,1,0

...,,1,0),()()()(

1

(14)

Note thatx(n)=0 for n


7/12

7

where

[ ]Tpppp paaa )(,),2(),1( =a , (19)

[ ]Tq pqxqxqx )(,),2(),1(1 +++=+ x , (20)

++

++

+

=

)(...)2()1(

............

)2(...)()1(

)1(...)1()(

qxpqxpqx

pqxqxqx

pqxqxqx

qX (21)

is a pp non-symmetric Toeplitz matrix.

Depending on whether or not the matrix qX is invertible, there are three cases of interest:

(i) qX is nonsingular. In this case, the coefficients )(kap are uniquely determined by Eq. (17).

(ii) qX is singular and a solution pa to Eq. (18) exists. The solution is not unique, i.e., zaa += pp~ is also

a solution.

(iii) qX is singular and no solution to Eq. (18) exists. In this case, we must set 0)0( =pa , and thus Eq. (18)

becomes

0=pq aX (22)

After obtaining the coefficients )(kap , the second step is to solve for the numerator coefficients )(kbq using

the first (q+1) equations in Eq. (15)

=

)(

...

)2(

)1(

)0(

)(

...

)2(

)1(

1

)(...)2()1()(

...............

0...)0()1()2(

0...0)0()1(

0...00)0(

qb

b

b

b

pa

a

a

pqxqxqxqx

xxx

xx

x

q

q

q

q

p

p

p

, (23)

or

qp baX =0 (24)

where

[ ]Tpppp paaa )(,),2(),1(,1 =a (25)

[ ]Tqqqq qbbb )(,),1(),0( =b . (26)Equivalently, coefficients )(kbq can be found using Eq. (14), i.e.,

= +=p

k

pq knxkanxnb1

)()()()( (27)

Note that the number of )(nx used isp+q+1 that is just the coefficients number, and that )0()0( xbq = .

Example 2. Pad Approximation (Example 4.3.2 on p. 139 in textbook)

Example 3. Filter Design Using Pad Approximation (Example 4.3.4 on p. 142 in textbook)


8/12

8

Special notes for Pad Approximation:

(i) The Pad Approximation is not an optimum method.

(ii) The model that is formed from the Pad Approximation will always produce an exact fit to the data over

the interval [0, p+q], i.e., x(0), x(1), , x(p+q), provided that qX is nonsingular. However, since the

data outside the interval [0,p+q] is never considered in the modeling process, there is no guarantee on

how accurate the model will be for n>p+q. In some cases, the model fits the data very well, while in

others it may fit poor.

(iii) The Pad Approximation will give the correct model parameters for those cases in which x(n) has a

rationalz-transform and the model order is chosen to be large enough.

(iv) Since the Pad Approximation forces the model to match the signal only over a limited range of values,

the model that is generated is not guaranteed to be stable.

(v) A given signal can be modeled using different LSI filters, e.g., pole-zero, all pole, and all zero models.

Denominator coefficients )(nap are found by

+

+

+

=

++

++

+

)(

...

)2(

)1(

)(

...

)2(

)1(

)(...)2()1(

............

)2(...)()1(

)1(...)1()(

pqx

qx

qx

pa

a

a

qxpqxpqx

pqxqxqx

pqxqxqx

p

p

p

Numerator coefficients )(nbq

are determined by

=

+=

p

k

pq knxkanxnb

1

)()()()(

Note that )0()0( xbq = .

THE PRONYS METHOD

Alternative modeling error:

In principle, the filter coefficients )(kap and )(kbq should be determined by minimizing the modeling error

)()()(' nhnxne =

, (28)which in the frequency domain is of the form

)(

)()()('

zA

zBzXzE

p

q= (29)

However, minimization of error e'(n) by setting )()('*

0

2kane p

n

=

=0 and )()('*

0

2kbne q

n

=

=0

usually yields nonlinear equations that are mathematically intractable like in the least squares method. Thus,

it loses its practical applicability.

If multiplying )(zAp on both sides of Eq. (29), then we have

)()()()(')()( zBzXzAzEzAzE qpp == (30)

which in the time domain may be written as


9/12

9

>+

+

=

=

=

qnknxkanx

qnnbknxkanx

nep

k

p

q

p

k

p

;)()()(

0);()()()(

)(

1

1 (31)

which is an alternative modeling error and is linear in the filter coefficients, )(kap and )(kbq . Thus,

minimization of the error e(n) will give linear solution to the coefficients desired.

The square error to be minimized:

In the Prony's method the error to be minimized for determining the coefficients )(kap is the square error

+= =

+=

+==

1

2

11

2, |)()()(||)(|

qn

p

k

p

qn

qp knxkanxne , (32)

which only depends on )(kap but not on )(kbq .

Determining the filter coefficients )(kap :

To minimize the square error, we may set

0)(

*

,=

kap

qp, that is

+=

+=

+=

=

=

=

1*

*

1

*

*1

2

**

,

)(

)()()()(

)(|)(|

)()( qn pqnpqnpp

qp

ka

nenenene

kane

kaka

+=

+= =

==

+

=

1

*

1 1

***

*0)()()()()(

)()(

qnqn

p

k

p

p

knxneknxkanxka

ne , k=1, 2, ,p (33)

In the above equation, we see

+=

=

1

* 0)()(

qn

knxne , k=1, 2, ,p (34)

which is known as the orthogonality principle, which follows from 0)()()(1

*==

+=qnex knxnekr . This

principle states that for the optimum linearpredictor the estimation error will be orthogonal to the data x. It is

fundamental in mean-square estimation problems. Inserting e(n) in Eq. (31) into Eq. (34), we have

0)()()()(1

*

1

=

+

+= =qn

p

lp knxlnxlanx (35)

or equivalently,

+==

+=

=

1

*

1

*

1

)()()()()(

qn

p

l qn

p knxnxknxlnxla (36)

by defining

+=

=

1

*)()(),(

qn

x knxlnxlkr (37)

which is a conjugate symmetry function, ),(),(* klrlkr

xx

= , Eq. (36) becomes

)0,(),()(1

krlkrla x

p

lxp =

=

, k=1, 2, ,p (38)


10/12

10

Eq. (38) is referred to as the Prony normal equationsthat are a set ofplinear equations with thepunknowns,

the coefficients )(kap . In the matrix form, the normal equations are

=

)0,(

...

)0,2(

)0,1(

)(

...

)2(

)1(

),(...)2,()1,(

............

),2(...)2,2()1,2(

),1(...)2,1()1,1(

pr

r

r

pa

a

a

pprprpr

prrr

prrr

x

x

x

p

p

p

xxx

xxx

xxx

(39)

or

xpx raR = (40)

where xR is a pp autocorrelation matrix with elements defined in Eq. (37),

=xR

),(...)2,()1,(

............

),2(...)2,2()1,2(

),1(...)2,1()1,1(

pprprpr

prrr

prrr

xxx

xxx

xxx

(41)

[ ]Tpppp paaa )(,),2(),1( =a , (42)

[ ]Txxxx prrr )0,(,),0,2(),0,1( =r , (43)

Note that ),( lkrx are the autocorrelation of the deterministic signal x(n) and obtained using an infiniteset of

datax(n) for n= q+1, , (see Eq. (37)).

The coefficients )(kap have been found by minimizing the error qp, in Eq. (32). Now we shall see the

minimum error.

Minimum error:From Eq. (32), it follows that

+= =

+=

+=

+===

1

*

11

*

1

2, )()()()()()(|)(|

qn

p

k

p

qnqn

qp knxkanxnenenene

=

+=

+=

+= =

+=

+=

+=

p

k qn

p

qn

qn

p

k

p

qn

knxnekanxne

knxkanenxne

1

*

1

*

1

*

1

*

11

*

)()()()()(

)()()()()(

Because of the orthogonal principle,

+=

=1

* 0)()(qn

knxne , for k=1, 2, ,p(but not for k=0 !), we have

qp,

+= =

+=

+==

1

*

11

* )()()()()()(

qn

p

k

p

qn

nxknxkanxnxne

+= =

+=

1

*

1

*)()()()()(

qn

p

k

p nxknxkanxnx (44)

Using the autocorrelation defined in Eq. (37), the minimum errormay be

=

+=

p

k

xpxqp krkar

1

, ),0()()0,0( (45)


11/12

11

Determining the filter coefficients )(kbq :

Once the coefficients )(kap have been found, the second step is to determine the q+1 numerator

coefficients. Using the same way as in the Pad method, the numerator coefficients )(nbq are found by

=+=

p

kpq knxkanxnb

1)()()()( , (46)

or by the matrix form

=

)(

...

)2(

)1(

)0(

)(

...

)2(

)1(

1

)(...)2()1()(

...............

0...)0()1()2(

0...0)0()1(

0...00)0(

qb

b

b

b

pa

a

a

pqxqxqxqx

xxx

xx

x

q

q

q

q

p

p

p

(47)

Example 4. Pronys Method (Example 4.4.1 on p. 149 in textbook)Example 5. Filter Design Using Pronys Method (Example 4.4.2 on p. 152 in textbook)

Special notes for Pronys Method:

(i) The Pronys method is an optimum method because the optimization theory is used to minimize the

squared modeling error. The orthogonal principle (Eq. (34)) is the result of the optimization, and it

makes the modeling error qp, be minimum.

(ii) The Pronys method uses an infinite set of signal values, i.e., x(n) for all 0n , to find the

autocorrelation ),( lkrx defined in Eq. (37).

(iii) The Pronys method is usually more accurate than the Pad Approximation.

Denominator coefficients )(nap are found by

=

)0,(

...

)0,2(

)0,1(

)(

...

)2(

)1(

),(...)2,()1,(

............

),2(...)2,2()1,2(

),1(...)2,1()1,1(

pr

r

r

pa

a

a

pprprpr

prrr

prrr

x

x

x

p

p

p

xxx

xxx

xxx

, k=1, 2, ,p

+=

=

1

*

)()(),(qn

x knxlnxlkr ;

Numerator coefficients )(nbq are determined by

=

+=

p

k

pq knxkanxnb

1

)()()()(

Note that )0()0( xbq = .

The minimum error is =

+=

p

k

xpxqp krkar

1

, ),0()()0,0(


12/12

12

The alternative form of the Pronys method

Representing

+++

++

+

=

...

)3(...)1()2(

)2(...)()1(

)1(...)1()(

pqxqxqx

pqxqxqx

pqxqxqx

qX (48)

and

[ ]Tq qxqxqx ),3(),2(),1(1 +++=+x (49)

we have

qHqx XXR = (50)

and

1r += qHqx xX (51)

The Prony normal equations in Eq. (40) become

1+= qHqpq

Hq xXaXX (52)

which the alternative form of the Prony normal equations.

THE SHANKS METHOD (refer to pp. 154-160 in the textbook)

In the Pronys method, )(kap are determined in an optimum way that the squares error

+=

=

1

2, |)(|

qn

qp ne =

+= =

+

1

2

1

|)()()(|qn

p

k

p knxkanx is minimuized, but )(kbq are not. In the Shanks method, )(kbq are found

in an optimum way that the squares error

=

=

==

0

2

0

2|)()(||)('|

nn

S nxnxne is minimized assuming that

)(kap is fixed.

After )(kap are determined with the Pronys method, using the Shanks method to find the optimum

coefficients )(kbq may lead to a further improvement of the models accuracy. Note that in this case both

)(kap and )(kbq are determined in optimum ways, but these optimum ways are different from the one in the

least squares method.

Some useful MATLAB functions

>> [B, A] = PRONY(H, NB, NA) % finds a filter with numerator order NB, denominator order NA,

% and having the impulse response in vector H.

Documents

pade and prony technique