8/13/2019 pade and prony technique
1/12
1
SIGNAL MODELING (1)
INTRODUCTION
Signal modeling:
The central issues in signal modeling:
Modeling a signal as the output of a linear shift-invariant filter:
Techniques for determining model parameters:
Optimization Theory:
PART I. DETERMINISTIC SIGNAL MODELINGThe signal model :
THE LEAST SQUARES (DIRECT) METHOD
The modeling error:
The error function to be minimized:
Determining the filter coefficients:
THE PAD APPROXIMATION
Modeling error:Determining the filter coefficients:
THE PRONYS METHOD
Alternative modeling error:
The square error to be minimized:
Determining the filter coefficients )(kap :
Minimum error:
Determining the filter coefficients )(kbq :
THE SHANKS METHOD
Some useful MATLAB functions
8/13/2019 pade and prony technique
2/12
2
SIGNAL MODELING
INTRODUCTION
Signal modeling:
Signal modeling is concerned with the representation of signals in an efficient manner. It has been widely
applied in speech and audio coding, imaging compression, data compression, and spectrum estimation.
Example 1.Consider a signal x(n) consisting of 2000 data values, x(0), x(1), , x(1999). The plot of the
signalx(n) is shown in Fig. 1. Now we want to model this signal. What should we do?
Fig. 1.(a) A given signal1
x(n) to be modeled.
(b) Difference between
the given and modeled
signals.
Step 1.From our knowledge, the signal in Fig. 1 is sinusoidal. Therefore, we choose a parametric modelof
sinusoid to model the signal. A sinusoid model may be of the form )sin()( 0 += nAnx with parameters,
A, 0 and .
Step 2. After having chosen a parametric model, we shall determine the parameters,A, 0 and , for the
model. The parameters should be found based on a certain criterion (e.g., least squares method) so that the
parameters provides the best approximation to the given signal. In the present case, we may use the direct
observation method to estimate the parameters. From the figure we can estimate that A=2,
57.12/ == , and 0314.0200/20 == , which makes )57.10314.0sin(2)( = nnx . With the model
parameters found, the model is established.
Step 3.After having established the model, we should evaluate the model and see how accurate the model
will be, by looking into the error (difference) between the given signal and the model signal (see Fig. 1(b)).
Now consider to send the given signal x(n) in Fig. 1(a) across a communication channel with two
schemes: (1) directly send these 2000 signal values; or (2) only send four parameter values: the amplitude A,
the phase , the frequency 0 and data length N=2000, and then reconstruct the signal using the model
established. Obviously, the latter scheme of sending the signal is much more efficient than the formal one.
With the established model, we can easily extrapolate the signal beyond n= 01999, e.g.,x(2000),x(2001),
, and interpolate if some values (e.g.,x(1400), ,x(1500)) are missing or severely distorted.
1Fig. 1(a) originates from ( )2/799.199/2sin2)( = nnx
8/13/2019 pade and prony technique
3/12
3
The central issues in signal modeling:
From the above discussion, we may summarize the following central issues in Signal Modeling:
(1) Choosing an appropriate parametric modelfor modeling a given signal, deterministic or random;
(2) Determining the model parametersthat provides the bestapproximation to the given signal;
(3) Evaluating the model performanceby looking into an error between the model and the given signal.
Modeling a signal as the output of a linear shift-invariant filter:
Fig. 2.Modeling a signalx(n) as the response )( nx of an LSI
filter to an input v(n).
The model that is chosen and will be studied in this course is one that models a signal as the output of a
causal linear shift-invariant filterthat has a rational system function of the form
=
=
+
==p
k
kp
q
k
kq
p
q
zka
zkb
zA
zBzH
1
0
)(1
)(
)(
)()( (1)
This is shown in Fig. 2. A causal LSI filter means that h(n)=0 for n
8/13/2019 pade and prony technique
4/12
4
ways to define the best approximation, and using different definitions, we will have different solutions to
the modeling problem along with different techniques for finding the model parameters.
The techniques for finding )(kap and )(kbq for deterministic signals are the least squares method, the
Pad approximation, the Prony's method and the Shank's method; and the techniques for finding )(kap and
)(kbq for random signals are modified Yule-Walker equations and extend Yule-Walker equations. Each
method is established based on a certain best approximation, or a certain error function.
We start with developing the signal modeling techniques for deterministic signals, and then extend the
techniques to the case of random signals.
Optimization Theory:
To find model parameters for the best approximation is equivalent to find model parameters for the minimal
error between the given signalx(n) and the signal model )( nx , i.e., e(n)=x(n) )( nx . To minimize the error,
we shall employ optimization theory.
Theorem.If ),( *zzf is a real-valued function of zandz* and if ),( *zzf is analytic with respect to bothz
andz*, then the stationary points of ),( *zzf may be found by setting 0),( * = zzzf or 0),( ** = zzzf
and solving forz.
At the stationary pointsf(x) are local and global minima!
Example. The stationary point and the minima
Consider that 2|)(|)( zezf = is a real-valued function of a complex number z, where e(z)=x(n)z+ b(n)is a
linear function of z and x(n) and b(n) are complex. The function2|)(|)( zezf = can be written as
)(*)(*),( zezezzf = . The stationary points off(z) can be in two ways.
(i) If )(*)(*),( zezezzf = is treated as a function ofzwithz* being constant, then
[ ])(**)(*)()(*|)(| 2 nbznxzedz
dzeze
dz
d+== x(n) (5)
(ii) If )(*)(*),( zezezzf = is treated as a function ofz* withzbeing constant, then
[ ] )(*)()()(**
)(|)(|*
2 nxnbznxzedz
dzeze
dz
d+== (6)
Setting both derivatives in Eqs. (5) and (6) to be zero and solving for z, we will get the same solution,
)(/)( nxnbz = , at which (the stationary point)f(x) is minimum since 0|)(|)( 2= zezf . In this course, the
second method, i.e., setting ( ) [ ] )(*)()(*|)(|2
nxnbznxdzzed += =0, will be used since it is more convenient
due to the autocorrelation of )(nx defined as )}(*)({)( knxnxEkrx = .
PART I. DETERMINISTIC SIGNAL MODELING
The signal model:
The problem to be considered is to model a deterministic signal,x(n), as the unit sample response of a LSI
filter with a system function in Eq. (2), namely, )(nx =h(n). Thus, the input v(n) is the unit sample )(n . It is
assumed that x(n)=0 for n< 0 and that the filter h(n) is causal. There are different methods available for
finding the filter's coefficients )(kap and )(kbq when different error functions are used to get the best
approximation ofx(n) from )(nx .
8/13/2019 pade and prony technique
5/12
5
THE LEAST SQUARES (DIRECT) METHOD
The modeling error:
In this method, the modeling error
)()()(' nhnxne = (7)
is used to find the filter coefficients, )(kap and )(kbq , that give best approximation ofx(n).
The error function to be minimized:
The error measure that is to be minimized is the squared error
LS =
=0
2)('
n
ne (8)
Note that e'(n) = 0 for n< 0 sincex(n) and h(n) are both assumed to be zero for n< 0 (i.e., causal).
Determining the filter coefficients:
To find the filter coefficients )(kap and )(kbq that give the minimized squared error, we apply the
optimization theory, setting
=
)(* kap
LS 0 ; k= 1, 2, ,p (9)
=
)(* kbq
LS 0 ; k= 0, 1, , q (10)
Using Parseval's theorem, the least square error may be written in the frequency domain as follows
LS =
deE j 2|)('|2
1 (11)
where )()()(' jjj eHeXeE = is the Fourier transform of )()()(' nhnxne = .
Substituting Eq. (11) into Eq. (9), we have
[ ]
=
=
d
ka
eEeEdeEeE
kaka p
jjjj
pp
LS
)(
)(')('
2
1)(')('
)(2
1
)( *
**
**
[ ]
=
deeA
eB
eA
eB
eX
jk
jq
jq
jp
jqj
2*
*
)(
)(
)(
)(
)(2
1
=0, k= 1, 2, ,p (12)
Similarly, we have
=
de
eAeA
eBeX
kb
jk
jq
jp
jqj
q
LS
)(
1
)(
)()(
2
1
)(**
=0, k= 0, 1, , q (13)
In Eq. (12) there arepequations, and in Eq. (13) there are q+ 1 equations. Thus, the optimum set of model
parameters, )(kap and )(kbq , are defined implicitly in terms of a set of p+q+1 equations that are nonlinear
because )(zBq / )(zAp and )(zAp )(zBq create nonlinear equations of )(kap and )(kbq , where
=
+=
p
k
k
pp zkazA1
)(1)( and =
=
q
k
k
qq zkbzB0
)()( . Although iterative techniques such as the method of
steepest descent, Newtons method, or iterative pre-filtering could be used to solve these equations, the least
8/13/2019 pade and prony technique
6/12
6
squares approach is not mathematically tractable and not amenable to real-time signal processing
applications. It is for this reason that we shall find some indirect, simple methods of signal modeling. In
these methods, the modeling problem is changed slightly so that the model parameters may be found more
easily.
THE PAD APPROXIMATION
The model error:
The Pad method sets h(n)=x(n) for n= 0, 1, ,p+ q, and thus, the modeling error becomes
+=
==otherwiseunknown,
...,2,1,0,for,0)()()('
qpnnhnxne
Determining the filter coefficients:
Unlike the least squares solution, the Pad approximation only requires solving a set of linear equations to
find the coefficients )(kap and )(kbq . Specifically, the Pad method sets h(n)=x(n) for n= 0, 1, ,p+ qin
Eq. (3). This leads to the following set ofp+ q+1 linear equations inp+ q+1 unknowns,
++=
==+
= pqqn
qnnbknxkanx
qp
k
p...,,1,0
...,,1,0),()()()(
1
(14)
Note thatx(n)=0 for n
8/13/2019 pade and prony technique
7/12
7
where
[ ]Tpppp paaa )(,),2(),1( =a , (19)
[ ]Tq pqxqxqx )(,),2(),1(1 +++=+ x , (20)
++
++
+
=
)(...)2()1(
............
)2(...)()1(
)1(...)1()(
qxpqxpqx
pqxqxqx
pqxqxqx
qX (21)
is a pp non-symmetric Toeplitz matrix.
Depending on whether or not the matrix qX is invertible, there are three cases of interest:
(i) qX is nonsingular. In this case, the coefficients )(kap are uniquely determined by Eq. (17).
(ii) qX is singular and a solution pa to Eq. (18) exists. The solution is not unique, i.e., zaa += pp~ is also
a solution.
(iii) qX is singular and no solution to Eq. (18) exists. In this case, we must set 0)0( =pa , and thus Eq. (18)
becomes
0=pq aX (22)
After obtaining the coefficients )(kap , the second step is to solve for the numerator coefficients )(kbq using
the first (q+1) equations in Eq. (15)
=
)(
...
)2(
)1(
)0(
)(
...
)2(
)1(
1
)(...)2()1()(
...............
0...)0()1()2(
0...0)0()1(
0...00)0(
qb
b
b
b
pa
a
a
pqxqxqxqx
xxx
xx
x
q
q
q
q
p
p
p
, (23)
or
qp baX =0 (24)
where
[ ]Tpppp paaa )(,),2(),1(,1 =a (25)
[ ]Tqqqq qbbb )(,),1(),0( =b . (26)Equivalently, coefficients )(kbq can be found using Eq. (14), i.e.,
= +=p
k
pq knxkanxnb1
)()()()( (27)
Note that the number of )(nx used isp+q+1 that is just the coefficients number, and that )0()0( xbq = .
Example 2. Pad Approximation (Example 4.3.2 on p. 139 in textbook)
Example 3. Filter Design Using Pad Approximation (Example 4.3.4 on p. 142 in textbook)
8/13/2019 pade and prony technique
8/12
8
Special notes for Pad Approximation:
(i) The Pad Approximation is not an optimum method.
(ii) The model that is formed from the Pad Approximation will always produce an exact fit to the data over
the interval [0, p+q], i.e., x(0), x(1), , x(p+q), provided that qX is nonsingular. However, since the
data outside the interval [0,p+q] is never considered in the modeling process, there is no guarantee on
how accurate the model will be for n>p+q. In some cases, the model fits the data very well, while in
others it may fit poor.
(iii) The Pad Approximation will give the correct model parameters for those cases in which x(n) has a
rationalz-transform and the model order is chosen to be large enough.
(iv) Since the Pad Approximation forces the model to match the signal only over a limited range of values,
the model that is generated is not guaranteed to be stable.
(v) A given signal can be modeled using different LSI filters, e.g., pole-zero, all pole, and all zero models.
Denominator coefficients )(nap are found by
+
+
+
=
++
++
+
)(
...
)2(
)1(
)(
...
)2(
)1(
)(...)2()1(
............
)2(...)()1(
)1(...)1()(
pqx
qx
qx
pa
a
a
qxpqxpqx
pqxqxqx
pqxqxqx
p
p
p
Numerator coefficients )(nbq
are determined by
=
+=
p
k
pq knxkanxnb
1
)()()()(
Note that )0()0( xbq = .
THE PRONYS METHOD
Alternative modeling error:
In principle, the filter coefficients )(kap and )(kbq should be determined by minimizing the modeling error
)()()(' nhnxne =
, (28)which in the frequency domain is of the form
)(
)()()('
zA
zBzXzE
p
q= (29)
However, minimization of error e'(n) by setting )()('*
0
2kane p
n
=
=0 and )()('*
0
2kbne q
n
=
=0
usually yields nonlinear equations that are mathematically intractable like in the least squares method. Thus,
it loses its practical applicability.
If multiplying )(zAp on both sides of Eq. (29), then we have
)()()()(')()( zBzXzAzEzAzE qpp == (30)
which in the time domain may be written as
8/13/2019 pade and prony technique
9/12
9
>+
+
=
=
=
qnknxkanx
qnnbknxkanx
nep
k
p
q
p
k
p
;)()()(
0);()()()(
)(
1
1 (31)
which is an alternative modeling error and is linear in the filter coefficients, )(kap and )(kbq . Thus,
minimization of the error e(n) will give linear solution to the coefficients desired.
The square error to be minimized:
In the Prony's method the error to be minimized for determining the coefficients )(kap is the square error
+= =
+=
+==
1
2
11
2, |)()()(||)(|
qn
p
k
p
qn
qp knxkanxne , (32)
which only depends on )(kap but not on )(kbq .
Determining the filter coefficients )(kap :
To minimize the square error, we may set
0)(
*
,=
kap
qp, that is
+=
+=
+=
=
=
=
1*
*
1
*
*1
2
**
,
)(
)()()()(
)(|)(|
)()( qn pqnpqnpp
qp
ka
nenenene
kane
kaka
+=
+= =
==
+
=
1
*
1 1
***
*0)()()()()(
)()(
qnqn
p
k
p
p
knxneknxkanxka
ne , k=1, 2, ,p (33)
In the above equation, we see
+=
=
1
* 0)()(
qn
knxne , k=1, 2, ,p (34)
which is known as the orthogonality principle, which follows from 0)()()(1
*==
+=qnex knxnekr . This
principle states that for the optimum linearpredictor the estimation error will be orthogonal to the data x. It is
fundamental in mean-square estimation problems. Inserting e(n) in Eq. (31) into Eq. (34), we have
0)()()()(1
*
1
=
+
+= =qn
p
lp knxlnxlanx (35)
or equivalently,
+==
+=
=
1
*
1
*
1
)()()()()(
qn
p
l qn
p knxnxknxlnxla (36)
by defining
+=
=
1
*)()(),(
qn
x knxlnxlkr (37)
which is a conjugate symmetry function, ),(),(* klrlkr
xx
= , Eq. (36) becomes
)0,(),()(1
krlkrla x
p
lxp =
=
, k=1, 2, ,p (38)
8/13/2019 pade and prony technique
10/12
10
Eq. (38) is referred to as the Prony normal equationsthat are a set ofplinear equations with thepunknowns,
the coefficients )(kap . In the matrix form, the normal equations are
=
)0,(
...
)0,2(
)0,1(
)(
...
)2(
)1(
),(...)2,()1,(
............
),2(...)2,2()1,2(
),1(...)2,1()1,1(
pr
r
r
pa
a
a
pprprpr
prrr
prrr
x
x
x
p
p
p
xxx
xxx
xxx
(39)
or
xpx raR = (40)
where xR is a pp autocorrelation matrix with elements defined in Eq. (37),
=xR
),(...)2,()1,(
............
),2(...)2,2()1,2(
),1(...)2,1()1,1(
pprprpr
prrr
prrr
xxx
xxx
xxx
(41)
[ ]Tpppp paaa )(,),2(),1( =a , (42)
[ ]Txxxx prrr )0,(,),0,2(),0,1( =r , (43)
Note that ),( lkrx are the autocorrelation of the deterministic signal x(n) and obtained using an infiniteset of
datax(n) for n= q+1, , (see Eq. (37)).
The coefficients )(kap have been found by minimizing the error qp, in Eq. (32). Now we shall see the
minimum error.
Minimum error:From Eq. (32), it follows that
+= =
+=
+=
+===
1
*
11
*
1
2, )()()()()()(|)(|
qn
p
k
p
qnqn
qp knxkanxnenenene
=
+=
+=
+= =
+=
+=
+=
p
k qn
p
qn
qn
p
k
p
qn
knxnekanxne
knxkanenxne
1
*
1
*
1
*
1
*
11
*
)()()()()(
)()()()()(
Because of the orthogonal principle,
+=
=1
* 0)()(qn
knxne , for k=1, 2, ,p(but not for k=0 !), we have
qp,
+= =
+=
+==
1
*
11
* )()()()()()(
qn
p
k
p
qn
nxknxkanxnxne
+= =
+=
1
*
1
*)()()()()(
qn
p
k
p nxknxkanxnx (44)
Using the autocorrelation defined in Eq. (37), the minimum errormay be
=
+=
p
k
xpxqp krkar
1
, ),0()()0,0( (45)
8/13/2019 pade and prony technique
11/12
11
Determining the filter coefficients )(kbq :
Once the coefficients )(kap have been found, the second step is to determine the q+1 numerator
coefficients. Using the same way as in the Pad method, the numerator coefficients )(nbq are found by
=+=
p
kpq knxkanxnb
1)()()()( , (46)
or by the matrix form
=
)(
...
)2(
)1(
)0(
)(
...
)2(
)1(
1
)(...)2()1()(
...............
0...)0()1()2(
0...0)0()1(
0...00)0(
qb
b
b
b
pa
a
a
pqxqxqxqx
xxx
xx
x
q
q
q
q
p
p
p
(47)
Example 4. Pronys Method (Example 4.4.1 on p. 149 in textbook)Example 5. Filter Design Using Pronys Method (Example 4.4.2 on p. 152 in textbook)
Special notes for Pronys Method:
(i) The Pronys method is an optimum method because the optimization theory is used to minimize the
squared modeling error. The orthogonal principle (Eq. (34)) is the result of the optimization, and it
makes the modeling error qp, be minimum.
(ii) The Pronys method uses an infinite set of signal values, i.e., x(n) for all 0n , to find the
autocorrelation ),( lkrx defined in Eq. (37).
(iii) The Pronys method is usually more accurate than the Pad Approximation.
Denominator coefficients )(nap are found by
=
)0,(
...
)0,2(
)0,1(
)(
...
)2(
)1(
),(...)2,()1,(
............
),2(...)2,2()1,2(
),1(...)2,1()1,1(
pr
r
r
pa
a
a
pprprpr
prrr
prrr
x
x
x
p
p
p
xxx
xxx
xxx
, k=1, 2, ,p
+=
=
1
*
)()(),(qn
x knxlnxlkr ;
Numerator coefficients )(nbq are determined by
=
+=
p
k
pq knxkanxnb
1
)()()()(
Note that )0()0( xbq = .
The minimum error is =
+=
p
k
xpxqp krkar
1
, ),0()()0,0(
8/13/2019 pade and prony technique
12/12
12
The alternative form of the Pronys method
Representing
+++
++
+
=
...
)3(...)1()2(
)2(...)()1(
)1(...)1()(
pqxqxqx
pqxqxqx
pqxqxqx
qX (48)
and
[ ]Tq qxqxqx ),3(),2(),1(1 +++=+x (49)
we have
qHqx XXR = (50)
and
1r += qHqx xX (51)
The Prony normal equations in Eq. (40) become
1+= qHqpq
Hq xXaXX (52)
which the alternative form of the Prony normal equations.
THE SHANKS METHOD (refer to pp. 154-160 in the textbook)
In the Pronys method, )(kap are determined in an optimum way that the squares error
+=
=
1
2, |)(|
qn
qp ne =
+= =
+
1
2
1
|)()()(|qn
p
k
p knxkanx is minimuized, but )(kbq are not. In the Shanks method, )(kbq are found
in an optimum way that the squares error
=
=
==
0
2
0
2|)()(||)('|
nn
S nxnxne is minimized assuming that
)(kap is fixed.
After )(kap are determined with the Pronys method, using the Shanks method to find the optimum
coefficients )(kbq may lead to a further improvement of the models accuracy. Note that in this case both
)(kap and )(kbq are determined in optimum ways, but these optimum ways are different from the one in the
least squares method.
Some useful MATLAB functions
>> [B, A] = PRONY(H, NB, NA) % finds a filter with numerator order NB, denominator order NA,
% and having the impulse response in vector H.