568 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. AC-12, NO. 5, OCTOBER 1967

Monte Carlo Evaluation of Methods for Pulse Transfer Function Estimation

FRED 141. SMITH AND W. BRUCE HILTON

A&sfracf-This paper describes the results of a Monte Carlo evaluation made of the methods proposed in current literature for the estimation of the pulse transfer function of a linear, time-invariant dynamic system with feedback. Considered are two basic methods for estimating the coefficients of a pulse transfer function, given only the normal operating input and output of the system obscured by noise and over a limited period of time. The most commonly pro- posed method is a linear method in which a set of simultaneous linear equations is formed from the sampled data and the coe5cients obtained by a matrix inversion. The other method is an eigenvector method proposed by Levin which uses the eigenvector associated with the smallest eigenvalue of a matrix formed from the sampled data.

This paper presents a set of examples designed to compare linear and eigenvector estimation methods and to verify experimentally the theoretical results and approximations given by Levin. The com- parison shows that the eigenvector method generally gives estimates with equal or smaller rms error dVariance+(Bias)2 than the linear method. The eigenvector estimates had bias magnitudes which were consistently less than their standard deviations; the linear esti- mates did not, and thus had rms errors which often consisted largely of the bias. The approximate covariance matrix given by Levin for the coefficients estimated with the eigenvector method is found to be reasonably accurate.

M IKTRODUCTION

ASJ- METHODS for estimating the pulse transfer function of a dynamic system have been suggested in the literature. However, in

the presentation of these methods little or no a t tempt has been made to evaluate the performance of these methods when used u-ith noisy data, or to compare the performance of the method presented with other previ- ously described methods. T o provide this evaluation and comparison the results of an experimental study are de- scribed n-hich n-as undertaken to compare methods for estimating the pulse transfer function of a linear, time- invariant dynamic system with feedback, given only the normal operating input and output of the system ob- scured by noise and over a limited period of time.[l]

The comparison was limited to two basic estimation methods since all methods were found to be variations of these txvo. hIost of the proposed methods used varia- tions of a linear method in which a set of simultaneous equations is formed from the sampled data and the co-

January 13, 1967, and May 31,1967. This work was supported by the Manuscript received Januay 18, 1966; revised July 20, 1966,

US Army Electronic \Varfare Lab., US Army Electronics Com- mand, under Contract D.4 28-043 A4MC-00379(E).

tronic Systems, Inc., Mountain View, Calif. The authors are with the Electronic Defense Labs., S>-lvania Elec-

efficients obtained by a simple matrix inversion.I21-[61 (Several of these references are concerned with esti- mating the Laplace transform of the dynamic system. However, the estimation method is the same for either transfer function since the type of transfer function estimated depends only on the quantities measured, i.e., continuous functions or sampled-data sequences.) One exception is a recent paper by Levin,[’] which presents an eigenvector method using the eigenvector associated \\-ith the smallest eigenvalue of a matrix formed from the sampled data. There also are other estimation methods v-hich, for white noise and noise-free inputs, iteratively compute estimates that are equivalent to those of the eigenvector method in that they minimize the same ob- jective function.f81

The linear method is computationally much simpler than the eigenvector method. However, it is a heuristic method which has no statistical justification, and it is known to give biased estimates.[*] The eigenvector method is based on the statistical estimation theory de- veloped by Ko~pmans [~ l and is claimed by Levin to be, for all practical purposes, unbiased. The eigenvector method is equivalent to a generalized least-squares fitting of the input and output functions. Also, under certain conditions it gives a maximum likelihood esti- mate of the coefficients.

The comparison was made on a variety of typical problems by using the 3Ionte Carlo method to deter- mine the variance and bias of coefficients estimated using the different estimation methods. During the study the properties of the distributions of estimated coefficients were also determined for the eigenvector method.

MODEL OF ESTIMATIOS SITUATION The model assumed for the estimation of the pulse

transfer function is shown in Fig. 1. The sequences f ( n T ) and g(nT) are, respectively, the actual input and output of the dynamic system. The x(nT) and y ( n T ) are observed sequences which have been corrupted by the additive noise uf(nT) and u,(nT), respectively.

The dynamic system is assumed to be exactly repre- sented by the difference equation

agf(fn) + Ul(tn- - T ) + * . + U M j ( B - MT)

= bog(&) + bg(Q - T ) + - * . + b,vg(tn- - X T ) (1)

SMITH AND HILTOK: PULSE TFUNSFER FUNCTION ESTIMATION 569

x hT) Y (nT)

Fig. 1. Model for estimation of pulse transfer function.

where t k is defined to be at a sampling instant. Thus its pulse transfer function can be defined as

a0 + U l Z ' + . . . + u'vz--nf

bo + blz-1 + * * . + blvz-" H ( z ) =

Equation (1) in vector notation can be expressed

c T s k = 0

'where the superscript T indicates a transposed quantity, c is a vector of the coefficients, and s k is a vector of the input and output signal sequences at time fl;, i.e.,

CT = (uo, a1, * . a f a" bo, 61, . * . , 6 N ) , S k T = u ( t k ) , f ( fk - T ) , ' * * , f ( t k - &fT),

- g(!!k), - g ( f k - T ) , ' . ' 9 - g ( t k - L17T)).

The noise which is added to the elements of SI, is

C k T = ( 2 t , ( f k ) , U l ( f k - TI, . . - 1 % ( f k - -VlT), - u g ( t k ) ,

-au,(tli - T ) , . . . , U e ( 4 - AT)). 'Thus the vector of observed signals is

x k = s k $- L'k.

The noise is assumed to have zero mean with covariance matrix 2.

For both the linear and the eigenvector methods, the order of the numerator and the denominator poly- nomials, X a n d X , must be known. They will be assumed to be known here. Generally they are determined in practice by making several different estimates of differ- ent orders and choosing the minimum order which gives a suitable fit to the data.[6]

ESTIMATION METHODS

Before proceeding with the results of the experi- mental evaluation the linear and eigenvector estimation methods will be summarized. The derivations will not be given since they are available elsewhere.

The linear estimation method is concisely derived in Appendix I of Steiglitz and McBride.[sl This method at tempts to find a set of coefficients which satisfy the difference equation (1) \\-hen the observed signals X k are substituted for the actual signals SA:. Generally, because of observation noise the equation

r;;iTC = 0

cannot be satisfied for all k . Therefore, the error

e ( t d = X k T C

is defined and the C chosen to minimize the sum of squared errors for all the X k vectors. The estimate which gives this minimum is

K

1' [ x k d t k ) ]

where xk' is the vector x k with the Y ( t k ) element re- moved, C' is the vector C with the bo element removed and all other elements divided by bo, and K is the total number of vectors. The inverse will exist with probabil- ity one, provided that K is at least as large as the number of coefficients in C', that all the modes of the system are excited by the input or the initial conditions, and that the input sequence f (nT) is not the solution of a homogeneous difference equation of order less than M . r 1 1

The linear estimation method is not intended to be optimal in any sense €or this problem. I t tends to force the difference equation to be an equality when noise- contaminated signals are used, but the estimate gen- erally is not directly related to an optimal estimate for this problem. The method acknowledges the presence of noise but does not use the noise statistics to improve its estimate. The major appeal of the method is its sim- plicity and the fact that under certain conditions it may give as good an estimate as more sophisticated methods.

The eigenvector estimation method presented by LevinL71 is based on earlier work of K o o p m a n ~ . [ ~ ] -4 full derivation of the method was given previously.['] This method uses the same observed signal vectors x k as does the linear method. However, the coefficient vector C is chosen to minimize the quantity

with respect to the S k , where the s k are constrained to satisfy difference equation (1). Obtaining the co- efficients by minimizing X in (2) can be justified in three

1) With @ =Z-I the eigenvector method gives a maxi- mum likelihood estimate of the coefficients when the noise is Gaussian with zero mean and covariance 2 and uncorrelated from vector to vector. The noise on suc- cessive XI: vectors can be made uncorrelated by appro- priately spacing the vectors. The spacing is a minimum with white noise where nonoverlapping vectors are re- quired. \],:hen the noise is correlated from sample to sample, the spacing between successive vectors must be large enough to make the noise on successive vectors uncorrelated; thus, the spacing depends on the noise statistics. This method of forming vectors does not make complete use of the data. However, this restriction

570 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, OCTOBER 1967

is necessary to make the maximum likelihood estimate computationally feasible.

2) With @ = I the eigenvector method gives a set of coefficients relating estimates of the actual input and output sequences-the S k that least-squares fit the ob- served sequences. For the best least-squares fit, adjacent Xk vectors can be used.

3 ) iITith @=Z-l the eigenvector method in any case gives a generalized least-squares fit to the observed in- put and output sequences. This estimate can be justified geometrically when the noise has zero mean but is not Gaussian and/or when the noise on different Xk vectors is correlated, as would be the case if successive vectors were used.

The constrained minimization of X with respect to S , can be shown, using Lagrange multipliers, to be equivalent to the minimization of

1 = C T X k X k T C

K b 1 CT@C XL-C (3)

with respect to C. Minimization of the quadratic form of (3) is a generalized eigenvalue problem which can be solved by standard computational techniques.[l01 The X' is the smallest value of [ which satisfies the equation

where

The estimated coefficient vector C is the eigenvector corresponding to [=A'. The conditions on the input for a meaningful eigenvector estimate are the same as those for the linear estimation method.

The coefficients obtained from the eigenvector esti- mation method are generally biased. However, accord- ing t o Levin,['] Koopmans has shown[91 that if the noise variance is small compared to the mean-square values of f (nT) and g(nT), i.e., if the signal-to-noise ratio is high, then the bias is small compared to the standard deviation of the coefficients of C. Levin also states that it can be shown that the bias when overlapping vectors are used, is no more than when nonoverlapping vectors are used.

An approximation to the variance for the coefficients obtained from the eigenvector estimation method was derived by Koopmans without using the assumption of Gaussian noise. This approximation is given by Levin as

where C* is the true coefficient vector normalized so that bo= 1 , and where Sk ' is formed from S k in the same manner as X,' from X k . The approximation is valid for high signal-to-noise ratios.

EVALUATIOK OF ESTIMATION METHODS

T o compare the relative performance of the linear and eigenvector estimation methods, bias and standard deviation of the estimated coefficients were determined experimentally for both methods by using them in a variety of examples of different sampling intervals, signal-to-noise ratios, and amounts of data. The RIonte Carlo method was used to determine the average per- formance of the estimation methods because the accu- racy of any one estimate depends on the specific, but random, values of the noise in the observed time se- quences used. The performance was determined only for n-hite noise because it was felt that with correlated noise in a practical problem the accuracy of the estimate will depend primarily on the accuracy with which the noise statistics can be estimated, which is a problem beyond the scope of this paper. In each of the examples the standard deviation of the noise is the same for both input and output and was chosen so that the average of the input and output signal-to-noise power ratios S/N is set to a specified value.

Performance for Different Samp1in.g Ra.tes

The first example was chosen to show the variation of the statistics with sampling interval, and to show the relative performance of the linear and eigenvector estimation methods. Four different pulse transfer func- tions were estimated. Each of these pulse transfer func- tions mas of the form

H ( z ) = u12-1 + u2z-2

1 + 612-1 + bzz-2

All four pulse transfer functions related input and out- put of a linear dynamic system sampled at four different rates, T=0.25, 0.5, 1.0, and 2.0 seconds. The Laplace transform of the system \vas

1

s2 + 0.5s + 1

In each case the system, initially at rest, was excited with a unit step function. The noise-free input and out- put for this system is shown in Fig. 2. The data duration used in this example was - T_<t 5 $8.0 seconds.

The sample standard deviations, bias, normalized sample bias, and rrns errors of the estimated coefficients are shown in Table I. The number of estimates averaged to form the statistics n-as 25. Also given in Table I are the true values of the pulse transfer function coefficients and the approximate standard deviations computed using (5). At all values of T the noise was Gaussian with S/:V= 100. The normalized sample bias is defined as

Sample mean-true value

Sample standard deviation

SMITH AND HILTOK: PCLSE TRAXSFER FUNCTION ESTIMATION 571

1.4

1 .2

1.0

”, 2 0.8 E 2 I E

c z I? 0.6 .A

0.4

0 . 2

0

/ / \

\ \

/- -I 0 2 4 6

TIME -- SECONDS

Fig. 2. h-oise-free signals for first example.

8

TABLE I SAblPLE STATISTICS FOR FIRST EXAMPLE, APPROXIMATE ST-4SDARD DEVMTIOXS IN PAREXTHESES

~ - Sa.m- P l W inter- val -

0.25

__

0.50

~

1 .oo

__

2 .oo

__

be bl a1

Linear Eigenvector Quantity

true value

bias

0.030 0.109 1.825 (0.219) standard deviation 0.020 0.564

normalized bias 0.186 0.309 rms error 0.111 1.91

a2

Linear Eigenvector Linear Eigenvector

0.030

0.102 -0.766 0.826 -0.312

0.125 2.133 (0.247)

0.162 2.23

Linear Eigenvector

0.882

-0.945 0.384

0.961 1.273

0.152 1.191 (0.500)

-6.232 0.322

~~

-1.824 0.172 1.457 (0.548)

6.088 -0.323 1.048 -0.471

1.060 1.532

0.151 0.312 (0.247) 0.104

0.189 0.118 1.254 0.378 0.242 0.334

0.187 0.175 (0.371) 0.541 0.054 2.896 0.310 0.573 0.183

- 1 5 6 2 0.154 0.103 (0.310)

0.779

-0.413 -0.010 -2.682 -0.096

0.441 0.104

true value

bias 0.108 0.221 (0.195) standard deviation

0.113

0.115 0.231 rms error -0.367 -0.311 normalized bias -0.040 -0.069

0.169 0.223 (0.239) 0.331

0.074 0.053 0.205 0.252 (0.249) 0.148 0.053

-0.883 0.606 0.138 0.148 (0.170)

-0.085 -0.006

true value 0.116 0.137 (0.152) standard deviation

0.393

-0.005 -0.004 bias normalized bias -0.046 -0.032 rms error I 0.116 0.137

0.437 0.235 0.184 0.229

0.720 0.211 0.253 0.258

-0.620 -0.039 0.163 0.148

true value standard deviation 0.173 * (0.167) bias 0.015 normalized bias 0.089 I

1.071 0.578 (0.608)

0.73:

-0.094 -0.162

0.586

-0.43: 0.485 (0.472) I 0.159 * (0.170)

0.368

-0.026 ,-0.053 -0.054

0.486 1-0.333 I 0.168 rms error I 0.180

* Eigenvector estimate identical to linear estimate because only four vectors are used in A to estimate four coefficients.

572 IEEE TRAIWSACTIONS ON AUTOMATIC CONTROL, OCTOBER 1967

From Table I the following observations can be made. 1) The sample standard deviations of the eigenvector

estimates vary considerably Al-ith T. They are minimum between T=O.S and 1.0 second, rise slowly from the minimum for larger T, and rise very rapidly at smaller T ; at T=0.25 second they are almost 10 times the minimum values. The location of this minimum and the standard deviations for T at and above the minimum are as predicted by the approximate standard deviations given by ( 5 ) . However, at 0.25 second the sample stan- dard deviations are 2 to 8 times larger than the approxi- mate standard deviations.

2 ) The sample standard deviations of the linear esti- mates are approximately the same as those of the eigen- vector estimates for Tat and above the minimum of the eigenvector estimates. However, for T below the mini- mum the standard deviation of the linear estimates continues to decrease.

3 ) The bias of the eigenvector estimates for all T is considerably less than the standard deviation, the largest normalized bias having magnitude 0.378. Since the standard deviation of the normalized bias is roughly 1/d/25 =0.2, it can be concluded that the bias of the eigenvector estimates is significantly less than their standard deviations.

4) In contrast, the bias of the denominator coeffi- cients obtained with the linear estimates increases n7ith decreasing T and, for T20.5 second, is two or more standard deviations in magnitude. Assuming that the estimates are Gaussianly distributed, there is a one percent probability of observing a normalized bias equal to or greater than 2.70/~'/2j=0.54. Thus, for T < 1.0 second i t can be concluded with high probability that the bias of the linear estimates is two standard devia- tions or more. At T = 1.0 second the estimated denomi- nator coefficients are also significantly biased, but less than one standard deviation.

5 ) The bias magnitudes of the linear estimates for a particular T have much more variation with coefficient than those of the eigenvector estimates. Because of this, for each T in the linear method the coefficient with the largest bias magnitude is more biased than the most biased coefficient for the eigenvector estimate, and also, for each T the least biased coefficient from the linear estimates is never significantly more biased than that of the eigenvector estimates.

6) The rms error of the denominator coefficients of the linear estimates has the same variation with T as the standard deviation of the eigenvector estimates, and thus the same as the rnms error of the eigenvector esti- mates because they are not significantly biased. S o over- all comparison of the relative rms errors for the denomi- nator coefficients of the two methods is possible. At T = 1.0 second they are essentially the same, a t 0.5 second the rms error for the eigenvector estimates is somen-hat smaller, and at 0.25 second i t is somewhat larger.

Performance with Diflerent Amounts of Datu

The second example was designed to show the relative performance of the two estimation methods as the amount of data used to make the estimate was varied. I t was varied in tu-o waqs. First, a s a more extreme test for the bias of the estimation methods, the signal-to- noise ratio was decreased and the amount of data was increased in a manner that kept the standard deviation approximately the same. Second, both overlapping and nonoverlapping vectors were used in order to answer the question, left unanswered in Levin's theoretical develop- ment, of the degree of improvement obtained when over- lapping vectors instead of nonoverlapping vectors are used.

In this example six pulse transfer functions were esti- mated. As before, they all correspond to the linear dy- namic system of (7) and were of the form of (6). How- ever, in this case the system, initially at rest, was excited with a square pulse of width one second. The noise-free input and output for this system are shown in Fig. 3. The data duration used was - T<t 5 +4.0 seconds. The correct pulse transfer functions for this example are the same as those of the corresponding sampling rate for the first exampIe.

The details of the different cases are given in Table 11. In all cases the noise n-as Gaussian. The number of esti- mates averaged to obtain the sample statistics was 43. The effects of compensating for Ion- signal-to-noise ratios by using more data can be seen by comparing case A with B, C with D, and E with F. If the analytic approxi- mation of ( 5 ) to the covariance matrix is accurate, the sample variances should be approximately the same for each pair. The 25 times as much data for cases A, C, and E were generated for each estimate by using 25 dif- ferent noisy observations of the same noise-free input- output sequence. The effects of overlapping the data vectors can be seen by comparing cases A with E and B with F.

The sample standard deviations, normalized bias, and rms errors for each of these cases are given in Table I11 along with the true values of the estimated coefficients. From Table I11 the following observations can be made.

1) For the cases with overlap the conclusions of the first example regarding the standard deviation and bias are supported, including the bias magnitudes even though they are not shown in Table 111. The only varia- tion is that the minimum in the standard deviations of the eigenvector estimates sometimes occurs at lorn-er T since the standard deviations are not always larger at T = 0.25 second. In each case with overlap, the rms errors of the eigenvector estimates of the denominator coefficients are consistently smaller than those of the linear estimates.

2 ) In the cases with no overlap the rms error of the linear estimates is a factor of 2 4 larger than with the overlap. With the same conditions the eigenvector esti-

SMITH AIiX HILTON: PULSE TRANSFER FUNCTION ESTIMATION 573

Fig. 3. Noise-free signals for second example.

TABLE I1 DETAILS OF SECOSD EXAMPLE

Case T S / M sequences vectors lapping Number of Number of Over-

?l 0.25 4 25 400 B 0.25 100 1

yes

C 16

1 .o 4 25 100 yes

D 1 .o 100 1 4 yes

E 0.25 4 150 no yes

25 F 0.25 100 1 6 11 0

TA4BLE I11 SAUPLE STATISTICS FOR SECOND EXAMPLE

~.

a1 a2 bl Quantity

b2

Linear Eigenvector Linear Eigenvector Linear Eigenvector I Linear Eigenvector

true value standard deviation normalized bias rms error

true value standard deviation normalized bias rms error

true value standard deviation normalized bias rms error

n .o30 0.030- 0.123

-0.311 0.123 0.031 0.124

0.030 0.055 0.081

-0.128 0 .OS6

0.196 0.083

0.393 0.049 0.075

-0.802 0.236 0.063 0.077

0.032 3.801 -0.166

0.146

0.126 0.148

0.029

0.060 0.091 0.029

0.813 -0.154 0.077 0.092

0.331 0.055 0.177

0.112 1 .;71 -0.141

0.179

- 1.824 0.040 0.348

32.917 -0.184 1.32 0.354

0.148 0.152 2.882 -0,140

- 1.824

0.928 0.154

0.081 0.303

0.443 5.316 -0.098

0.306

-0.883

0.883 0.042 0.333

-28.560 0.186 1.20 0.339

0.883 0.143 0.143

0.438 0.139 0.145

-2.912

0.607 0.088 0.297

-4.705 0.423

0.098 0.300

true value 0.393 standard deviation 0.071 * normalized bias 0.027 rms error 0.071

0.117 * 0.331

~ -0.312 0.123

-0.883 I 0.199 0.191 * 0.607

-0.395 0.428 I 0.214 0.208

true value 0.030 0.029 -1.824 0.883 standard deviation 0.043 0.705 0.056 0.078 7.493 0.081 7.438 normalized bias 0.711 -0.091 2.372 0.107 16.756 ?.?OS -114.490 -0.203 rms error 0.053 0.733 I 0 .1M 2.272 I .64 1.173 7.58

F true value i 0.030 0.029 -1.824 ! 0.883 standard deviation 0.108 0.153 0.249 0 546 0.487 0.776

0:170 -:::;: 1 normalized bias i 0.001 0.033 I 0.265 -0.080 -0.210 0.379 rms error 0.108 0.153 I 0.155 0.250 0.556 0.924 0.499 0.827

I

~ ~

* Eigenvector estimate identical to linear estimate.

574 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, OCTOBER 1967

1 -

I

TRUE VALUE + TRUE VALUE

\\ I I I I I I

-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

C O E F F I C I E N T

Fig. 4. Histogram for Gaussian measurement noise.

-

- -

b\ - - TRUE VALUE TRUE VALUE

\\ -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 .o 1.2 I .4

C O E F F I C I E N T

Fig. 5. Histogram for uniform measurement noise.

mates with no overlap are degraded by a factor of 20 with S,;'N=4 and a factor of 6 with S/N=100. As a result, with no overlap the linear estimates have lower rms errors by a factor of 7 with S/N=4 and a factor of 2 with S I N = 10. The degradations of the eigenvector estimates by factors of 20 and 6 are more than the factor of ~ '16/6=2.31 which might be expected from ( 5 ) .

3) Compensating for the lower signal-to-noise ratio by using more data did not significantly change the nor- malized bias of the eigenvector estimates. The normal- ized bias of the linear estimates was even greater at the lower signal-to-noise ratios. The standard deviations of the eigenvector estimates, which in light of (5) should be approximately the same, agree within a factor of 2, except for cases E and F which differed by almost a factor of 10. This larger factor may be caused by the rapidly changing statistics in the vicinity of T = 0.25

-

second which magnify the effects of other influences. The rms error of the linear estimates is also degraded by roughly a factor of 2 at the loxer signal-to-noise ratio.

Distribution sf Estimates

A third example has been chosen to demonstrate ex- perimentally that the distribution of the estimates is Gaussian even u-hen practically the smallest possible number of data vectors were used to make the estimate and when the measurement noise was uniform.

To keep the number of data vectors to a minimum a second-order pulse transfer function

SMITH AXD HILTON: PULSE TRANSFER FUNCTION ESTIMATION 575

with only two unknown coefficients was estimated. This system corresponds to the linear dynamic system

1

s + l

initially at rest, responding to a unit step input with the input and output samples at T=2.0 seconds. Four nonoverlapping data vectors from - Tst 5 +8.0 sec- onds were used. Both the Gaussian and the uniform noise had S/iV= 50.

The histograms for each of the coefficients have been plotted for both Gaussian and uniform noise in Figs. 4 and 5, respectively. The shapes of these histograms are typical of the shapes for the corresponding distributions of the other cases in this paper and elsewhere.['] They are also typical of the shapes for corresponding distri- butions of coefficients estimated by the linear

-4s can be seen from the histograms the distributions are in both cases unimodal. If the x? ('goodness-of-fit test"[''] is used to test the hypothesis that the distri- bution is Gaussian, the hypothesis is accepted for all four histograms when they are tested at the five percent level of significance. Thus, both for Gaussian noise and for uniform noise the distributions should be well de- scribed by just their first- and second-order statistics.

Covariance Xatrices T o indicate the quality of the approximation of ( 5 ) ,

approximate and sample covariance matrices from the second and third examples have been compared. Fig. 6 shows the joint distribution of the two coefficients esti- mated in the third example when the noise was Gaussian. Superimposed on the figure are the two lo ellipses cor- responding to the sample mean and sample covariance matrix of the estimates and to the approximate covari- ance matrix of (5) centered at the true values of the co- efficients. As can be seen from the figure for this exam- ple, and at this signal-to-noise ratio, the approximate and sample covariance matrices agree quite well. The angles of the ellipses agree within a few degrees. The sizes and shapes of the ellipses are also in general agreement.

To compare the approximate and sample covariance matrices of the second example, Table I V gives standard deviations and correlation coefficients obtained from these matrices. Xs can be seen from the table the approxi- mate covariance matrix of (5) gives a good prediction of the standard deviations in this case. However, in case C of the second example, which has the same ap- proximate covariance, the sample standard deviations for three of the four coefficients are twice those pre- dicted. The predictions of the correlation coefficients in case D are less exact than those of the standard devia- tions. The pairs of correlation coefficients other than those for a1 are correlated with the same sign but the magnitude of the approximate correlation coefficients is smaller, ranging from approximately 0.6 to 0.85, while

T CO!rARIANCE

KqTR I X

SAMPLE o.6

MTF. I X . CGVfiPIAKCE

I t f 0.2 O e 4

1 I -0.1 - C . b -c .2 0.2 0.4 0.5

Fig. 6. Distribution of estimates with Gaussian measurement nom.

I t l

TABLE IV

TIOSS AND CORRELATIOK COEFFICIEXTS FOR CASE D, .APPROXIMATE (ABOVE) A S D SAMPLE (BELOW) STAhmARD DEVIA-

SECOh?) EXAMPLE

Coefficients a1 a2 bl br

a1 0.0648 0.0711

0 a2

0.0902 -0.4810 0.1165

bl 0 0.6957 0.1596 -0,3466 0.9332 0.1994

bz 0 0.2759 -0.9057 -0.9710 0.1912

-0.5905 -0.8488 0.1564

the sample values are 0.9 or more in magnitude. For a1

the correlation coefficients predicted to be zero actually had magnitudes in the range 0.3 to 0.5.

COKCLUSIOSS Irrefutable statements about the performance of the

estimation methods cannot be made on the basis of three examples. However, on the basis of the general properties of the estimates observed in these examples and similar examples in Smith and Hilton['] several conclusions can reasonably be drawn.

In these examples the rms error of the eigenvector estimates was generally less than that of the linear esti- mates. With the second-order system, the rms error of

576 IEEE TRA”L\@TIONS O N AUTOMATIC CONTROL, OCTOBER 1967

both methods varied with T and was minimum at a sampling rate 10 to 15 times the characteristic fre- quency, was slightly greater at slower sampling rates, and was almost 10 times greater at rates 30 times the characteristic frequency.

As predicted by Levin, the bias magnitude of the eigenvector estimates is generally less than their stan- dard deviations. The bias magnitude of the linear esti- mates is generally greater than that of the eigenvector estimates and their standard deviation smaller. Thus, its bias magnitude is often greater than the standard deviation, and as a result, much or most of the rms error is caused by the bias.

The properties of the eigenvector estimates are as predicted from theoretical considerations by Levin.r71 The approximate covariance matrix predicts the esis- tence of a sampling interval with minimum standard deviations and the value of the sampling interval. For the sampling interval at and above the minimum i t also gives a good prediction of the standard deviations. The distributions of the coefficients were unimodal and close to Gaussian. The approximate covariance matrix pre- dicted the approximate shape and, generally, the spread of the distribution. Using overlapping vectors gives a decided reduction in the standard deviation of the eigenvector estimates over that of the maximum likeli- hood estimate using nonoverlapping vectors, without increasing the bias of the estimate.

REFERESCES [‘I F. V . Smith and W. B. Hilton. “Laplace transfer function

estimation from sampled data,” Sylvania Electronic Systems-West,

28-043 AMC-O0379(E); November 30, 1965. h’Iountain View, Calif., Tech. RIemo. EDL-31874, Contract D;\

P I V. S. Levadi, Parameter estimates in regression models,” IEEE Trans. Automatic Control (Correspondence), vol. AC-9, p. 589, October 1964.

estimation,” IEEE Trans. Autotnatic Control, vol. .IC-& pp. 347-35i, 131 P. Eykhoff, “Some fundamental aspects of process-parameter

October 1963. [41 G. G. Lendaris, “The identification of linear systems,!‘ Trans.

AIEE (dpplications and Industry), vol. 81, pp. 231-240, September 1962.

[SI Y . C. Ho and E. H. lVhalen, “An approach to the identification and control of linear dynamic systems with unknown parameters,’! IEEE Trans. Automatic Control ( CorresPonde~zce), vol. .IC-8, PP. 255-256, July 1963.

_ _ P I 1. Zaborszkv and R. L. Bercer. “An inteual sauare self-outi-

malyzinng adapt& control,’’ Trans. AIEE (Apzlicati&s and Indus- t r y ) , vol. 81, pp. 256-262. November 1962.

the presence of noise,” IEEE Tram. Az~toma.tic Control, vol. .\C-9, (’1 XI. J. Levin, “Estimation of a system pulse transfer function in

pp. 229-235, Julv 1964. P I K. Steiglifi and L. E. McBride, ‘‘-1 technique for the identiti-

cation of linear systems,” IEEE Trans. Automatic Control (Short Papers). vol. AC-IO, pp. 461-464, October 1963.

Series. Harlem, The Netherlands: DeErven F. Bohm. 1937. P I T. Koopmans, Linear Regressiort Analysis of Economic Tim Pal F. E. Hildebrand, ;Ilethods of Applied AIathematics. Engle-

wood Cliffs, N. J.: Prentice-Hall, 1952.

Random Datu. Kew York: IYiley, 1966. PL1 J. S. Bendat and A. G. Piersol, Xeaszuement and Analpsis of

Fred W. Smith was born in Istanbul, Turkey, on February 22, 1935. He received the B.S. degree from Carnegie Institute of Technology, Pittsburgh, Pa., in 1959, and the 11,s. and Ph.D. degrees from Stanford Cniver- sity, Stanford, Calif., in 1961 and 1964, respectively,-all in electrical engineering.

From 1959 t o 1964 he was associated with Stanford University as a Research Assistant working on the ap- plication of trainable threshold elements to the field of automatic control. Since 1964 he has been with Sylvania Electronic Systems, Inc., R,Iountain View, Calif., where he is conducting research in the fields of automatic con- trol, system identification, and in the application of threshold elements and regression functions to auto- matic data analysis.

Dr. Smith is a member of Eta Kappa N u and Sigma S i .

W. Bruce Hilton was born in Delta, Utah, on October 4, 1928. He received the B.S. degree in mathematics in 1957 and the B.S. and 1I.S. degrees i n physics in 1960 and 1961, respectively, all from Brigham Young Uni- versity, Provo, Utah.

In the summer of 1957 he was emplo>-ed by Boeing Airplane

Company, Seattle, U‘ash., where he n-orked on flight- test data reduction. From 1958 t o 1960 he was associ- ated with Brigham Young Universitl- as a Teaching As- sistant in the Departments of Mathematics and Phvsics and as a Research Assistant on a study of spectroscopic eclipsing binary stars. I n October 1960, he joined Sylvania Electronic Systems, Inc., AIountain View, Calif. Xt the present time he is an Advanced Research Engineer in their Special Studies Department, en- gaged in space systems analysis and signal extraction. He has had extensive experience in the field of astro- dynamics, i.e., trajectory analysis, orbit determination, tracking, and orbit correction. He has also performed threat studies of tactical infrared systems.

11r. Hilton is a member of Phi Kappa Phi and Sigma Pi Sigma, and an associate member of Sigma Xi.