Embed Size (px)
Citation preview
N PS ARCHIVE1966POPE, J.
REAL-TIME ESTIMATION ANDRANDOM SIGNAL DETECTION
JOHN WILLIAM RIPPON POPE, JR.
SCHOOL93940
This document has hoen approved for public
release and sale; its distribution is unlimited.
REAL-TIME ESTIMATION AND
RANDOM SIGNAL DETECTION
by
John William Rippon Pope Jr.
Lieutenant Commander, United States NavyB.S., University of Tennessee, 1954
Submitted in partial fulfillment
for the degree of
DOCTOR OF PHILOSOPHY
IN
ELECTRICAL ENGINEERING
from the
UNITED STATES NAVAL POSTGRADUATE SCHOOL
May 1966
?o?£, 7,ABSTRACT
This work investigates some problems arising in application of
(Kalman) linear filter theory to real problems, where practical es-
timates must replace exact theoretical quantities in problem form-
ulation o The principle objective is application of linear filter theory
to random signal detection/clctWifft?8*Mlh . However, an example of
classical estimation, error estimation in shipboard inertial navigation
systems, is offered to illustrate general points discussed. A unified
treatment of models for random time series is presented, including a
comparative review of models which have been proposed and pro-
cedures for obtaining model coefficients. Correlation detection of
deterministic signals is discussed and the resulting principles ex-
tended to the case of random signal detection. Application of
linear filter theory to the problem is indicated. Finally, an ex-
perimental study in random signal detection/classification is
included. Experimental signals used are hydrophone recordings of
sea noise and sea noise plus diesel submarine. Consistency of
successful results obtained suggests practical utility of method in
certain random signal detection/classification problems.
DUDLEY KNOX LIBRARY
NAVAL POSTGRADUATE SCHOOL
MONTEREY, CA 93943-5101
TABLE OF CONTENTS
Chapter Page
I. INTRODUCTION 13
II. LINEAR LEAST SQUARES ESTIMATION - THEKALMAN FILTER 17
Basic Assumptions 17
The Signal Process Model 18
The Optimal Filter 20
A Priori Information 22
An Example - Estimation of System Error in
Shipboard Inertial Navigation Systems 23
III. MODELS FOR RANDOM TIME SERIES 36
General 36
Fundamental Models for Random Time Series 37
Method of Hidden Periodicities 39
Linear Models 40
Autoregressive Process 40
Process of Moving Averages 41
Linear Filter Model 41
Estimation of Linear Model Parameters 43
Examples 47
IV. DETECTION OF RANDOM SIGNALS IN NOISE 55
Outline of the Problem 56
Correlation Detection of Deterministic Signals 5 6
Random Signal Detection 61
V. AN EXPERIMENTAL STUDY IN RANDOM SIGNALDETECTION 69
Description of the Problem 69
Model Determination 70
Study of Performance 81
VI. RECOMMENDATIONS FOR FURTHER WORK 96
Appendix
I. OPTIMAL ESTIMATION OF AN ARBITRARY LINEAR
COMBINATION OF SIGNAL GENERATING PROCESSSTATE VARIABLES 98
JOOHr-
A&pendix Page
II
.
THE KALMAN FILTER AND RECURSIVE ESTIMATIONOF A GAUSSIAN POPULATION MEAN 100
III. PROGRAM NAVERR 107
IV. PROGRAM MODEL 113
V. PROGRAM OTEST 127
BIBLIOGRAPHY 134
LIST OF ILLUSTRATIONS
Figure Page
2-1 Block Diagram of General Continuous Linear
Dynamic System 19
2-2 Block Diagram of General Discretely DescribedLinear Dynamic System 19
2-3 Block Diagram of Kalman Filter 21
2-4 Equatorial and Local Coordinate Systems 26
2-5 Coordinate System Misalignment 2 6
2-6 Error Locus with Constant Gyro Drifts 29
2-7 Linear Model for Latitude Error 29
2-8 Observed and Estimated Latitude Error -
Unexcited Model 32
2-9 Estimated and Actual Latitude Error - UnexcitedModel 33
2-10 Observed and Estimated Latitude Error - Excited
Model 34
2-11 Estimated and Actual Latitude Error - Excited
Model 35
3-1 Model Poles - 50 cps Sine Wave in Random Noise -
db Signal/Noise Ratio 49
3-2 Autocorrelation Functions Computed from Dataand Model - S/N = db 50
3-3 Power Spectral Densities Computed from Data
and Model - S/N = db 51
3-4 Model Poles - 50 cps Sine Wave in Random Noise -
-10 db Signal/Noise Ratio 52
3-5 Autocorrelation Functions Computed from Dataand Model - S/N = -10 db 53
3-6 Power Spectral Densities Computed from Data
and Model - S/N = -10 db 54
4-1 Optimum Receiver for Deterministic Signals 60
4-2 Optimum Receiver for Gaussian Signals 64
4-3 Optimum Receiver Using Recursive Likelihood
Function Evaluation 68
5-1 Sea Noise Plus Diesel Submarine ("Signal") 71
Figure Page
5-2 Sea Noise ("Noise") 71
5-3 Autocorrelation Function of "Signal" and "Noise" 72
5-4 Z-Plane Poles of Autoregressive Signal Models 73
5-5 Z-Plane Poles of Autoregressive Noise Models 74
5-6 Poles of Representative Signal Model 75
5-7 Autocorrelation Functions from Signal Data andSignal Model 7 6
5-8 Power Spectral Densities from Signal Data andSignal Model 7 7
5-9 Poles of Representative Noise Model 78
5-10 Autocorrelation Functions from Noise Data andNoise Model 79
5-11 Power Spectral Densities from Noise Data andNoise Model 80
5-12 Likelihood Functions for Signal Data ComputedUsing Signal and Noise Models 82
5-13 Likelihood Functions of Signal Using Signal Model 83
5-14 Likelihood Functions of Signal Using Noise Model 84
5-15 Likelihood Functions of Noise Using Signal Model 85
5-16 Likelihood Functions of Noise Using Nbisael Model 86
5-17 Likelihood Functions of Signal Using Signal Model -
Mean Square Signal Received Equal One-fourth
Assumed Value 88
5-18 Likelihood Function of Signal Using Noise Model -
Mean Square Signal Received Equal One-fourth
Assumed Value 89
5-19 Likelihood Function of Noise Using Signal Model -
Mean Square Signal Received Equal One-fourth
Assumed Value 90
5-20 Likelihood Function of Noise Using Noise Model -
Mean Square Signal Received Equal One-fourthAssumed Value 91
5-21 Likelihood Function of Signal Using Signal Model -
Mean Square Signal Received Equal Four TimesAssumed Value 92
5-22 Likelihood Function of Signal Using Noise Model -
Mean Square Signal Received Equal Four TimesAssumed Value 93
Figure Page
5-23 Likelihood Function of Noise Using Signal Model -
Mean Square Signal Received Equal Four TimesAssumed Value 94
5-24 Likelihood Function of Noise Using Noise Model -
Mean Square Signal Received Equal Four TimesAssumed Value 95
LIST OF ABBREVIATIONS
x(t) State vector of continuously described linear dynamic
system
y(t) Observation vector of system
z(t) Vector of noisy observations, equals y(t) + v(t)
w(t) White noise excitation vector
v(t) Vector of white noise contaminating observations
F(t) Matrix describing the homogeneous system
G(t) Input matrix of linear dynamic system
H(t) Output matrix of system
Q Covariance matrix of excitation noise vector w(t)
q Variance of scalar valued excitation noise
R Covariance of vector valued observation noise
r Variance of scalar valued observation noise
z" 1 Unit delay
x(k) State vector of discretely described linear dynamic
system
y(k) Observation vector of discrete system
z(k) Vector of noisy discrete observations
w(k) Vector of white noise excitation
v(k) Vector of white noise contaminating observations
$(k) State transition matrix of discretely described linear
dynamic system
T(k) Input matrix for discrete system
H(k) Output matrix for discrete system
S Covariance of system state vector
x(k/j) Optimal estimate of state vector x(k) given data
to time j
P(k/j) Covariance of error of optimal state estimate at
time k given data to time j
B(k) Kalman filter gain matrix
n Order of linear dynamic system
N Number of signal samples to be processed
R( t) Autocorrelation function
I Identity Matrix
10
ACKNOWLEDGEMENT
The author wishes to express sincere appreciation to Professor
Harold A. Titus for his guidance and encouragement throughout the
period of this research.
11
CHAPTER I
INTRODUCTION
It is frequently an integral requirement in engineering information
processing systems that measurement data be employed in subsequent
decision making or action. Invariably, the processing problem lies
in a stochastic environment because of inaccuracies in the measure-
ment process or random perturbations of the observed process during
or between measurements. Some examples are determination of
satellite orbit parameters from noisy radar or radio-location data,
target track determination in the radar fire control problem , filtering
of signals received in the presence of noise, and state estimation in
linear dynamic systems from noisy transducer information.
What might be termed "classical" estimation theory in engineer-
ing has been based upon the pioneering work of Wiener published
in 1942. This theory rests upon three main assumptions: (i) linear
operations on received data, (ii) a quadratic loss criterion, and
(iii) stationary statistics describing the signal and noise processes.
In applications, the objective is to obtain a specification of the
electrical filter (the Wiener filter) giving optimum estimation, or
isolation, of the signal from corrupting additive noise. Posed in
terms of integral equations, the problem results in the Wiener-Hopf
integral equation, which must be solved to obtain the impulse re-
sponse of the optimum filter. Broad application of the theory has
suffered from the difficulty of obtaining solutions for the Wiener-
Hopf equation.
In 1960, Kalman posed the estimation problem in terms of
differential equations, using the concept of "state" from linear
system theory. Characterization of the problem in terms of
differential equations offers several advantages . Chief among them
are suitability of the new approach for machine computation, re-
laxation of the stationarity assumption, and the intimate relationship
between the optimum filter and a linear model describing the signal
13
process. The work here is motivated by the contributions of Kalman
and considers application of his techniques to problems in statistical
communication theory.
In the title, the term "real-time" means that computational effort
for estimation procedures to be employed does not increase with in-
creased numbers of observations. Further, following the practice of
recent years, the collective term "estimation" is considered to include
the traditional problems of smoothing, filtering, and prediction. Since
digital computation is to be employed in examples, and since the
required mathematics is less obscure, the present work will deal
almost exclusively with estimation of discrete signals. For engineer-
ing purposes, little generality is lost. The description of continuous
filters can in general be obtained by considering the limiting case of
the corresponding discrete filter as the sampling interval is allowed
to approach zero.
In Chapter II, basic assumptions underlying optimum estimation
theory developed by Kalman are reviewed, and some practical con-
siderations are discussed. As an example, the problem of estimating
latitude error in shipboard inertial navigation systems is outlined,
and application of the Kalman filter to the problem is described.
Chapter III considers the important problem of finite parameter
models for random time series, a subject not heretofore treated in
a unified manner. First, the Spectral Distribution Function, intro-
duced for the discrete time case by Wold in 1938, is discussed and
its three constituent distribution functions noted. Then models which
have been proposed for time series are considered with regard to their
capacities for representing the various constituent functions. The
combined autoregressive-moving average model is shown to be
equivalent to the linear filter, and the relationship between their
respective parameters is indicated. A discussion of parameter es-
timation is then included. Present difficulties in solving for moving
average model parameters is noted. It is shown that the approach of
14
Ho and Lee [16 ] to the identification problem amounts also to a
recursive means for estimating autoregressive model coefficients.
Finally, the recursive approach is employed in an example to obtain
autoregressive model coefficients for a sample signal. The signal
consists of a 50 cycle sine wave combined additively with noise from
a laboratory random noise generator. A plot of z-plane poles of the
model is given. Also included are comparative graphs of the auto-
correlation functions and power spectral densities derived from the
signal and model respectively.
The detection of random signals in noise is considered in Chapter
IV. As it happens, optimum detection of Gaussian random signals
constitutes a simple (at least conceptually) extension to the idea of
correlation detection of deterministic signals. In the case of Gaussian
random signals, the optimum receiver forms a cross correlation of the
received signal and a least squares estimate of the Gaussian signal
given the received signal . The extension is shown following a brief
development of correlation detection of deterministic signals. Diffi-
culties involved in directly evaluating the Likelihood Function for
each alternative classification are noted. Then employing the approach
of Schweppe [31 ], it is shown that the Likelihood Function may be
evaluated recursively. For random signals which are single time
series, no matrix inversion is required. Quantities required in the
recursion relations may be obtained from Kalman filter estimation of
the Gaussian signal given the received signal. On-line computations
may be greatly reduced by storage of short lists of required quantities
which do not depend upon the received signal. Using stored lists,
on-line multiply-add combinations per received signal sample are
reduced to approximately 2n + 2 for each classification alternative,
where n is the order of signal models used.
An experimental problem in random signal detection investigated
in Chapter V represents the first testing against actual signals of the
methods developed or discussed in this dissertation. The experimental
15
signals used are hydrophone recordings of (a) a diesel submarine plus
sea noise and (b) sea noise alone. After obtaining autoregressive
models for the signal-plus-noise and noise processes, recursive
Likelihood Function evaluation discussed in Chapter IV is employed to
test performance of optimum linear classification on actual signals.
Consistent correct classification is obtained within less than 100
samples of signal record (representing less than 60 millisecs of signal
record). Furthermore, classification efficiency is maintained over
rather wide deviations between actual and assumed received signal
parameters. Hence initial results suggest practical applicability of
the method to certain real-time random signal detection/classification
problems
.
16
CHAPTER II
LINEAR LEAST SQUARES ESTIMATION -
THE KALMAN FILTER
Interest in time-domain estimation techniques and applications
has been stimulated in recent years by the contributions of R. E.
Kalman [19 ]. Subsequent investigators have related Kalman's
derivation and results to more familiar estimation techniques, thereby
enhancing general understanding and utility of the new approach. Per-
haps the simplest derivation of Kalman's results lies in the Bayesian
approach employed by Peschon and Weaver [26 ], [35 ]. Another
step toward better perspective was the demonstration by Lee [21 ]
and Fagin [10 ] of the equivalence between Kalman's results and
generalized recursive least squares estimation. A unified and
general treatment of modern estimation theory by Deutsch [9 ]has
recently been published.
The purpose of this chapter is to review the basic assumptions
underlying Wiener-Kalman filter theory and implications of the
assumptions. Further, practical considerations arising in appli-
cations are discussed. In this regard, a comparative discussion of
the Kalman filter and generalized, recursive estimation of a Gaussian
population mean is introduced to explicitly show the role of a priori
information. Finally, a practical problem is considered to illustrate
an application of the methods treated here.
I. BASIC ASSUMPTIONS
The two basic assumptions underlying optimal filter theory are
(1) linear operations on the data are to be performed, and (2) the
criterion for goodness is mean squared error. Under these assump-
tions , it is well known that only second order statistical moments
are employed in the estimation scheme [8 ]. An immediate result is
that for non-Gaussian signal processes, the optimal linear estimate
obtained is the same as would have been obtained from the corres-
ponding Gaussian signal process (i.e. possessing the same first
17
two moments). Moreover, the optimal linear least squares estimate
of a Gaussian process is identical with the Maximum Likelihood
estimate. The conclusion follows that linear least squares estimation
may be improved upon only for non-Gaussian processes, and then
only by an estimation procedure which takes into account third or
higher statistical moments of the process. Conversely, in dis-
cussions restricted to linear least squares estimation, assumptions
of Gaussian process statistics can be made without effect on the
estimation outcome.
II. THE SIGNAL PROCESS MODEL
It is commonly assumed in engineering literature on stochastic
estimation that the scalar or vector-valued random time series to be
processed can be regarded as the output of a linear dynamic system
excited by white noise. Implications of this model will be discussed
in greater detail in Chapter III. It is also assumed that observations
of the system output are obscured by additive Gaussian white noise.
Block diagrams of continuous and discrete descriptions of the model
are given in Figs. 2-1 and 2-2 respectively.
The general continuous linear dynamic system model may be
described by the vector differential equations
x(t) = F(t)x(t) + G(t)w(t)
y(t) = H(t)x(t)
z(t) = y(t) + v(t)
where
x(t) is the n x 1 vector of system states.
F(t) is an n x n matrix describing the homogeneous system.
w(t) is a p x 1 vector of Gaussian white noise
G(t) is an n x p matrix which distributes excitation noise
across the states.
y(t) is an m x 1 vector of system output.
H(t) is an m x n observation matrix.
v(t) is an m. x 1 vector of white Gaussian measurement
noise
18
w(t)
Hftx(t)
s= / H
v(t)
32£=>
Block Diagram of General Continuous
. Linear Dynamic System
Fig. 2-1
w(k)
v(k)
Block Diagram of General Discretely
Described Linear Dynamic System
Fig. 2-2
19
z(t) is an m x 1 vector of observations.
Excitation and measurement noise sources are assumed to be indepen-
dent. Characteristics of the noise vectors w(t) and v(t) are:
E[w(t)]=E[v(t)]=
E[w(t)w(T)T ]= (Q t==TlO t^T
E[v(t)v(T)T]= {* * = T
LO t f T
For the corresponding discretely described linear dynamic system,
the describing vector difference equations are
x(k) = $(k)x(k-l) + T(k)w(k)-1)
y(k) = H(k)x(k)
z(k) =y(k) + v(k)
Corresponding to the continuous case, the noise vectors w(k) and
y(k) are assumed to be samples of a vector-valued Gaussian noise
process, and are assumed to be independent from sample to sample.
For w(k), this assumption of uncorrected samples represents no loss
of generality, since the excitation noise is fictitious anyway. On
the other hand, measurement noise encountered in certain applications
may not be totally uncorrelated from sample to sample. In such cases,
the correlated portion may be accounted for by augmenting the system
order as necessary. The form of the resulting augmented system is
simply arrived at. Actual numbers to describe the measurement error
correlation may be obtained by considering the points discussed in
Chapter III
.
III. THE OPTIMAL FILTER
References cited above include derivations of the Kalman filter
employing variously geometric (orthogonal projections in Hilbert
space), Bayesian, and Maximum Likelihood approaches . Addition-
ally, equivalence of the optimal filter and recursive linear least
squares prediction has been indicated. It will therefore suffice for
the work to follow to include here only the equations describing the
optimal filter and the corresponding block diagram (Fig. 2-3). The
20
*0O_±B
xdc/k)
x(k/k-
H
,-/
$
x(k-l/k-l)
Block Diagram of Kalman Filter
Fig. 2-3
21
equations given are for the discrete formulation.
B('k) = P(k/k-l)HT (HP(k/k-l)HT + R)" 1
x(k/k) = <fcx(k-l/k-l) +K(k)[z(k) - H &x(k-l/k-l)]
P(k/k) = [I - K(k)H]P(k/k-l)
P(k+l/k) = *P(k/k) 4 T + T Q T T
In the equations given, P is an n x n covariance matrix of the
estimate. The double index is used to mean that P(k/k-l) is the
variance of x(k/k-l) / the optimal state estimate at time increment k
given data only up to time k-1 (x(k/k-l) = $x(k-l/k-l)).
While the equations above describe optimal linear estimation of
the signal process model states, what frequently is required is the
optimal estimate of some linear combination of the states, for ex-
ample y = Hx. In order that the equations above be useful in such
a case, it must be so that the optimal estimate of a linear combin-
ation of states is the same linear combination of the optimal estimate
of the states. A proof of this point is included in Appendix I.
IV. A PRIORI INFORMATION
As is readily noted, the difference equations above which de-
scribe behavior of the Kalman filter or recursive least squares es-
timation require a starting point. What is needed is an initial
estimate of the states x(l/0), and an initial covariance matrix
P(l/0), indicating the uncertainty in the initial estimate. These
quantities represent a priori knowledge about the process to be es-
timated since they are required to be specified before any obser-
vations are made. An intuitive feel for the role of x(l/0) and P(l/0)
may be gained by comparing behavior of the Kalman filter and gen-
eralized recursive estimation of a Gaussian population mean, out-
lined in Appendix II. It is shown there that a priori information
appears to the filter just as additional equivalent observations of the
process to be estimated.
It is difficult in many problems to arrive at meaningful values for
22
the a priori quantities required. If the signal process model is stable
and the excitation noise covariance Q is known, a priori knowledge
may be obtained directly from knowledge of the model. Motion of states
of the model is a Gaussian random process, a linear function of the
Gaussian excitation process. Since the excitation has zero mean, the
expected value of states of the model given no observations is also
zero. Thus x(l/0) equals the zero vector. Further, the covariance
describing uncertainty in state values is the solution matrix S of
the difference equation
s = $s$T + rQrT
At any stage, the matrix P(k/k-l) is defined as the error co-
variance matrix at time k given observations to time k-1. Hence
P(l/0) = E[(x(l) - x(l/0))(x(l) - x(l/0)) T ]
= E[x(l)x(l)T ]
= S
Another suggested approach when a priori requirements present
difficulty is to specify some reasonable value of x(l/0) and let
P(l/0) = cl, where c is a large number. It is motivated by the fact
that weighting of measurement data is in inverse proportion to un-
certainty in the data relative to uncertainty of the current best estimate
before data receipt. Thus effects of the initial assumptions may be
washed out by the incorporation of new data [21], [10].
In the example below, both the above methods are tried and com-
parative results noted. Further use of the two methods is made in
the experimental study of Chapter V. There the latter method is used
in the recursive model determination program, and the former is used
in the program for computation of Likelihood Functions
.
V. AN EXAMPLE - ESTIMATION OF SYSTEM
ERROR IN SHIPBOARD INERTIAL NAVIGATION SYSTEMS
An example to which the techniques described above may be very
naturally applied is the problem of error estimation in shipboard inertial
23
navigation systems (SINS). Gyroscopes used in the system to provide
a stable inertial platform are subject to certain random drifts. Hence
in order to maintain the necessary accuracy of the system over in-
definite periods at sea, it is necessary to determine system errors
for use in recalibrating or resetting the SINS. Such an error deter-
mination necessarily depends upon information from external sources;
in this case upon LORAN position information and position/heading
information from star sightings, etc. However, information from these
sources is imperfect due to measurement inaccuracies. Additionally,
it is desired to have to resort to LORAN or other observations as
infrequently as possible, so optimal use of observational data is
indicated. These conditions, optimal estimation from noisy obser-
vations, comprise justification for considering the problem here.
The problem will be pursued here only far enough to illustrate
application of the method. Thus only the problem of estimating
latitude error will be discussed in detail. Without further theo-
retical complication, the treatment may be expanded to encompass
remaining aspects of the error estimation problem. Optimal control
techniques may then be employed in system reset, a step under
current development [5 ] .
1 . Background
As stated above, SINS employs gyroscopes to stabilize a gimbal-
supported inertial platform. Remarks to follow will relate to the
four-gimbal system -1
- , the innermost two gimbals of which are termed
the "latitude gimbal" and "heading gimbal." Upon the innermost
gimbal, the latitude gimbal, are mounted three stabilizing single-
degree-of-freedom gyros with mutually perpendicular input axes.
'-The present discussion will be only detailed enough to outline
the estimation problem to be considered. More detailed discussionof inertial navigation systems may be found in [24 ], [32 ].
24
This basic structure will be discussed further below.
Two coordinate systems fixed in the gimbals describe operation
of the system. They have been termed the "instrumented equatorial
coordinate system", fixed in the latitude gimbal with axes E' , Y'
,
and P 1
, and the "instrumented local coordinate system", fixed in
the heading gimbal with axes x' , Y' , and z 1
. These axes correspond
to "earth axes" E, Y, P, x, and z shown in Fig. 2-4. The latitude and
heading gimbals are pivoted to each other on the Y 1 axis.
Identification of axes shown in Fig. 2-4 follows:
E is parallel to intersection of local meridian plane with
plane of equator, positive outward.
Y is in local horizontal plane and extends eastward.
1? P is parallel to polar axis of earth, positive toward
North pole
.
x is in local horizontal plane and extends northward,
z is parallel to local vertical, positive downward.
Having introduced the coordinate systems employed in SINS,
orientation of the siabilizing gyros may be more completely described.
As stated, they are mounted on the innermost, or latitude gimbal,
in which is fixed the instrumented equatorial coordinate system. The
gyros are termed the equatorial, latitude, and polar gyros, and are
mounted with their input axes parallel to the E' , Y 1
, and P' axes
respectively. Drift rates of the three gyros are denoted, respectively,
by €£, cL , and €p.
In order that the instrumented equatorial coordinate system
correspond accurately with the earth's equatorial coordinate system,
the latitude gimbal is torqued about the P' axis at the rate Q+ \ where
Q= Earth rate
X = Ship longitude rate obtained via an accelerometer on
the inertial platform.
Additionally, the heading gimbal is torqued with respect to the latitude
gimbal at a rate (-L 1
) equal to the negative of the SINS indicated
25
Equatorial and Local Coordinate Systems
Fig. 2-4
Coordinate System Misalignment
Fig. 2-5
26
latitude rate (also obtained via an accelerometer on the inertial
platform). In operation, the system indicated latitude is given by the
angle between the latitude and heading gyros , and indicated longitude
is given by the angle between the latitude gimbal and a reference.
This completes a brief description of system operation.
2 . Error Generation
For the present discussion, operation is assumed to be in "damped
inertial" mode, reducing Schuler oscillation errors to comparatively
insignificant dimensions. Nevertheless, it remains necessary to
estimate errors resulting from imperfect system resets and the gyro
drifts c r , c , and e p mentioned above.
When the system is misaligned, the instrumented and earth equa-
torial coordinate systems do not coincide as desired. Misalignment
consists of small angular displacements about the Y axis and
0£ about the E axis. The situation is depicted in Fig. 2-5, where
the vector P 1 has unit length.
In the absence of gyro drift, the P' axis is motionless relative to
inertial space, while the earth coordinate system rotates about P
at the earth rate. The projection of P' in the equatorial plane thus
describes a circle centered at the origin and repeated every 24 hours.
Differential equations describing the circle are
L = -fi0E
0£ = 0,0-^
Introducing gyro drift, ^ acquires an additional rate of change
equal to cT
, the latitude gyro drift. Similarly,E
acquires an addition-
al rate of change equal to c Ecaused by equatorial gyro drift. The
equations become
0*L = -fi0E + € L
E = O0L + €£
If it is assumed that the drift rates e, and e P are constant, the
27
equations may be written
d
dt (0L +fi
)- - fi( E -fi
d
dt uE-
fi
") = fi(L+
€E
fi)
The locus is still a circle but is centered off the origin as shown in
Fig. 2-6.
3 . Error Estimation
Importance of the error locus lies in its relation to physically
measurable errors. Under the common assumption of perfect knowledge
of the local vertical (i.e. z 1 and z vertical), it has been shown
[32 ] that
L- CL
E= £ HcosL
where
£ L = Latitude error
£ H = Heading error
In general, only position information is conveniently and regularly
available. Hence only can be directly estimated. By use of
three or more latitude error observations, the circle may be determined
to within the Y axis displacement €L ,
fi
Current operating procedure involves a manual graphical fit of
a 24 hour period sine wave through plotted succeeding values of
latitude error versus time. The method is patently both time con-
suming and vulnerable to human error. In the discussion below,
this same reduced problem will be treated with a recursive least
squares procedure (Kalman filter). While, as stated, no further
theoretical complication is involved if treatment is extended to the
complete problem, all essential features needed to show the
application are present in the reduced problem unobscured by
arithmetic.
28
Error Locus with Constant Gyro Drifts
Fig. 2-6
w(t)
v(t)
1
s >
f'
-©-t
V—w
SX+<*Z
z(t)
Linear Model for Latitude Error
Fig. 2-7
29
4. Recursive Least Squares Error Estimation
In order to employ the Kalman filter, it is necessary to specify a
model for the observed process in terms of a linear dynamic system
excited by white noise. Furthermore, it is necessary to possess a
priori information about the amount of noise contaminating obser-
vations and about the mean squared value of the observed process.
Since only estimation of latitude error (which equals ^ T) is to
be considered in the reduced problem, a simple form for the modeil
may be written by inspection. It appears in block diagram form in
Fig. 2-7. The error locus of Fig. 2-6 may be considered to be pro-
jected upon the E axis. Hence error estimation in the reduced
problem amounts to estimation of the unknown amplitude, phase,
and offset ( - € E ) . The singular model of Fig. 2-7 has a null GQ
matrix, and its behavior is thus determined solely by the unknown
initial conditions. In Fig. 2-7, the pure integrator leg preserves the
initial offset;, and the sfecorid order legrrepresents the 24 hour period
sine wave of unknown amplitude and phase. Using this model, a
priori quantities used in the filter are x(l/0) = and P(l/0) = 10,0001.
The model of Fig. 2-7 was converted to discrete form suitable for
digital computation. The resulting matrices describing it are given
below (for T = 1 hour)
.
r = * =
1 1
.9659 .9886 H = 1
-.0678 .9659
In order to obtain a priori quantities in the alternate manner
noted above, the model of Fig. 2-7 may be made absolutely stable
by the addition of damping in each leg. The precise 24 hour period
of oscillation and essentially constant offset (constant drift rate)
suggests that only slight damping be used. Then a value of q ,
variance of the white excitation noise, is specified such as to
obtain steady state peak oscillation of about 5-10 minutes of arc,
30
a rough mean peak oscillation observed in practice.
In the simulation, latitude measurements were assumed available
every three hours, and were all assumed to be of the same quality.
Assumed standard deviation of measurement noise was five. It was
assumed that an updated latitude error estimate, with its corres-
ponding variance, was desired hourly. Hence updating without data
is done in the intervening hours between receipt of measurements.
Filter computations are straightforward, and need not be elab-
orated upon. Results of the simulation are displayed in Figs. 2-8,
2-9 for the unexcited model, and Figs. 2-10, 2-11 for the stable
model. Comparison of the resulting estimates shows that both
models achieved approximately the same estimation accuracy. The
simulation program listing is included in Appendix III.
31
cO
IICO cO
z: £:3 ^
+ +
*-5 LO
I'i:
LU UJ
co <oi
010-
Observed and Estimated Latitude Error - Unexcited Model
Fig. 2-8
32
«4 to
^ <r
^m 000 S00- 010-
Estimated and Actual Latitude Error - Unexcited Model
Fig. 2-9
33
+
+
+^.—-~^~^
+
\
1^
+
IT*
+
+
+^- CM<3?<S?
s ^^^"
^
+• fS?
&£•—I
v. v.
<o COh- I-
5E8
+ +
g$»-! LO
8 .
II
LU LU
<¥_J
x V
S00 000 S00- 010-
Observed and Estimated Latitude Error - Excited Model
Fig. 2-10
34
CO
10
^*~^~««^
1
CNCD
t—l
CD
col- h-
5 §$
+ +£
<5 <£>•
LO
a 1
UJ UJ
4 a<r> en
X >-
S00 000 900- 010-
Estimated and Actual Latitude Error - Excited Model
Fig. 2-11
35
CHAPTER III
MODELS FOR RANDOM TIME SERIES
I. GENERAL
The purpose of this chapter is to treat some valid questions which
arise when application of Kalman estimation techniques is considered
for problems in random signal detection and pattern recognition. In
such problems, the nature of possible random time series to be pro-
cessed becomes quite general, and some uncertainties faced are:
(i) . Is it still a justifiable assumption that the time series
to be processed is the realization of a Gaussian process?
(ii) . Can it be expected that an acceptable finite-parameter
representation of the signal process may be found?
(iii) . What finite-parameter schemes have been considered in
the literature on time series analysis ?-
(iv) . What schemes are available for obtaining the various
finite-parameter representations ?
In considering further the questions above, it will be helpful
to review pertinent work done in the field of time series analysis.
In recent years, works of Tukey [4 ], Grenander and Rosenblatt
[11, 12 ], Parzen [25 ], and Wold [39 ] are representative and
contain references to additional literature on the subject. The central
problem in time aeries analysis is the following: one is allowed to
observe a time series, a partial realization of a stochastic process,
and on the basis of this observation is required to make statistical
inference about the time series. In general, the required inference
concerns the possible mechanism generating the time series or pre-
diction of future behavior of the series .
Work done to the present in time series analysis has primarily
dealt with single time series 1, resulting from stationary (and ergodic)
It is only the single time series, i.e. the single-output signal
generating process, which it is desired to consider here. Certainwork has been done in the analysis and model determination of
multivariate time series, and may be found in [3 ], [40 ], [30 ].
36
random processes with finite first and second moments. In fact, the
great majority of effort in the analysis of random time series has been
directed toward spectral analysis or correlation analysis, employing
only the first and second moments of the generating process. This
has come about for two reasons. First, it is possible to accomplish
a great deal with only second order quantities, and second, the use
of higher order moments introduces as yet intractable difficulties
[13 ]. Thus the theory which has been developed has been tailored
to fit the methods used. Of course in applying the theory, it is
necessary to keep in mind that there are time series about whose
statistical behavior only limited information is conveyed by the
correlation or spectrum. Two time series may have the same auto-
correlation function or power spectral density, yet possess quite
different statistical properties. A classic example is provided by
the so-called random telegraph signal and the output of a proper
low-pass filter excited by Gaussian white noise.
From the above discussion and the fact that the Kalman tech-
nique employs linear processing, the answer to (i) is clear. As
long as the processing scheme to be used employs only the first
two moments of the time series to be analyzed, then without loss of
generality, the subject time series may be replaced by any time series
having the same autocorrelation function [6 ]. In particular, it may
be replaced by a Gaussian time series, a sample function of a
Gaussian random process, which is completely described by its
first and second moments
.
Remaining sections of this chapter will deal with the questions
of finite parameter representations of time series and methods of
estimating the parameters.
II. FUNDAMENTAL MODELS FOR RANDOM TIME SERIES
As stated above, the principal matter of concern in time series
analysis is the making of inferences about the structure of a random
process {y(t) } based on observation of a partial realization of the
37
process, a time series y(l), y(2), . . . y(N). In general, analysis
requires a mathematical model of the time series. For practical reasons,
it is highly desirable that the model be of finite-parameter form, and
moreover, that it be of as low an order as possible.
Since attention here is being restricted to linear processing
schemes, for which as seen above, the random series may be adequate-
ly represented by its first and second order moments, use in analysis
of either the power spectrum or autocorrelation function is equally
valid. For investigating modeling potentialities, it turns out to be
more convenient to deal with the power spectrum than the covariance
sequence. *
In 1938, Wold [38 ] show4dl that for the discrete time case,
there exists a non-decreasing bounded function 3 (o$ , defined for
-tt £ a) ^ tt , such that
7T
= 1R(t)= J e^ T d3(u) T = 0, ±1,
The function 3 ( lc) is called the Spectral Distribution Function of the
time series. This theorem by Wold corresponds to that of Khintchine
in 1933 for the continuous case, where he showed that there exists
a non-decreasing bounded function 3 ( w) , defined for -®£ u> s °°/
such that
R(t) = J e j WT d 3(o>)
3 (u>) can be uniquely written [5 ] as the sum
3 (u>) = 3ac(u>) + 3 ft( w) + Bsc(a>)
where the three distribution functions have the following properties.
The work here will deal with discrete-time random time series.
Though the supporting theory has been developed for the continuoustime case, it is considerably more complicated [3 ] than that for
discrete-time sequences. Besides, for engineering purposes, usefulresults in the continuous case may often be had by taking discrete-
time results to the limit.
38
3ac(u>) is absolutely continuous and is the integral of a non-negative
function F(oj) called the Spectral Density Function or Power Spectral
Density of the time series. The function 3d(a0 is a discrete func-
tion which represents the contribution of power at discrete frequencies
to the Spectral Distribution Function of the time series. It may be rep-
resented as
3d(u>) = I A 3(u>i)
where
A 3 (wi) = 3 (o>i + 0) - 3 (wi - 0)
The remaining constituent function 3sc(u>) is a singular continuous
function, constant except on a set of Lebesgue measure zero [11 ] .
It has been asserted [13 ] that this part does not appear to be mean-
ingful observationally, and it is commonly neglected in the literature.
Having noted the elemental parts of the general Spectral Dis-
tribution Function 3 (to)/ it will be constructive to examine models
which have been proposed for time series and to consider their
capacities for representing the various constituent functions.
Method of Hidden Periodicities
One of the earliest models in the history of time series analysis,
the so-called scheme of hidden periodicities was introduced in 1898
by Schuster in connection with meteorological studies . The model
assumes that the observed random sequence { y(k) } may be
represented as the sum of a mean value function m(k) and white
noise v(k)
y(k) - m(k) + v(k)
where the mean value function m(k) represents a systematic
oscillationm
m(k) = I A. cos(o),k + . )
.
i=l iiiIn the model, the amplitudes A, , angular frequencies &., and
39
phase angles 0. are constants, some of which are known and some
to be estimated. The samples of white noise v(k) have the properties:
E[v(k)] = E[v(k)vO)] = {f *T"J
Obviously, the spectral distribution function of any signal gen-
erated by such a harmonic model would correspond only to the dis-
crete spectral distribution function above. But time series observed
in nature generally possess at least a mixed spectrum [11 ] , whose
spectral distribution function contains both the discrete and abso-
lutely continuous components. Furthermore, to a good approximation,
many physically observed time series may be considered to have
spectra corresponding only to the absolutely continuous spectral
distribution function 3ac(oj). Hence, as might now be predicted
by the theory (still nonexistent at the time the scheme of hidden
periodicities was introduced), such a harmonic model did not perform
satisfactorily for many time series analysis problems [39 ].
Linear Models
Of greater practical interest are the various linear models which
have been proposed. The spectra of such models correspond to ab-
solutely continuous spectral distribution functions. This class of
models includes the Autoregressive and Moving Average schemes
introduced in the late 1920's by Yule and Slutzky, and the linear
filter excited by white Gaussian noise used by Rice [29 ]. Models
of the class are defined below, and techniques for estimating their
parameters are considered in the next section.
Autoregressive Process
The current value of the time series is represented as the sum
of a linear combination of the past n values of the time series and
an uncorrelated disturbance w(k) , where n is the order of the
process .
In difference equation form, the autoregressive process is
y(k) = b^Ck-1) + b2 y(k-2) + . . . bny(k-n) + w(k)
40
where the disturbance w(k) has the properties
,
E[w(k)]=0 E[w(k)w(j)]=/a2 k =j
k M%A necessary and sufficient condition that the model be convergent is
that all roots z^ of the polynomial
z + b , z ~ +...b , z + b1 n-1 n
lie within the unit circle.
Process of Moving Averages
For a model of order m, the current value of the time series is
represented as a linear combination of the past m values of the
uncorrelated disturbance. In eguation form,
y(k) = a w(k) + a w(k-l) + . . . a w(k-m)1 m
where w(k) is defined as for the autoregressive model above.
Linear Filter Model
In discrete-time formulation, this model takes the form
x(k) = $x(k-l) + T w(k)
y(k) = Hx(k)
where
x(k) = Vector of states at time k of the linear dynamic system
comprising the filter.
$ = State transition matrix of the linear dynamic system.
r = Vector which distributes the excitation noise across states
of the filter.
H = Observation vector, which relates the scalar observable
to states of the filter.
y(k) = Value of the random time series at time k.
w(k) = White Gaussian excitation noise as defined above.
Models a and b above, on the one hand, and model c on the
other, have been employed by different groups of researchers for
seemingly very different problems . Models a and b, though initially
41
proposed for studies of sunspot activity, have in recent years been
most extensively employed in econometrics research and have to a
degree come to be associated with that field of study. Conversely,
the linear filter model has been studied almost exclusively by the
engineering community concerned with random signal representation
and stochastic control problems. It is important, however, to note
their basic similarities.
The picture may be cleared by noting the capacity of each model
to approximate the power spectral density of a time series to be
modeled. As stated above, the spectra of linear models correspond
to 3ac(a))/ the absolutely continuous spectral distribution function.
The related power spectral density F(u>) may be approximated ar-
bitrarily closely by a rational function in oj of finite order [12 ].
Now in the z domain, for a spectral density which is approximately
zero beyond the appropriate Nyquist frequency, there is a corres-
ponding spectral density function rational in z, so that
A(z)=l^-Q(z)
where P and Q are polynomials
.
Since the present discussion concerns discrete-time analysis,
it is the relation between the linear models above and A(z) which
is of interest. Also, since the time series to be dealt with are real,
A(z) may be written as follows for z on the unit circle:
A(z) = -PfeL = A{z)A(8)Q(z) B(z)B(i)
z
where
A(z) =a z m + a zm_1 + . . a1 m
^* \ i n , n-i .
g(z) = b z + b z + . . . b1 n
Whittle [36 ] has shown that a rational spectral function where A has
order m and B has order n always corresponds to a process structure
42
bx(k) + b x(k-l) + . . . b x(k-n)1 n
= a w(k) + a w(k-l) + . . . a w(k-m)1 m
Such a model structure has been termed a mixed autoregressive-
moving average model, the obvious title.
From the discussion above, relationships between the linear
models mentioned and the power spectral density is clear. The linear
filter model and mixed autoregressive-moving average model are
equivalent and provide a rational approximation to the power spectral
density. A moving average model of order m produces an mth order
polynomial approximation to the spectral density function, and an
n"1 order autoregressive model produces an n order inverse poly-
nomial approximation to the spectral density function.
This section has discussed questions (ii) and (iii) raised in
section I . A remaining question of key practical importance is
parameter estimation for the model selected. It will be considered
in brief detail next.
III. ESTIMATION OF LINEAR MODEL PARAMETERS
It has been suggested above that linear models have valuable
practical utility for representing time series. Final employment in
real applications, however, rests upon the ability to estimate the
necessary model coefficients. The problem differs quite significantly
from the problem of plant identification in control system engineering.
In the present case, inputs to the filter are known only in statistical
terms, and as a matter of fact, are not physical quantities at all,
but are fictitious
.
In the section above, the linear filter and mixed autoregressive-
moving average models were asserted to be equivalent. It is there-
fore helpful to express the linear filter in such a form that corres-
pondence between parameters of the two models may be easily shown.
The "standard canonical form" of [21 ] is such a form. It is obtained
43
through a linear transformation of the state variables, and involves
no alteration of the filter input-output relation. The observability-
requirement for such a transformation is no restriction in the present
situation. In the "standard canonical form", the three 'matrices de-
scribing the filter input-output relation have the form:
r =
'i
^2
<| =
-b.n
-b2"bl
H =
1
After normalizing coefficients of the mixed autoregressive-
moving average model to obtain bn
equal unity, remaining auto-
regressive coefficients b. directly equate to elements of the $
matrix bottom row as shown. Equation of moving average coefficients
a.A with filter parameters is not as direct and is given below.
m
1
b.
>n-l >1 1 yn
(m=n-l)
The relation above may be obtained either by generalizing from low
dimensional systems or deriving directly for the general case [14 ].
No general techniques presently exist for estimating the para-
meters a. and b. of a linear filter in a straightforward manner from
observed data. However, if the time series can be considered to be
sufficiently represented by a moving average model, then the
following relations may be solved to obtain the coefficients of the
44
polynomial A(z).
a al
6Q
m
am-l
a al
a
[V 'r(O)"
al
r(D
•
• •
•
l mJ
•
r(m)_
where t{t) are values of the observed autocorrelation function of
lag t. As it happens, there are 2m possible ways in which zeros
of the product A(z)A(I ) may be assigned to A to obtain the samez
power spectral density (and autocorrelation function) . Hence there is
an indeterminacy of order 2m in the resulting coefficients of A(z).
This indeterminacy may be resolved by using the fact that the time
series is real and by forcing all zeros of A(z) to lie on or within the
unit circle. The net effort, however, has been sufficient to dis-
courage use of moving average models of very high order. It is a
useful observation to note that the autocorrelation function of a
series generated by a moving average model is zero for lags greater
than m.
Estimation of parameters for an autoregressive model is more
straightforward. The problem is to obtain a least squares fit for
the coefficients b. in the relationi
n
L b y(k-i) =w(k)1-0
x
b=l, k = 1 , 2 , . . . N
subject to the restriction that the roots of B(z) should fall within
the unit circle. The results of such an approach are the consistent
set of linear equations [13 ] (bQ= 1), where the r(r) are values of
the observed correlation function of lag T.
45
r(0) r(l) -
r(l) r(0)
I
i
I
r(n-l)
r(n-l) Vb2
l
r(0) r(l)1
1
r(l) r(0).
.b
r(l)
r(n)
An alternate approach, a recursive approach, is also possible for
estimating autoregressive model parameters. Least squares estimation
problems with uncorrelated residuals can be conveniently handled
within the framework of the Kalman filter [10 ]. Such an approach
has been employed by Ho and Lee [16 ] to estimate the coefficients
of B(z). From the autoregressive model definition,
where
Let
y(k) =]
y(k-i) + <^y(k-2) +
, = -b , , o= ~b
2etC
T= (
j_ 2» • • • n )
y, = (y(k-l) y(k-2) .... y(k-n))—
k
ry(k-n) + v(k)
Then
y(k) =£, + v(k)k^
For the corresponding Kalman filter, the required quantities are:
TT q r x =
3 =1
X = z(k) = y(k)
H = h=y_k
R = r = E[v(k)2]
At a glance, it is clear that the computations required are simple.
Since the filter output is a scalar, no matrix inversion is required.
Further, gain computations are simplified here since P(k/k-l) =
P(k-l/k-l). It may first appear that ignorance of r will present
difficulty. However, normalizing the weighting computations with
46
respect to r and employing the suboptimal but practically useful
device [21 ] of a very large P(l/0) avoids the problem.
The recursive approach is quite flexible. If it is desired to
estimate the parameters of the best autoregressive model for the
observed time series, each sample in turn is processed. On the other
hand, if it is required to estimate the denominator coefficients of a
linear filter having a numerator of order m, then processing. only
every m+1 samples guarantees uncorrelated residuals (recalling
that correlation in a moving average model is nonzero only over m
lags) . Then the numerator coefficients may be obtained by pro-
cessing the residuals left after treating the observations with the
denominator in an autoregressive fashion.
The approach of Ho and Lee has been used in experiments to
estimate the model coefficients for an autoregressive representation
of a random signal. Results are given in the next section.
IV. EXAMPLES OF MODEL DETERMINATION
Various facets of the discussion above may be made clearer by
an illustrative example. Recursive parameter estimation was employed
to obtain an autoregressive model for a contrived laboratory signal.
The model to be found was arbitrarily chosen to be of sixth order.
The contrived signal consisted of the additive combination of
a 50 cycle sine wave and the output of a laboratory random noise
generator. Originally generated in analog form, the signal was
sampled by an analog-to-digital converter under control of a CDC
160 computer, and the digital samples were stored on magnetic
tape in 4000-sample blocks. Since initial analysis disclosed little
noise energy above approximately 400cps, the sampling interval was
effectively increased from the original 400 jas to 1200 jis by using
only every third sample, giving a Nyquist frequency of 416 cps . Of
course, the resulting sample size used in obtaining the model was
*The very useful programs used to accomplish the digitizing/
recording operation and necessary subsequent unpacking are due to
N. Barrett [2 ].
47
reduced to a third of the original 4000 samples.
Two runs were made, using signal-to-noise ratios of approx-
imately db and -10 db. Results are displayed in Figs. 3-1 to
3-6. Plots of z-plane poles of the resulting models are given in
Figs. 3-1 and 3-4. The remaining figures for each run contain com-
parative plots of autocorrelation functions and power spectral den-
sities derived from the signal and model. A visual indication of
model efficiency is provided by the comparative power spectral
density plot.
48
209
347
416
Model Poles - 50 cps Sine Wave in Random Noise
db Signal/Noise Ratio
Fig. 3-1
49
Autocorrelation Functions Computed from Data and Model
S/N = db
Fig. 3-2
50
<5>
I—»-
<5>
00
OT
V -SCALE * Q.00E-63 UNITS/INCH.
X "SCALE - 2.O0E+02 UNIT3'INCH.
Power Spectral Densities Computed from Data and Model
S/N = db
Fig. 3-3
51
209
4/6
ZlB^^
a
^^\/39
347/
^s.
/ >/o
\?°
/X
\ ^^fs' o
a \ j
Fig. 3-4 Model Poles - 50 cps Sine Wave in Random Noise -
-10 db Signal/Noise Ratio
52
X-SCflLE - 3.88E+81 LWTVINCH.Y -SCALE - WE-ei IWT^INCH.
Fig. 3-5 Autocorrelation Functions Computed from Data andModel - S/N = -10 db
53
Fig. 3-6 Power Spectral Densities Computed from Dataand Model - S/N = -10 db
54
CHAPTER IV
DETECTION OF RANDOM SIGNALS IN NOISE
The problems of detecting both deterministic^and random signals
in the presence of additive Gaussian noise have received considerable
attention in the literature [14 ], [22 ], [23 ], [34 ], [37 ]. It is
well known that for the former case, the optimum receiver is a cross
correlatbFii or matched filter, which correlates the input with a replica
of the transmitted signal.
Random signals considered previously, and to be considered here,
have been assumed to be of the somewhat restrictive but tractable
Gaussian form with zero mean and known autocorrelation . (This
assumption will be discussed in greater detail below.) Kailath [18 ]
has shown that the optimum receiver for such Gaussian signals in
Gaussian noise forms a cross correlation of the input and a least
squares estimate of the signal based on fcnput received in the inter-
val under scrutiny.
The results obtained in [18 ] suffer practical drawbacks, however.
Critical computational difficulties are faced in obtaining the least
squares estimate when the number of samples considered is large,
as it is required to invert a matrix of the same dimension as the
number of samples. Moreover, statistical descriptions of possible
signals, which were assumed known, are in practice not easy to
obtain.
The present work approaches this problem in a manner somewhat
similar to that of Weaver [35 ], and employs the results of Schweppe
[31 ]. The serious computational difficulties noted above are
avoided. Specifically, estimation techniques developed in recent
years by Kalman and others [19 ], [20 ], [26 ], [27 ] are employed.
Furthermore, in view of the similarities between correlation signal
In the discussion to follow, a deterministic signal will be
defined as one whose form is completely known at the receiver, anda random signal will be one whose description is known at the
receiver only in statistical terms.
55
detection and linear discrimination methods in pattern recognition, it
is felt that the approach to be followed here may constitute a new and
more powerful approach to certain pattern recognition problems .
For simpler and clearer presentation of results, the analysis to
follow will treat the discrete, or sampled, situation. However, the
techniques to be employed are equally applicable to continuous pro-
cessing .
I. OUTLINE OF THE PROBLEM
The problem to be considered is the following: a signal con-
sisting of a finite number N of samples of a Gaussian process
y(t) is received during a limited time interval T in the presence of
additive Gaussian noise. The additive noise and signal process are
assumed independent. In practice, real time processing is necessary,
and it is required to announce at the end of the interval whether or
not the signal y(t) has occurred. It will be seen later that the present
limitation to a single signal is not restrictive. The same approach
applies to the problem of detecting and discriminating members of a
finite set {ym (t)} of signals.
A word about the assumption of Gaussian signals is in order.
Aside from transmitted signals whose forms are simply not known at
the receiver except in terms of their first and second order statistics,
it happens that the scatter-multipath structure of ionospheric prop-
agation perturbs transmissions into Gaussian signals [27 ]. The
same situation is produced in radar and sonar by clutter and rever-
beration resulting from the effects of a multitude of small random
scatterers
.
II. CORRELATION DETECTION OF DETERMINISTIC SIGNALS
It will be convenient to first consider the problem of correlation
detection of deterministic signals in order to establish notation
conventions, more clearly outline the problem, etc.
From the problem statement above, the task at hand is clear.
The received signal, a mixture of the transmitted signal and additive
56
white noise, must be processed to yield an indication of which trans-
mitted signal in a finite set of possible signals {ym
(t)} was received.
It is, of course, desired that the indication be as reliable as possible
under the circumstances.
The essential feature of the problem is extraction of information
about the transmitted signal from the mixture of transmitted signal and
additive noise. Information concerning the transmitted signal is
clearly more germane to the issue than is determination of signal-to-
noise ratios [37 ], though the information sought and signal-to-noise
ratio may often be monotonically related. Possessing as much infor-
mation about the transmitted signal as it is possible to isolate from
the received signal available for processing, one may proceed with
the required decisions. The decision making will commonly also con-
sider other factors, such as relative penalties for incorrect decisions,
any a priori information, etc.
As mentioned, only a mixture, here called z(t), of the trans-
mitted signal yi(t) and noise v(t) is available for processing, i.e.
z(t) = yi(t) + v(t). The noise v(t) is assumed to be white Gaussian
noise with mean zero and known variance r. Since the discrete case
is being considered here, what is available for processing is a se-
quence of samples z(k) of z(t), where z(k) = yx (k) + v(k).
At this point, it will simplify notation if we define vectors y1
and z , where
yi = yi(l), yi {2) yi(N)
z = z(l), z(2) z(N)
From the problem statement, it can be seen that what must be
performed is a multiple-alternative hypothesis test [22 ]. Stated
another way, as viewed by the receiver, y is a discrete random
variable having the possible values y 1i = 1, m. All information
at the receiver about the transmitted signal y is contained in the con-
ditional probabilities of y given the observations z, p(yVz ) [37 ].
It is these conditional probabilities which will be considered further.
57
Remaining aspects of the decision-making problem (relative costs
of errors, etc.) depend upon the specific situation.
The conditional probabilities p(yVz ) may be stated in terms of
given or easily determined quantities through use of Bayes Formula
[17],
p(yi/z ) = p(z/yi )p(yi
)
p(z)
On the right hand side, ply1) is the a priori probability of signal
y1 (assumed given). p(z) is not a function of the transmitted signal
yi and may be considered a normalizing factor. The remaining quantity,
pfe/y 1) is a function both of the transmitted and received signals, and
must be considered in detail.
Expanding the notation,
p(z/yi) = p[z(l), z(2) . . . .z(N)/yi(l), yi(2). . . .y (N)]
Due to whiteness of the additive noise, consecutive samples of
n(t) are uncorrelated . Hence
z(l)/yMl), yM2) . . . y1 (N) = z(i)/yi (l) ~ NLyMl), r]
z(2)/yi(D, yM2) . . .yMN) =z(2)/yi(2) ~ N[y i(2), r]
etc.
Furthermore, zOJ/y1^) and z(k)/y1
(k) are independent random
variables for all j ^k. Therefore,
p[z(l), z(2) . . ^(Nj/yMD, yM2) . . . yMN)]
= f] pEzO/y^)]j=l
In more compact form,
_N Np(z/yM = (2 nr) 2 exp - 1_ L [z(j) - y
1(j)]
Z
2r j= i
x N N N= C exp --LIE z(j) 2 + I yi(j) 2 - 2 I zOy^l
2rtj=l j=l j=l
J
58
Inside the brackets, the first term represents received signal energy,
a factor which will cancel in later comparisons of conditional prob-
abilities. It can therefore be incorporated into C, The second term
represents energy of the transmitted signal y1. If all the possible
transmitted signals have been normalized so as to possess the same
energy, this term can also be incorporated into C. Otherwise, it
may be carried along as an additional constant C1. The situation
becomes,N
p(z/yi) = CCi exp I £ zOyiQ)r j=l
It is seen that the optimum receiver, which furnishes the con-
ditional probabilities p(yi/z) to the decision making processor,
forms a cross correlation of the received signal and the transmitted
signal and uses the result in computing the desired conditional
probability. The resulting optimum receiver structure is shown in
Fig. 4-1.
The subject of correlation detection is far from exhausted. In
particular, the decision philosophy to be employed and details there-
of have been omitted. The purpose of this section has been to es-
tablish a frame of reference to be extended below in discussion of
random signal detection. Further discussion of the problem of
correlation detection, as well as references to the literature on this
subject, may be found in [22 ], [37 ].
While the discussion above has followed the decision-theo-
retically optimum Bayesian approach to signal detection, situations
exist where no meaningful values for the required a priori prob-
abilities can be obtained. It is customary in such situations to
employ the philosophy of Maximum Likelihood suggested by R. A.
Fisher. Evaluation of the Likelihood Function, defined as the same
conditional probability density function p(z/yx) considered above,
is required. The resulting quantities are weighted by relative costs
of misclassification and compared, yielding a decision that thak-eignal
59.
z(k)
ys(k)
y"'(k)
\ »
^>
IITZ
K» I
DECISION
MAKING
PROCESSOR
Fig. 4-1 Optimum Receiver for Deterministic Signals
60
y1 is present which corresponds to the maximum Likelihood Function
[8 ]. The essential point is that the Likelihood Function must be
evaluated in either approach. Hence one may approach the task with
relatively more confidence that efforts are well directed.
III. RANDOM SIGNAL DETECTION
In this section, the ideas developed above will be extended to
handle detection of Gaussian signals in white Gaussian noise. First
the approach of Kailath [18 ] will be discussed and its computational
difficulties noted. Then application of recently developed filtering
and smoothing techniques to this problem will be considered.
Here the signal is assumed to comprise a sequence of N samples
from a Gaussian process whose mean value function and covariance
function are known. Hence, each signal vector y1 possesses a
multivariate normal distribution with mean vector \x= and co-
variance matrix K x.
y
K i = Efriy T) = (R 1
) i, j = 1, 2 . . . . Ny ij
Just as in detection of deterministic signals above, the problem re-
duces to evaluation of the Likelihood Function p(z/y1). Since the
signal and additive noise are assumed statistically independent, the
received signal covariance is
K i = K i + R where R = rl
z y
The Likelihood Function is
p(z/yi ) = (2 77)- f IK1 \~\ exp -1 [z T
(K1
)
_1 z]z z l 2 z
It may be expanded using a matrix inversion lemma arising from the
Frobenius-Schur Relation [7 ]. (Since only a single signal is being
considered for the moment, notation will be simplified by omitting
the superscript temporarily.)
K_1 =R_1 - R
_1(R
_1 +K' 1J"
1 R" 1
z y
61
Thus,
p(z/y) = C exp - I [ z TR_1
z - zT [R_1
(FT1 + K
~ l)~ l R"T] zj
The first term in the exponent does not involve the signal, hence will
cancel in the decision process, and may be carried as a constant C,
leaving
p(z/y) = CC^expI fz T [FT 1(R_1
+ Ky-1
)
_1 R_1]z]
Now
(R^ + K" 1 )" 1 !*" 1= (I + RK
_1)
_1x
y y
= Ky
(Ky+ R)-1
= K K_1 = A
y z
It is easily shown [18 ], and included below, that A is the
weighting matrix yielding the least mean squared error estimate vec-
tor y of y given the observation vector z.
What is required is the matrix A such that if
y = Az
the mean squared error between y and y will be minimized . Error
in estimating the i the component of y is
Ne. = y. - L a. . z .
J=l
Hence,N N N
ef=y\-2L3ij
z. y. + I L a a.k
z.zk
j= 1 j=W
N N N-R (0)-2La R (i,j) + l L a a^ R
Z2 «,k)j=l j=l k=l
For a minimum,
=0 j = 1, . . . . Noa
ij
62
ThusN
Ryy (i/j) = I aik R zz (j,k)
k=l
(Note that R ( t) = R ( r).) In matrix form,y*s yy
K =AK H> A = K K_1
y z -^ y z
Having computed A, the Likelihood Function to be determined takes
the form,
p(z/y) = CC 1exp ^ (z
TAz)
2r
= CC1exp
-J-(z
Ty)
2r*
It is seen that the optimum receiver for detecting Gaussian signals
forms a cross correlation of the received signal and a least-squares
estimate of the Gaussian signal given the received signal. As in the
case of deterministic signal detection.,: this cross correlation is used
to evaluate logarithms of the conditional probabilities pte/y1)
i = 1, . . . .m, which are in turn furnished to the decision making
processor. The resulting receiver structure is shown in Fig. 4-2.
Recalling that the covariance matrices K_, and K„ are ofy c
dimension NxN, where N is the number of samples to be processed,
the computational difficulties in obtaining A =K K~* are evident.
Using the fact that K is a Toeplitz matrix, Trench [33 ] has
developed an algorithm for obtaining the inverse requiring effort
proportional only to the square of the matrix dimension, vice its
cube. Thus the problem is reduced in magnitude, but remains quite
a task for even moderately large sample sizes, say 200. Unfortunately,
an even worse problem must still be faced. In the present case of
random signals, Kz
is a function of K* . The quantity which was
carried as a constant C (since it would later cancel) in the Like-
lihood Functions of deterministic signals, must now be evaluated.
Evaluation of the determincint of K is involved. Even with exploitation
63
z(k)
^-bz>
21^-bo-*
L
z
\
y™(k)mA
N
Ms/
Fig. 4-2 Optimum Receiver for Gaussian Signals
64
of the Toeplitz form of K , the problem remains a considerable one.
The signal above was assumed to consist of a sequence of
samples from a Gaussian random process with known mean (zero) and
known covariance function. Description of the Gaussian process in
an equivalent but different manner will permit solving the problem
while avoiding the difficulties above. As discussed in Chapter III,
the Gaussian process may be represented to any desired degree of
approximation as the output of a linear dynamical system excited by
white Gaussian noise.
Statistical description of the Gaussian message process will
then be in terms of the mean (zero) and covariance q of the excitation
noise, and in terms of matrices T, <$, and H defining the input-
output relation of the linear dynamical system. As before, this
statistical description of the message process is assumed known.
An alternate approach to the random signal detection problem may
be pursued with the message process description above using the
results of Schweppe [31 ]. It involves developing a difference
equation for the Likelihood Function (or rather its logarithm) , from
which the complete Likelihood Function, not just its exponent, may
be determined recursively as signal samples are received. Both the
problems of inverting the NxN matrix Kz and evaluating its
determinant are avoided.
Let
L(yM = ptz/y1)
Then
-2 In LlvT(y
i )=ln (2tt)N
|k I + zTK-1
zN ' z ' z
Now,
Lk (y
1) = p [z(l) z(k)/y l
]
= p [z(l) . . . .zfr-D/y 1 ]p [z(k)/z(l) . . .zfc-lhy 1]
65
Hence,
lnL^y 1) - In 3^ (y
1)
= In p [z(k)/z(l) . . . z(k-l), y *]
The conditional probability density function
p[z(k)/z(l) z(k-l), y1
]
must be considered further. Under the hypothesis that the transmitted
signal is y 1, the matrices describing its statistical characteristics
are r 1, & 1
, and H 1. A Kalman filter using these matrices and oper-
ating on the received signal data computes values x(k/k-l) and
P(k/k-l) which define the conditional distribution
p(x(k)/z(l) z(k-l), y1
)
= C exp -I[(x(k) - x(k/k-l)) T P(k/k-l)_1
(x(k)-x(k/k-l))]2
From this, it follows simply that
p[z(k)/z(l) z(k-l), y1
]
= (2 7r)_2 [HP(k/k-l)H T+ r] ^ exp - [z(k) - z(k/k-l)3 2
2(HP(k/k-l)H T+ r)
Defining
~(k) = z(k) - z(k/k-l)
yields
-2 lnp[z(k)/z(l) z(k-l), y i]
= In 2 7T[HP(k/k-l)H T + r ] + z"(k)2
[HP(k/k-l)HT+ r ]
A difference equation in -2 In L (y1 ).has now been determined.
The original requirement was to evaluate L^ (yi), or, equivalently,
-2 In L^j (y1) . Hence, the differences are simply summed, k = 1,
2, . . . N. The result is
-2 In L (y1
) = In (2 ff)N
I K I + zTK_1
zN ' z ' z
66
N N= £ In 2 ff[HP(k/k-l)HT + r ]+ £ ?(k) 2
k=l k=l [HP(k/k-l)HT + r]
It can be seen that -2 In L^y1) has been built up from sums of scalar
quantities. Required quantities z(k) and [HP(k/k-l)H + r] are
already available in the Kalman filter. The net result is that the great
practical difficulties of inverting and evaluating the determinant of a
large matrix have been avoided by an approach requiring no matrix in-
version at all. The structure of the optimum receiver using this
approach is shown in Fig. 4-3.
On-line computations can be greatly reduced by considering use
of the Likelihood Function above and some details of filter behavior.
First of all, since Likelihood Functions for the various alternativesN
are to be compared, the term y In 2 if will cancel and hence may
k=l
be discarded. Further, it frequently happens that the quantity
[HP(k/k-l)H™ + r] settles within a few time increments to its steady
state value. In such cases, all necessary values of filter gain vec-
tors B(k) and [HP(k/k-l)H^+ r] may be computed beforehand and
stored with modest memory requirements. This represents quite a
saving since the greater part of computation required by the Kalman
filter asi associated with updating the variance equation and computing
gains . On-line computations are thus reduced to updating the state
vector x(k/k) with new observations and adding in the new terms to
the Likelihood Function evaluation. From the filter equations listed
in Chapter II, this may be seen to amount to approximately 2n + 2
multiply-add combinations per received sample for each classification
alternative, where n is the order of the model.
67
ZOcJSCO ^H|)[HP(k/k-i)H
r+r]
/»[HP(k/k-imr]
2ft)^^Q^)[HPW-itfrJ
A [HP(K/K-i)Hr+r]
*\ M Z) *
Jc-I
o<otoUjVJo&
^icX5
^*>-
) -. t/ *Ml
aQ
Fig. 4-3 Optimum Receiver Using Recursive Likelihood
Function Evaluation
68
CHAPTER V
AN EXPERIMENTAL STUDY IN RANDOM SIGNAL DETECTION
Finally, utility of theoretical developments in engineering appli-
cations rests upon ability of the developed techniques to perform
successfully inthe "real world". Sensitivity of performance to dis-
crepancies in actual and assumed parameters must be low enough to
permit satisfactory operation in the face of such discrepancies. In
terms of the detection/classification scheme discussed in Chapter IV,
employing recursive evaluation of Likelihood Functions, some points
to be investigated are:
(i) Sensitivity to variations in the model quality as measured
by the ratio of "correlated" to "uncorrelated" components
of the received signal,
(ii) Sensitivity of detection performance to discrepancies be-
tween actual and assumed values of received signal power,
(iii) Rapidity with which the technique delivers a clearcut
classification decision.
In this chapter, the above points are discussed in the context of an
example problem using actual signals.
I. DESCRIPTION OF THE PROBLEM
In the following example, the two-alternative or detection problem
is studied. Signals used are hydrophone recordings of (a) sea noise
plus sounds of a diesel submarine, and (b) sea noise alone. In dis-
cussions below, (a) is referred to as "signal" and (b) as "noise". The
resulting problem is as follows: given a segment of received signal
record, produce a Maximum Likelihood judgement as to whether the
received signal is submarine or noise alone.
Signals used here were originally in analog form on magnetic
tape, and were digitized for use in computation using programs de-
scribed in connection with the example in Chapter III. The basic
sampling interval was 200 jxs. However, as initial analysis disclosed
little signal or noise energy above 800 cycles or so, only every third
69
sample was employed in computations, extending the sampling interval
to 600 Jis. The total effective number of samples per block was thus
reduced to 1333. Figs. 5-1 and 5-2 display plots of IBO samples of
signal and noise, respectively. Fig. 5-3 contains a comparative plot
of sample autocorrelation functions of the signal and noise.
For study purposes, ten 4000-sample blocks each of signal and
noise were digitized. The first five blocks of each group were used
for obtaining models. Testing of the detection procedure was then
done using the remaining five blocks in each group.
II. MODEL DETERMINATION
For ease in determination, the autoregressive form of model was
selected for the signal and noise sources. . Order of the models was
chosen to be six, since the signal was known a priori to have three
prominent "lines" at 240, 390 and 625 cps. For less or no a priori
knowledge about the signal structure, a procedure might be to begin
with low order models and successively increase the order until the
comparative power spectral density plots indicate satisfactory rep-
resentation. For the present problem, recursive estimation of model
coefficients as in the example of Chapter III was performed. Resulting
models were found to be quite consistent from block to block for the
signal. Noise models had consistent dominant poles, but somewhat
scattered secondary poles. Figure 5-4 shows the z-plane pole lo-
cations for signal models from the first five blocks of signal data.
Figure 5-5 shows the same information for noise models.
Selection of a specific model for use in detection might be made
by, say, averaging the corresponding coefficients of the five sample-
derived models for each case. A simpler but less precise method
would be to just select a representative model from each group. The
latter method was used for the experiments here. Figures 5-6 and
5-9 contain z-plane pole plots for the selected signal and noise models
respectively. Figures 5-7 and 5-8 display comparative plots of auto-
correlation functions and power spectral densities derived from the
70
K-3CALE - 1.88E-+81 UN1T3/JNCH.
Y-3CALE - 2.aeE4ee unjt3/jnch
S+N
Fig. 5-1 Sea Noise Plus Diesel Submarine ("Signal")
X -SCALE - 1.8GE+81 UNJTS/JNCH.
y -scale - ^.aeE+ee units/inch.
Fig. 5-2 Sea Noise ("Noise")
71
Pig. 5-3 Autocorrelation Function of "Signal" and "Noise"
72
4/7
55*^*' 1 ( ^"^v.278
695 /'f >
7W N\/39
833
\ / \ v» y
!
PLOT BLOCK
1 1
z Z
3 3
4 45 5
Fig. 5-4 Z-Plane Poles of Autoregressive Signal Models
73
4/7
833
55t^t
^78
495/Cp$1 \
1
3
3 /*« y
* 4 / \/39
1 \
n%* / 3
2
54 \
r \
PLOT BLOCK
I 13
2 14
3 IS
4 it
5 \i
Fig. 5-5 Z-Plane Poles of Autoregressive Noise Models
74
Fig. 5-6 Poles of Representative Signal Model
75
Si
-
-
.
.' i
1 . :'
i
i
.
\
|
1
i
1 A
/\ i \
//\ \
»e \ / 005\ |
\ //
I II
, \ /» /\ I
1/W
aie // \\ 915/ U T \\ //
RS
Fig. 5-7 Autocorrelation Functions from Signal Dataand Signal Model
76
cv
<s V- SCALE - 5.00E-03 UNITS'INCH.
1
X-5CAL;E - 2.00E+02 UMIT5/IMCH.
csl-Oen
i
1
NOs
i
\
CJ
en
•
|,
c>>l
C9
en
A j000 _J\002 \/x/ i 30H. ^>£^=* MB
Frequency
Fig. 5-8 Power Spectral Densities from Signal Data andSignal Model
77
; j±&.
Fig. 5-9 Poles of Representative Noise Model
78
05
&§
K-3CAL
Y-SCfll
E - 6.08E+a0
.E - 2.00E-81
UNITS'INCH
UNITS/IKH
. i
!. .
i
\
I \ A Ai00 W/ ^^s v&\y \
4
R3
Fig. 5-10 Autocorrelation Functions from Noise Data andNoise Model
79
Frequency
Fig. 5-11 Power Spectral Densities from Noise Dataand Noise Model
80
data sample and model respectively. Figures 5-10 and 5-11 convey
the same information about the noise model.
III. STUDY OF PERFORMANCE
For an honest evaluation of discrimination power of the technique
heing tested, it is important that signal classification be made solely
on the basis of differences in correlation structure of the alternate
possibilities and not on energy levels. For this reason, each batch of
4000 samples from which sub-blocks of 100 samples were processed by
the detector was normalized to have a mean squared sample value of
unity
.
In terms of the discrete signal process model pictured in Fig. 2fc2,
this means that E[z 2] = 1. But since
E[z 2 ]=E[y 2 ]+r /
it remains to be determined what is the most likely ratio of E[y 2] to
r. The ratio is a measure of the degree to which the model obtained
represents the actual observed signal.
Using a value of .9 for this ratio, the Likelihood Function for
each alternative is computed as a function of the number of samples
processed and is plotted in Fig. 5-12. Since the quantity -2 In
Likelihood Function is actually computed and displayed, higher like-
lihoods are represented by lower ordinates on the graph. In Fig. 5-12
samples processed were actually "signal". Hence correct classification
is indicated by the graph.
Program QTEST then evaluates the Likelihood Function for each
classification alternative for values of this ratio from .1 to .9.
Figures 5-13 to 5-16 display the resulting values of the Likelihood
Function versus the number of data samples processed. Note that
correct classification was achieved for all values of the ratio between
.1 and .9. It might also be noted that the ratio value of .9 gave the
greatest margin of correct classification. Correct classification was
also consistently obtained in the same manner for all blocks of in-
dependent data (not utilized in model determination).
81
Fig. 5-12 Likelihood Functions for Signal Data ComputedUsing Signal and Noise Models
82
Fig. 5-13 Likelihood Functions of Signal Using Signal Model
83
Fig. 5-14 Likelihood Functions of Signal Using Noise Model
84
J
1
• K -SCALE -
Y-5CflLE -
\
• 2.88E+ei UN]
2.00E+01 UN.
1
TS'INCH.
TS'INCH.
1
1
1
i j
\
'
.-
1
^^iT3"!
R 9GQ m 966 dee
Fig. 5-15 Likelihood Functions of Noise Using Signal Model
85
eg
K -SCALE -
Y -SCALE '
1' 2.88E+81 UNJ
• 2.00E+01 UN
TS'INCH.
ITS'INCH.
a
\ M-
i
Jf"
CJ
9
iaw mi 086 888
.1
.2
.3
.4
5
?
8
9
Fig. 5-16 Likelihood Functions of Noise Using Noise Model
86
The normalization process described above is roughly equivalent,
in practical terms, to automatic gain control (AGC) . It is important
that the detection scheme continue to function efficiently under de-
viations between actual and assumed amplitudes (and corresponding
mean square values). Hence a test was undertaken to check classi-
fication efficiency against signals with actual mean square values
ranging from one fourth to four times the assumed value. Again, con-
sistent correct classification was maintained. Figures 5-17 to 5-24 t
contain the resulting Likelihood Function plots for actual mean square
values equal to one fourth and four times the assumed value, in all
cases equal to unity.
87
Fig. 5-17 Likelihood Functions of Signal Using Signal ModelMean Square Signal Received Equal One-fourth
Assumed Value
88
Fig. 5-18 Likelihood Function of Signal Using Noise ModelMean Square Signal Received Equal One-fourthAssumed Value
89
Fig. 5-19 Likelihood Function of Noise Using Signal ModelMean Square Signal Received Equal One-fourthAssumed Value
90
Fig. 5-20 Likelihood Function of Noise Using Noise Model -
Mean Square Signal Received Equal One-fourthAssumed Value
91
Fig. 5-21 Likelihood Function of Signal Using Signal ModelMean Square Signal Received Equal Four TimesAssumed Value
92
Fig. 5-22 Likelihood Function of Signal Using Noise ModelMean Square Signal Received Equal Four TimesAssumed Value
93
Fig. 5-23 Likelihood Function of Noise Using Signal ModelMean Square Signal Received Equal Four TimesAssumed Value
94
Fig. 5-24 Likelihood Function of Noise Using Noise ModelMean Square Signal Received Equal Four TimesAssumed Value
95
CHAPTER VI
RECOMMENDATIONS FOR FURTHER WORK
A frequent happening in applied research is that there are as many
new questions raised as old questions resolved. Such has been the
case in this instance.
Perhaps the problem most ripe for attack is that of estimating num-
erator coefficients of the linear filter model for random time series
.
This is equivalent to estimation of the moving average coefficients in
a mixed auto-regressive-moving average model. A promising approach
might lie in numerical processing of the residuals after recursive de-
termination of the autoregressive coefficients.
Further study of applications of the methods used here to practical
problems in signal detection/classification would also be interesting
and potentially valuable . In particular, multiple-alternative signal
classification should be interesting to consider. Performance in such
problems should be improved by better models resulting from successful
determination of linear filter model numerator coefficients
.
Reliability of detection/classification as a function of record
length processed and as a function of some measure of difference be-
tween power spectral densities of alternate classifications should be
valuable to investigate. For the detection problem, differences between
power spectral densities of signal plus noise and noise alone relate to
signal-to-noise ratio in the conventional problem formulation.
Classification methods used herein also hold potential contri-
butions it character recognition. Required for application of the
methods here are temporally vice spatially distributed patterns. How-
ever, the scanning process used in several present approaches to
character recognition already produces thffnecessary spatial-temporal
transformation. The advantage to be gained should be an insensitivity
to character translation, a problem which plagues many present
approaches
.
Finally, the problem of model determination for vector-valued
random time series remains. Some work on the problem has been done,
96
and is described in [3 ], [30 ]. A comparative study of work referenced
and the general linear filter model should hold some promise of bene-
fiting the general analysis of vector-valued random time series both
in engineering and other disciplines
.
97
APPENDIX I
OPTIMAL ESTIMATION OF AN ARBITRARY LINEAR COMBINATION
OF SIGNAL GENERATING PROCESS STATE VARIABLES
Assume the usual model of the signal generating process with
state variables x(k) and noisy observations z(k) = Hx(k) + u(k). It
is desired to estimate an arbitrary linear combination of the state
variables s = Dx based on the noisy observations z. The optimal
linear estimate (in the Bayes sense, with a quadratic loss function)
can be shown [17 ], [35 ] to be the expected value of the conditional
distribution
p[s(k)/z(k), z(k-l), .... z(l)]
Since the process is linear, and since the excitation and noise
sources are gaussian, the conditional distributions of both x and s
given measurements z are gaussian. The problem at hand will be
solved if the conditional distribution of s given z can be obtained,
from which the mean may be extracted.
Consider first the conditional distribution of x given the
measurements z. By Bayes rule,
p[x(k)/z(k), z(k-l), . . . z(l)]
= p[z(k)/x(k),z(k-l), . . z(l)fo [x(k)/z(k-l), . . z(l)]
p[z(k)/z(k-l), . . z(l)]
As pointed out, the form of the left hand side is known, and is:
p[x(k)/z(k),z(k-l), . . . z(l)]
= C exp -| [(x(k) - x(k/k))T
P(k/k)-1
(x(k) - x(k/k))]
On the right hand side,
p[z(k)/x(k),z(k-l), . . .z(l)]=p[z(k)/x(k)]
= C exp -|[(z(k) - Hx(k))T R" 1 (z(k) - Hx(k))]
p[x(k)/z(k-l), . . . z(l)]
= C exp -|[(x(k) - 0x(k-l/k-l)) T P(k/k-l)_1
(x(k) - 0x(k-l/k-l))]
98
p[z(k)/z(k-l), . . . z(l)]
= C exp -|[(z(k) - H 0x(k-l/k-l))T (HP(k/k-l)HT+ R)"
1
(z(k) - H 0x(k-l/k-l))]
Defining parameters of the left hand side distribution may be obtained
from the right hand side by matching terms [26 ] . The Kalman filter
equations are obtained, which define the left hand side distribution
parameters recursively, and one has
x(k)/z(k),z(k-l), .... z(l) ~ N[x(k/k), P(k/k)]
in terms of known quantities.
In Anderson [l ] , it is proved that
x ~ N[m , L] => s = Dx ~ N[D fl , D ED T]
The theorem includes the cases where x may have either a non-
singular or singular distribution and D may be nonsingular or of
rank less than the dimension of s.
Invoking the theorem, the optimal estimate of s is seen to be
Dx(k/k). Hence the optimal estimate of a linear combination of the
states is the same linear combination of the optimal estimate of the
states
.
99
APPENDIX II
THE KALMAN FILTER AND ESTIMATION OF A GAUSSIAN MEAN
The following discussion outlines an interpretation of the Kalman
filter (discrete casej^in which it is considered to be a generalized es-
timation of a Gaussian population mean. The term "generalized" is
used to mean that, contrary to usual assumptions in estimation of
the mean, the population to be estimated here will be permitted to
move between sampling instants. Movement will be assumed to
consist of both a prescribed component and a random component.
The analogy to be described permits achieving an intuitive feel
for rates of convergence, effects of uncertainty in iiaatial conditions,
and effects of random signal process excitation. Such an intuitive
feel for filter behavior is frequently difficult to obtain from conven-
tional formulation of the estimation problem
.
The univariate case will be considered first, for simplicity. The
same analogy, however, carries over to the multivariate case, and
will be considered intfrief detail later.
"No Dynamics"Situation
To begin with, the classical mean estimation procedure will be
written in recursive form. Consider a scalar random variable Z
where
Z ~ N(x, r) x = unknown population mean
r = known population variance
If Z is the sample mean of a sample of size n from the population,
then
Z ~ N(x, p) where p = —n
Since the sample mean is the best (minimum variance) estimate of
the mean of a Gaussian population [17 ] , x = Z.
Let z(l), z(2), . . . . z(k) denote sample values of the random
variable Z.
100
Then
x(k) = ±L z(i) = x(k-l) + I [z(k) - x(k-l)]KM K
p(k)= £iiL =(1-1 )p(k-l) k = 2, 3, .
k k
4 1If b(k) S i. then
k
x(k) = x(k-l) + b(k) [z(k) - x(k-l)]
p(k) = [1 - b(k)]p(k-l) k = 2, 3
It will be useful to note that as each new sample value is con-
sidered, the weightings given it and the previous best estimate are
in inverse proportion to their respective variances. For example, at
the k thi stage above,
£(k) = k^Ix^k-l) + I z(k)k k
Var x(k-l) = —I
—
k-1
Var z (k) = r
Var 5(k) = J_k k-1
r +- r
k-1
Weighting given to sample value = — = ,
k 1r k
r_
Weighting given to previous best estimate = k = k-1
k-1
Ah alternate view is to note that weightings given each sample
value and the previous best estimate areiproportional to their relative
information content, the reciprocal of their relative uncertainties.
From this point of view, the information content of a current best
estimate (the reciprocal of its variance) is the sum of the information
contents of the sample and previous estimate.
Next, the behavior of a Kalman filter of the following description
will be compared with that of the sequential estimation procedure above
101
The restrictions that the transition matrix (a scalar here) be unity
and that there lie no signal process excitation will be relaxed later.
x = scalar H = h = 1
$=0=1 R = r
Q=q=0 P=pThe four equations describing filter behavior are repeated below for
reference as desired.
P(k/k-l) = $P(k-l/k-l) 4 T+ Q
B(k) = P(k/k-l)H T [HP(k/k-l)H T + R]" 1
x(k/k) = <|x(k-l/k-l) + B(k)[z(k) - <*x(k-l/k-l)]
P(k/k) = [I - B(k)H]P(k/k-l)
Three cases will be discussed, outlining filter behavior under three
different levels of uncertainty in itifeial estimates (a priori input to
the filter). Single subscripts will be used in the present section
since there is no ambiguity under the conditions assumed, i.e.
p(k/k) =p(k/k-l).
Case 1 . Great uncertainty in initial estimate of x. Let
p(0) = ar where a = large number
b(l) = ph(hph T + r)_1 =_§£_= § m l
(a+l)r a+1
p(l) = (1 - b(l)h)p(0) = (1-_^_)ar = § M r
a+1 a+1
b(2) «-I— = i b(k) = I2r
—*S k—-^
P(2) M (1 -i)r-JL P (k)=-i-2 k
Case 2 . Uncertainty in initial estimate of x approximately
equal to uncertainty in the observation process, i.e. p(0) = r.
b(l)=-r^= h -^ b(k) =I=> ' k+1
= r ^i^\ ^ rp(l) = [1 -b(l)] P (0) -JL- p(k)k+1
102
Case 3 , Uncertainty in initial estimate of x less than uncer-
tainty in observation process.
Let p(0) = —— where I is some positive integer.
I 1
p(l) = (l -b(l))p(0) = (1 --f--)l+l I l+l £ l+l
-1 = 1b(2)=^L-(_I_ +r)l+l l+l l+l
p(2) = (l -b(2))p(l) = (1 --A- )-I
r+ii r
i+i I
1*4 1
2+4 l+l
__ r
2+1 l+l l+l
b(k) = _L_k+l
P(k) =_ r
k+l
In the Kalman filter for the particular process just described,
effects of varying degrees of uncertainty in the initial estimate are
isolated and clearly observed. It can be seen that for great uncer-
tainty in the initial estimate, the filter behaves asymptotically as the
sequential estimation procedure outlined, i.e. p(k) and b(k)
(k = 2, 3, . . . n) decrease from r and 1 respectively as — .
k
Eor uncertainty in the initial estimate equal to or less than uncertainty
imposed by the observation process such that p(0) = -r r ( l-
positive integer) , then the filter data weighting and convergence rate
are the same as the data weighting and convergence rate of the
sampling estimation scheme from its (^-l)th sample. In other words,
a priori information may be regarded as providing equivalent additional
observations of the process to be estimated.
It is thus clearly shown that under the conditions stated, p and
b decrease asymptotically as i-. Interpolating between the assumedk
rational values of the ratio —-— yields a gain schedule b(k) =p(0) k+a
(a = —-— ) for the filter in the simple situation being considered.p(0)
103
Unexcited Dynamic Situation
Next, a mpre general univariate situation will be considered. In
the Kalman filter representation, the restriction above that 0=1will be relaxed. The corresponding situation in the sequential estimation
procedure will be that during the sampling process, the population mean
moves about in a prescribed fashion, i.e. x(k+l) = 0x(k). As pointed
out above, the sequential estimation procedure forms a weighted sum
of the previous best estimate and the new data (sample value) , and
specifies a new estimate variance equal to the "parallel combination"
of the sample and previous optimal estimate variances.
The essential difference between the present case with relaxed
restrictions on and the case previously discussed is simply that
just as the population mean is moved between samples, the previous
best estimate must be adjusted by prior to adding in effects of the
new sample value.
A double-subscript notation will be helpful here. Let x(k/k-l) =
The optimal estimate of x at sample time k given samples up to
sample time k-1
.
Since x(k) = 0x(k-l) , x(k/k-l) = 0x(k-l/k-l)
and since x(k-l/k-l) ~N(x(k-l), p(k-l/k-l)).
x(k/k-l) ~N(x(k), 2p(k-l/k-l)).
Hence,
p (k/k )= r0 2 p(k-l/k-l) = rp(k/k-l)
r + s p(k-l/k-l) r + p(k/k-l)
= b(k)r
The resulting gains with which x(k/k-l) and z(k) are added are
pfcA? and pfrA)p(k/k-l)
andr
'
No additional discussion of the Kalman filter is required for this
situation, where is not constrained to be unity. If the filter a
priori variance p(0) equals p(k/k-l) of the sequential estimation
procedure for any k, the filter will behave from the beginning just
104
as the estimation scheme does from the k th stage on.
Randomly Excited Dynamics
The situation may be made still more general by relaxing the
restriction in A that q = 0. In the sequential estimation procedure,
the interpretation is that not only will the population mean move in a
prescribed fashion between sampling instants, but will have an
additional random motion imparted to it by a gaussian excitation in-
dependent from sample to sample.
Again, no additional discussion of the Kalman filter is required.
Further, treatment of theipresent situation in the sequential estimation
procedure is a direct extension from B.
The first few steps are
P(D =r
b(l) = 1
p(2/l) = 3r + q
p(2/2) = (03r + q)r
(02 + l)r +q
b(2) = p(2/2) = 3r + q
r (02 + l)r + q
etc.
The steps in general are
p(k/k-l) = 3 p(k-l/k-l) + q
p(k/k) = p(k/k-l)r
p(k/k-l) +r
b(k) = p(k/k-l)
p(k/k-l) + r
The general updating steps agree with those of the Kalman filter.
Again, if the filter a priori variance p(0) equals p(k/k-l) of the
sequential estimation procedure for some k, the same remark as
in B is valid. For the present situation, p does not continue in-
definitely to decrease, but reaches a non-zero steady state value.
This steady state value of p and the corresponding steady state
gain may be computed by setting p(k+l/k+l) = p(k/k) and solving.
105
The Multivariate Case
Discussion of the multivariate case will be limited to situations
where H = I. Though this excludes cases of major interest where
the Kalman filter has been employed, the additional detail required
would obscure the comparison sought here.
Under the frequent assumptions of diagonal P(0) and R matrices,
the case where $ = I and Q = (no dynamics) really amounts to n
uncoupled problems of the type covered above, and is of little
additional interest. Ho has shown [15 ] that P decreases asymp-
totically as -I for this case without the assumptiin of a diagonalk
P(0).
The randomly excited dynamic situation is of more general
interest and is a direct extension of the approach in the univariate
case. As before, if the filter a priori covariance matrix P(0) =
P(k/k-l) of the sequential estimation procedure for any k, its
effect may be regarded as supplying k equivalent additional ob-
servations to the filter. Adjustment of the optimal estimate covariance
between samples is:
P(k/k-l) = &P(k-l/k-l) 4 T+ Q.
Updating of the covariance matrix and computation of the new gain
is also a direct extension:
Univariate case: Multivariate case:
p(k/k) = p(k/k-l)r P(k/k) = P(k/k-l)(P(k/k-l) + R)_1
R
p(k/k-l) + r
b(k) = p(k/k) B(k) = P(k/k)R-1
r
A check with the filter equations above will disclose that this is
exactly the same updating sequence as in the Kalman filter.
106
1— Li 1
1
ato z a UJ > a Ul^r Ul a X <—
•
to to•—1 to O 1— (- Tl •—1
a 7?: _i UJ a D <7 in Oo 2: _j d: UJ a: u _) ID 2*
a z _J Z) •—1 a Ui Ul X O *(V < cc s UJ a Ul u >—
1
aU> 1—
1
•—1—
t
< QuX 3
Ul >O
_j <]'
UJ1—
O. ^^aUJ
a < tt: 2- \- 1— 3 •— *:. O |>"
o 5 uj Li I < »—
1
p _l t-4 ^* to b O t—
D — •—
t
z D 2' i— t—1 <r 2' V UJ U) u- <h- t— •—
4
>—
1
a_ *
—
* <s 1— _n i—< to lO _i— c<~i I/) > CJ _J «-r O < ^ 5 < toI— ~ «^ UJ z O a- _l 1— to Ui -< vO V *—
<
K cr: • u_ co •—
l
(.0 21 U to_l
^^,
«- XX *•
Zt—i 2T
a Ui Ul1—
u • aQ
<r <,
Oa
UJ
to <f m »-^ < O _i U' < 'Jj UJ 2" 'T X -~ a X^L c r- O 5; 1—
1
_j c 5 1— to >—
1
>--• < u r\j
i—
i
yQ •~ — (O O 1— •—
.
Z) — <:" 2
'
u Ul (>: ~l 2" <—
1
to r-H ^01
< ^ (— rr '-' 1—
1
QLU -1
tl) to toUJ
UJUJ X
to U o vO M LU —
«
i— I— LU 1— h- LL a _i a 1— H- — Cj <r»—
«
2! O f—
i
W • #- 2 A. ^-1 < to < a Ul O 0. z> <r ••—
.
•—
i
U X O — —
1
or _J to ii 1 >• 10 a ro Z U. a»—
»
— #* h- c \— U_l « H- a •-< »-^' to >—
•
>< UJ —•
.
• c 1— >- ?: —
<
C\ Ul Ul in a — # l- a < aX jr X vO — iTv U. UJ _i Uj UJ U' CD
~>-~ 1— X >- to X a•—
i
•—
<
y- «— vO ~ O a _i t— u 1— CI' X ^- i— U 1 X UJo 1— 1— — CL < U") C <r O u U 1 -1 •—1 <2* to a u> > r- z Or u > C. K •^ _i U' a K- -U U o >* K tO O — to u. >- u 1 > <j »- K- < <a Ll_ •N •> > —
1
or. Ll u Z' KD a —
>
2T 1— Z) <c 't c _»a. a — —- •* 1— a: t-—
|
u_ 2:: •—
•
^•: < —
<
•-» l- S< o O «0 \D — < Ui U < 2T O a. U »— 10 a XLL vO — - (\J u U 1
1—
1
<y 2: ^ O ZL' —
1
LU X UJ CI v0 .—
1
*—
i
u_ CL UJ a to a O _J ^r iTi u r-l
s* 2 r* *— — _j O 0^ H < (— t— a a 1—1 a < • r. 2* 2: (.O »< < «"> K Ul n. <r > <-! <r 3 Ul t- 5 to r r»> < < LP Xa (V ^ 1- _J c^ ^T U.J s X X O < —
•
CJ • 1—
1
U mid (— -— •> h- < UJ o: *—t (— t— X UJ 5. to Ul X c: t/ a UJ •
o a: c *—» —
«
_i \— 2: a X 1- r-< Ul < •—a o •* vO 1- UJ CO 5 to — C' (V 3 UJ a 1— Ul — 111 K > a to ~^
CL a *— ^ .—
.
X UJ O Ul 4 z t- to '7 •~ n < a cr.
«£l tr.) •• »— a 2' ?: < -< CO a a X •-* K u X D <
a 2: •* r> —
~
a. • (X c 3> >- 1— UJ u> -~ l/J If O >LU >-i kO — O to to a. i/; •^ a 'i X — »- r Cf^ X K-i— a w vD C (— UJ ill D_ 10 1/1 > Ui _l X • Ul h- <1- <f #* X X * a 2Z> ^L a •—
•
* lT\ to X UJ a: U' <: _j > » u a CO — • • X — h- ^H ro a IX' *0. U! UJ X vO — UJ h- 1— UJ a U 1 u ..r >- O c- C- m <r •— rvj (XI —
.
a' r\ ai;. 1— > Cl. ~ a f- \n 1 a 10 —> UJ 0- CO 1—
«
r-H r-l • • •> • * — r-H *;'
o h- < >- O > V O »
—
* Ui a h- —
<
u. a r c <r X X X 1— r-H CI •>
U •—
H
21 2" l-H •- 1< (- to UJ u X _J t/; a Ul C: ro <M — r*l i—
1
vD m u'l X c iox t^ 2-(V o a — <c Ci h- DO «-.
'
O Ul w -^ CD -" u — —
-
•^ 1—
1
r-« Ul ^-1 c-^ *3: jr *— » c L>i u-j 2: < UJ •—: < O '•> UJ a •^ w ^> «^ •— — O r-l
< to «~ 1.9 UI .X u _l »—
«
a Ul to »—
<
H 1— 1- 1— H- H- 1— t- h" 1— Ih nj mjr at 2f vO id
—)— 1— #^ Q 1—
1
1— z h- z> Z) < <r < <: <£ < <r < < < f\J< CM< ld u * ~— Di O t— to JT t— < < 1—
.
V) ^. ^r 2L ZL jg i-. >. 5^ ii 2" ^-l i-- l-H Oa o 5 a. a >—
<
< 1>- < 1/; > > »— >- 2» a a a J- a a a a a q: a r- a II <.
»D c-. 1— —• X ?: e? 1— _l a u. < X 10 6 c>: O c c; O c; O O xo D UJ
o a D CL * to X >—t to a c: Ui ID »—
1
a a u. a u_ a a U. a u. <i-u. Z aa •—1 CM — u > cl a O Uj 1/) >—
<
UJ t— 1— t—
1
a X UJ < UJ 3 a <j, a CO Ul X < .—
<
f\j <^~ <t m vO r~- CO t—
4
— 1— l— 2: _J to a — < 3 I— > Z LTl in ur> IT\ uo in m m in vO
UUUUUUUVJUUUUU
1007
jj**»
'
< o_J —Z> l-s: <cO Z)
< 5-
i- << <v
Q CDO
3 crcr q.zz
li_ 1-H
oQ
h- LUCL CO—. COU. U:u uLU O
ClzU.1 COLU I—S ZH- LULU 7-'
Cfi LUa:
CO Z>cr co
Z> <C
O LiJ
X 2U_ Ll.
o oa: ccLl.' LUCO aJ
S ^'
3 3z zCO lO
lO -• —or
o
ccLU
LU
DCcC£ctLU
oCLLUQo
CO
>-
CD
QLUq:LU>oucOCC
ZjoXLL.
occLUCD rH
II
->
<I—
oLY
co » z:— c * •* rl C
H W II II
Z m m -# —
I
Z! < oLu O •-"
co or o x
X
zQor~l
— — aZT ^r. *•>. » O
i—!«—<».II II —
i
_ — x
—. _ <r.- — >C X G. ^- <T* * LU •
co co h~ r\j • •
in m to n co oii ii
Cj C _J<I < J HIM nLU LU < — ~ »-
or q: u x x x
u u u u
cc LU •
LU LV LU h-CO < z COCO o >—.
o If)
Zo _J
2! o LU <o
1—CDz
a < >- t-H
z* > <LU a: y CCL LLi LULU CO LU LVQ CDO
cv:
UJO1—
H- X CO
o -IZ" _J<
cOz z
o o >—>
a Lu1—
1—
XCO < o2 •* h- >—
1
o LU Z) u:i—
i
\.J a ^h- Z *;.
< UJ O ID1— X c_; ^Z> H-
1
a i« K2 • > _J
O > 1— Z)
u »—_J
COLU
LU _J < Q£CO < Z3LU Z) O UJX o X »» —h-
orLl.
<
1—
lZ z X• LU CO s. * *
z* X <r ^~ -~
o H- i_ "1 ~)-~ I o f^ r
> 1— CY 1— z •—I >—
» < O <[ "^ ~«— X 1— Ll. LL "T" c c
s.. •— 2 «— —
\
o LU rH 1—
1
* — — or t— Cl 1— cr Q. Clt—
1
X > —
-
Z' s: Q Lu c Z' ^ * +II # LU Ll. #> o LU a Ll. » »• — ^~
1—
t
— C t- r—
1
i-H a u U Ll. 1 1—
1
«—
1
•—
•
•—
i
•- Z" C.V. VC II rsl v: X O CD II • II ~ —r—
1
— £T G i—
i
V^ ll ii o LU 1— 1— c —) X O •
i—
1
X CO > 1— .
—
— p* UJ II ll o• .—1 + _l * + II 2J (XI >Z >/. •—
1
l/) c z C<1 ~ CO + —II
o >- _J > > HH «— —~ h- z LU — ct •—• 1— —
«
oll o II < II II a or O v: v X o 2 _J II o «- o II ^- 1—
1
>- o >- uf—
1
f—
1
r-t
> M V 0. a ISI X
LU
1—
<>
COCO<
1—
u.
Ll
O
c o c h-
CO
rsl
108
/•
UJI—<
I
QT.
UJtoCDo
U.i
to u.
o
toUJ
a.
QT. Uj< UZ) U!
e Drl
toa:
i— UJto h-
< u_ —UJ < -~ ~>
_i *—
*
—— _l
1—
'
u.u ->
>1— 7
"Zi
_i \- 2?" •— * »-» r *•
< N < a .
—
sZ i—
1
1—1
.,
^~ * •—
*
crO
a.
—4 r
i—
1
II II
->
— i— •—
i
<t a * a II•* 0k
o Q. •— > * —i
*—4 -~ «—
•—< O O X -> — •v •-H —>
X + U; * «^ -> «"» >— 0k
* U_ »- 1— o X •> •—i t- •—
«
— O o •—
•
<r 1 + •—
i
»- (X —i— i—
i
.—
1
«^ 51 o — <— X X Z Q.
z V —
-
z. N 4L o H f—
1
z 2- •—
•
^ *' o V ^- «-» » *w#
# •» X o 1•* i-H i— a. 4* •
>
—
#* *• r—
i
r-<
f-H *—
»
+ 1—
1
rxi <—* X CO ii 1—
1
• .-H > r-l •—< a U- r- en c^ 00 II CO
II w c 1— II II UJ a II O II \~ II ii ii in m a-, in Lf". 1—1 m•—
t
c> f-H < II 1 •— •—
1
n ~)ii 1—
1
~i *»
II N z —
<
c; " — -) h- I— i— i— 1— O- 1-
o- —. II —
1
_l lT\ -~ z: • vO •—
i
»0 >—
i
o r- •* 2T" zr 2! 2: zr <—< ~z_
1—
1
o 3>3 •—
•
1— •— «cT w «^ I—
1
»—
*
—
H
>—
i
*—
<
»—
1
•—
4
o ^ r-H (V I— O m H- C o > o > o c ^ 12 c: D_ a: or. O or.
D o N Ixi
(—
UJo
1^ o X
in
<CaX>
I—<>
C: 1— c i— a o 0. a C^ c Cl a
1-4
->
UJ
<•—4
rr<>
— ?~~- <j
1—«/) U.i
x »-
* <
>-a.
>-
x+
CO» >-
1- v: >- 11 ^Uj O •* ll — II
11 1— %o «r\ C• «— CO U. " i^ —
• >-o >- o h- 1— Q. — a(Njoooooaiion zzi/a^
II tl I
—
>_(_•—•— — h^ •—
o>oo>-tootox)D'o:a-toa.OQ.ooa>-tO>-aaa>->-x
a
11 n x— -) +
00
o
u (.J u u
109
K--
UJ
CO COO < •
C _) LJ_ 1—
,
_J
c. <r CO1
J
> <aO
<T _l
KD o «<
z 15 LU >1—
1
^ 27 <Di —
.
D3 t— co CO
G < CO<<
LiJ a. cO
L z> CO UJ
z »—
«
h-
< >- <—
»
_j <r 21cc QT. i— •—
>
< Z) < 1—
> cXo CO
LUo sz #v Lll o<
LUZ t—z
10 JT —
4
Ll! 5l •
(— < o a<r Li Ll. «-~
2^ ID 5 h- _J—
4
o dt. < ~~ ^1— or. O fV —) -)
CO D. Lu d — —u_
COa:Lu'
CLco I
IT/ < C_ o X CL
UJ u >;< >i<
K CO z ^» —< • I—
1
1—
1
~) _J
c <r * **
D. h- Ll CO i—
*
—
1
-DU21
'
1—1
H! a— < o X +2~ u. »—
1
z a. —w c tv < + ~>
a < -~ •-
c 1- > * >—
>
-H
c D. CO <rJ z — 2' «—
_J t—
•
C or. •* * % o * (—
Ll z 3 «—
1
• f—
1
o i—
t
t-H 1—
t— u < O II O II II X II II
z. U X *—> II-) — ii _J —
u or CO »"» ~> — ~)
COU-
o:zUJ
u_h-<
DiZH 1-1 1—
1
•* —
•
rH •*
o «_ a wlv u 51 >- o ^t o 1— i—
i
o 1—
^ — or. o X c t— X c 1—
L t- 1— LU
I UJ CO > |-H
h- CO UJ LU r-l
X— CL~) +
o+
+
~)
.— I—I t—i ) «*
.^ I 1 ' ^-~ »v 2)K-, — O C —
' ^ —Z — Z •QZrHZZfH«- —» C- * C # •• D- * » D. O > "—
.— « •—1 .—
t
|| (J H || r-H I—t || H ll! LLII X M — ll II — II ll — D. O I—•— II -> "J — _J-> — ~> ~) M ZDi
^- •> ~) •> «• ^~ D_ O(nj •—i r\i >—
• » m w m r> w "i co
h- C; — O O •— _J 5-
0(// C>-hOO'-h OO'-i— <- ll
~ >*—
i
•X Xa — z
z s + Z 1—1 .^« m ^> - z. —
.—>•
t-4 »-< t—t -~ •* X >II II — II
•—t 1—
1
* LU1—
.
• ~i > — >— LU II — cc K- > 3 —- — z
C"' II m II (N 1— 2 — c_e—
1
*^ —
t
*^ rH II •~
'
•
coi—i + _J
«— ^> ^h 2Z o > _lC ^> c > O ^- c II o II <.O h- D 1— X u >- D >- u
(NJ CO m <\j LA
LJ u u u
110
CO LO I/) CO Ll CO Ll'
< < < < O U 1•—
1
X_l _l _J _l O 1-
* • »- «> >' CO Ll
O o c. o t— u £Tl i- c ••- •> » » — a IT <; < z ID
<J" <J- <* <) vO 1- 1—
1
\— —
•
LU•> * •> •> •* < O tO Q ">*
vO vO vO vO vx:^>- Z Ll. D •—4
•* •> • •> w. CO >- Ul 1—
o c C) o - UJ Ll. c, 1-«> •• » »> \— u 1- O <1 <J) ^~
eg <NJ r\i m r. z »—-
1
I— LLI fM» •> » »• »-• --T 1
1— O Oo o o o vO >— 3 O 1/3 h- 1—
t
—
»
•> •v > •> •* cy U.I
r\i r\i m (NJ vO < >- ? »- Q.•> ^ *• «> -— > CD z> • z O• • • • n. \n 1- u COm m in m t- u C </< D _J O* * •• » u <i a < * Ll
o o c> O ~~ UJ h- ^ > O O» m « •> vD 1- »— m >— —
i
*UJ LU U.I Ul •* 'C U — X O X_l _J _l _l VJ"> 1— X u. a + »—
•
h- 1- »- 1— — (/) u 5 _j u «—
I
DT•—1 »-m 1—
1
t—
«
Ul X ¥-1— 1- 1- 1— ^~ 1— > y 1— ^ (/- CL <—
1
i—
i
•—
1
»—
1
2T r Ll CT —
-
•— * ZEL
CO<_Jr
<_J•>
cr
<r
_J
a<_i
••
Or
vD
<u
CO>10
>u
u
10
>
1U*~~
1
X
5^
a*
>OU
— f\J o C — C\J o m a c U.' u i _i 3 •— LUvO r-H •> #> WO »-• •> «> » 1—
1
X — < 7T X H•> r> Q o « # o O t) a 1- s <-' O 3 a <
t—
1
r» O o «-< r- o o •s • 3 > 1 11 1-II ii
^T 5:-ii ii s: z. -I »~ C0 Z c Uj H- — CO
*m' 1'
*
at—i >—• •> X
Q.
vr: LLi
C Ll
>0: + >-
#* ^ H- » • ^- w O <i X X O ^ a— — CL LP »~ — o) >^ m _) a t- c ^* — <—* •— >• >~ i—i *—
«
>- M < *— D < O _J 0. LUw — #* *. _ »— * v > O U u K- <s' ^ •— 1—
Ul UJ a Q. UJ LU a \£ «< _l <r* • z r\' X CO
_J _l X X _i _J X X < vD < »—
•
UJ 7l • aH 1— •* •> 1- h- h •* LU w. u _J u Z < •— O X
»-» >—
1
>—
i
a a -H >—
1
Q. h- H- -^ z < »- f—
1
X rH
0£ 1— H z h- z 1- 1— z. z (/: X Ul «? < ^ < UJ .3- f'1
—
I-H —
.
•—
.
— 1/3 — *—- «—
<
«— w a. 2" —
»
<Tj 3 00 *• »u. 1— >- — — >- i— w w UJ — u c- LL O «* X X zh- + 2" 3: !5 CO ^ 5^ z z K- < _J e LU X c •DL U. * X •* •h < X < >- •» ^ < < ^—
*
~
>
> a UJ .— rvi <—
1
^o a 3 D- r-H tf vD ^O q: m <}- a: sO vO a: q: h- — a iS < i- UJ w —
-
•«- * 11
in w. Z <;
'
LO m o c X in lo o X o -) m a: u d 1- < U 1— \— »- • •—
* > —
«
II II ll II i—t <r CO CI O <r^ cr, K UJ l/. ^ z < <t <> + ii h- II O O _i -i n ii a o _J ll H _J OH UJ
—"\ < HH »—h Ul i- *•_ 5; m •—
1
ii >^ Z o m < < _i Q UJ _l o CO < <. _j a CD _J d Cj s: K u. >~ X a: oc (Y DC 11
> ii Q. c 0. c < U. Lu < O < < o < UJ Ll, < o <1 < z ZJ •— l/' •— _J O Ul coo O ON v: L_y z 5 _i Cl Q: u 1l _J u ^ _J £V £V u ?: _l U LU CO u u u.i a Ll u. Ll LL O D
>—
.
U u h- Q. Ll(\J X X a 3 0. fc- ^H nj
H 1- C" _J <t O m li^ m
O O KJ U U U
111
2^ I• Ol
l-H II
II —
a*
CoII I—
i
— IT*
II
-)
V -> -)
* #» *-} VT V*
—
^ *—
'
a 1— ah- (— i-
* >!< *•~* —
.
*-•
V i/ V•s •i *•
i—
«
»—
»
i—i
w —
-
—O a Q.i— h- h-+ + +--~ *^ .—
i
~) ~> "i— oQh-
z —
II
o1—
I
Q.
-i oit a^
n •
v: ii
i- o.—(.-• f\J (M z ^ z h <r
* c * t— • • »(— » •» o •>—«—•—* i— •H || H II H H H II' H H II H B II II II .—I
II —« II — II II II — II •« — It — •— —
>
— II
•» •* * . •* o o *<T>—<<ft—"iTiLOlTlt—i\OvO*—« vO •—< <—t i—
I
•—if.
r—I -~ r\i
ii~> in II lT\
c ao t-
o oh- Q.
C D-c a
o oa a
Ol-Ol-OOOOOOt-OI-OOCLOOH-QI-OOQt-OO^QH-OOl— Q
r- — -zc •—
C f-i Dio a d.
vO o
OCi -z.
— Q OC or ^ ^ca iljuj
00
112
ooo
\0 t*- 00 t> oo o o o o -*o o o o o oo o o o o ©
<M CO ^ lf»
r-t •—* <—i <-l
o o o oo o o ©
•tf
<1—
1
o LU < a 1— z 1 »
o > O LU 3 3 UJ Ct »»
•-H
•—co <
l-H
CO Lu< cx •
_l _Jz<
2: 2:O O
LU X2: LU < •
(NJ
.^
u •tf CO LU < —
1
LU _J LU O LU -J X X 3 »- X * inQ •» lu x I— X < a U> _l O Cl Lu Lu Z < Lu (NJ 'MU o
oxO CO
1- Z3 LU
1—
2; X<zO>:
1
M O OLU
_J XO 1—
<t-i •k
x o LU LU a Lu 1- u LU LU LU O CO q: a COo o X —> z — O LU < •—
1
UJ Lu 1— 1— O 1— LU •t •—
1
«-»
li- *o
O xh- LU v (X
< X1—
LU 1- CO O < <X X
O 251
zLU
21
3 _l •
en 1—
1
3 CO CO U UJ z: O IX H- LU LU X •—
•
Z c LU OvO » < LU O .—
.
LU a O z z Lu O u O i-H
<f LU ~ _l X z 1— < CO _J LU LU O Lu »—
'
ct <t O UJz m t- 5: or co UJ t—
1
Z LU a O O Lu LU *-> 21
< n CO •—
'
LU _J z •— •• X Lu Ox m lu h- CO X a. »—
H
K z < CO CO LU • IU 0£ NJ •-« CO> t- St CO u • jr < a O (X 1- t- O r-« O O #> (N cO m.—
>
q: V 51 LU < or < 1— 1—
1
Lu Lu O O a: n ^ *^ »—
1
UJo LU LU O 2: lu LU cO co LU 1— X O -J _J O 2: X f\J — U II
X u. ""v X O •—
<
.—
<
(X LU 1— a cl X »— » 1— Ot—
1
CO f- Z t— - u. X < _J X CA 1- O O 1— oe CiQ z CM < »—
•
t— z a 3 H Lu O z u 1— 2: O » a UJZ •—
«
V. Z ££ 5: h- >—
<
O s u. c0 O CO ~ CO < O «— »~ >LU CO 1—4 z _l LLI —
•
< 0. >- z X ^~ (NJ — C _iCL z T~* < < >r LU LU a Z u CO Lu —
•
»— w ^-t f\J fM < 2:0. LU V. 1— z Q O LU —4 h- Z LU LU •— _l 2 •—
•
< «— 1—1 •~ Z UJ< H- I/) co cc: < Lu >—
•
^ CO < 1- O > > U LU O 3 J— LU *- CO — a:h- V. O a: 3 t— < a, u 1—1 •—
1
LU Q Z < LU > CO fcH
*•* O Lu _l X CO LU h- H- a LU _l O X Z < CO < UJ «~
X «t O LU CO < u co C O K- < < co 2: O LU v: CL O _i *-
3 v. 1- a: X ^ h- < —
>
• . 1 3 X X O • • u » u. < < UJ
O LU »- u U LU -J > a < <: X LU cO O -» —
1
* »~ c2: *v O U LU z O y- a a > Z 2: r-4 (NJ — uj x< 1—1 LU C£ Li 1 u O 1—
•
Cf 3 2; 2: Lu •— —
<
O 1—
1
CM 1—1 z z > 0.x V. z O CO 1— • O co a O O CO < X O w 1—
<
» V — <o •—1 1—
»
z _J ST CO •J u O CO l— LU t-t O — CM u — •—
4
1o — CO Z O CO LU CO _i UJ LU CU 1- — » >—
•
1—
1
f*^ CO X UJa 1 CO LU LU —
.
•—
1
CO LU u »—
•
< » O X _l u ^ X — -J • z -J0. a LU •—
'
_l Q »— < X U _J z 3 O t—
«
u r\j w CO CO O LU x a •w O Lu 0_ LU — a: —
1
LU < -J O 2 LU •—> »~ < »-4 l-H 21 o: 2: xX _J 1—
«
2: co O _J Lu a hr X Ci U 1- <. _J «— <\l -J •> Lu' NH UJ <O LU CO u < 1—
1
UJ O Lu 3 a • • Z 3 X q: CO N 1—
(
» < » O to **
00 o .— LU CO O O a < 2 < < — < O < *—
i
• — »— (— O •-H Xo a »— LU O CO z <— o-: »- H- O LU ^» Cu r\j < l-H X m X
3 2 5: co 3 51 LU CO •— 3 O LU < < co a: ct z z (M Oi 1—
1
•—
1
00 .-< vD
< • c _l h- CL C X c — O a •—
•
«—
1
•> — V mD •n ~ f\| —~) • 21 Q < LU •—
'
LU a Z 3 t— Lu _J »— • — LU «B^ ** •—
<r < h- 2! Z > g LU O _J • O z •> cO r\j CO _J Z »- I h- H- 1—LU » or O Lu < 3 3 » 1 < t—
|
< O Lu Z z LU LU z f-i l-H K- < < < <.<Q. a O 0C O Q CO CO CO U CO u LU O < <- •—< • z »— LU — *—
'
•—
>
2: 2; 2 2 2 2o * a CO < CO •—
.
UJ —
*
Z cO 2" UJ 21 c X t- 2: X C£ OCX XCl LU x -J CO < LU ^ Lu _l X .—
1
CO _J _l •—
•
Cu »—
1
1-4 X »—
«
000 000« ^ a CO LU < x a z Lu 0^ CL < Lu LU UJ > X z u C» a » » u u. U. Lu Lu Lu• _J >- d LU h- <o LU 2 < (— _l O < CO 1—
1
(M ro
a * X O Z X »-* LU LU O < X CO O O QT h- X •—
•
O •—
«
f\l €*> «t
o Z 1- SI -> < 3 a O U CO O a 2: 2 Q. < 1— O m m lIMIMAo 1—
u u.1 1 u u u u u U U u U u u u u u u u w u
113
fc*
\ON CO O^ OH N (^ J-.—!-<.—I.-IC\JC\JC\JC\JC\Joooooooooooooooooo
vo r- goC\J CM (NJ
o o oo o o
o o •—ir\ico\Or-cooO'-"c\jco>}-mvor-
ooooooooo cooooooooooooooooeeoooooo
X I X* -t <1-
11 II II CO O^-* •^ *^
00 m CO CO r*>
r- G^ ft »• •^
rvj vO C\J lO co m vO OO, 1—
1
1—
1
f-H
»-. LU X X X ^ — — X X XC\J CO <t 4- -4 CO CO 00 CC' CO 00»—
t
CO II II II < < < II II 11
ft ""* LUU CO CO CO
-J 3 _lCO »o O
V* to1— or
ft
m 4) m M M w CO cOX a a w «-. «« CC CO CO
»—
1
<CL
C\J
COLU_i
cO<3
to<3
10<3
<3M
<3(A
<3
LU * < —1 a • OLU 2: «• 2: a LU kfi) W3 M X X X XX 11 t—
t
— < LU Ql <r <r <r <r Oj m 00
0.c ft
II co1—
«
COO
1-
X) lO11 11 11 11
Lu UJ en O •> u_ LU a LU Q> vO CO to ro CO
o 10I/)
I—OT
LUCO CO
CO
to2 O •
CO r-1 in
ft
1—1 4*ft ft Oj in 00
LU UJ < CO —1 _J or u_ u X • 1—
1
1—
1
1—1 fM X X XU u 0. LU LU z U h- (X: z X X X — M » »» cc cc 00z U CC Q O Lu s -t -d- <r 00 l/J lO CO II 11 II
< or 3 O * CO CC DO 1—1 u II 11 11 < < < < — *— »»t—
<
CL < OT h- 3 •—
<
a. J5 u LU —- — — 3 _J 3 _l r\i in COQT UJ 0. < < O Lu CO CT\ ro CO — »* ^-
< 0-1 a OT i- _J LU 1- O >;< ft » ft CO i/l CO> LU CO LU z LU CD O _i z tM ifi CC M M W M CO CO COo 3 H 2^ 2; » Q 1/3 Z LU •—
1
— — «— < < <u a O u Z> CD ±C O O >— Q CO LT \n _l 3 3o 5: O Z Z \ \- < 3"- * _l < < <Q < or -J » ^ _J <N < 3 3 3D CO co U_ _J Z LO C •v > M <A &LU UJ O LU VT u_ 1/5 r-H or OCO u. DC DT O or ft u U II LU _l t« tA w 1—
1
a < LU CC ^ LU _l < 3 h- LU 1—1 <d" r- CO-~ ^- CO LU •* M LU -J 2 < Z O r~ m z<NJ X DC 3 (X) 2: O ^ a CD u_ O 1— >—
1
V r-1 CO O *• vO Z O • 3 or Z LU C O or Z ft O <r CC 1—1 1—1
O (\l Lu z ft CO 2 u. U_ O V X X X X l-H >d- r^ II
I—1 ft X 1—1 ~ — ». z or z Z Z tD X X X <r <r <r <r XUJ X p- X LU vO X 1—1 co X ft H u. LU O z 1—1 Z <r <f <r 11 11 11 11 X X X 00 —(M O tM «0 Oj 1—1 r- X <r CM 2: O a CO UJ O _l ft ^ ft —
1
11 11 11 — -~ — — CO CC CO 11 00r-l CO "* co 1—
1
(M 1—
1
vO <M Zor
LUco23 3 1—
a2:
1—1
iTi
r—
1
11 CO m j«i
CO CO (X. CO. ll II II
cw
h- 1— h- h- h- h- 1- 1— h- H- in H- LU -i Z a < < ft ft » CO r-4 <r r- 1—
1
< < < < < < < < < < CO Q 3 2 _J CO h- 1— 1— 11 i— f—
1
•tf r- 1—1 1-^ r-i 1—1 ^ -^ —- ^ c2: 2: 2: 2: 2 2 21 2: 2: 21 Q 3 or Z cO c Z z z Ul CO cO cO co cv,
or or K or or or CC or or CC < 2: c *—
•
u or CO t—* >—
1
1—1 X •—
.
CO CO uO LO C^: CO CO CC Oj CD ccO O O O c O O LU 1/; >—i DC CC or < 3 < < < < < < < < < < <
u_ u. u_ u_ u. LL Lu LL. Li- u_ or _i CO 1—
1
X 3 2 CL a a 2 NZ 3 3 3 3 _J _J _J _J _l 3 _J QLU »—
<
< < O _llT\ \0 r- 00 O 1—1 (X) en <i- O co H- h- or LUm m m m in *£> vO tO vO nO VT Z z Z Z u_ O
U U U VJ u u u
114
ooooooooooooooooooooooooooooooooooooooooooooOO»-»CMC0.4- (n»0r*- © »*vor^-r-r-r-r^r-r-r"- cocoooooooooo ooooooooooo oo
10z H-o <
<-Ju.
-» <\J tosO »H »•*
• _l
II II »mmX
UJ LU ZI— _l _J LUZhl-QI, I 1 t—« *-* -*
Q h- I
CMX< < —
o<O co co <
>0 «0 *0 QO Q Q -J
< < < -JUJ LU UJ <cc at en u
oCM o UJ»n «-i r>
z •i- o •- oZ K I- II
— z >a: o o < o
CO ^
o 2:rsl X+ X ^> < >< s: <N II NII X II
~ > < >O < 2T <
><M
X I
< —2: ^
UJUJXa.
u_oUJuz<
<>
Z<UJI— •< z o2: • ii
•— i—
i
i— ii •—
lO M M
• <a. en uj
UJ i-uj i- o<a3o
Xu<UJ
uj aUJ UJX h-Q. Z
OfoI-
UJ>
M <
oII UJ
• orH II
II ~"»
oo
• z >-*
«-H • II
II •—I —
—
—> II •—
•
uj eni- a.<21 UJ-- Q£h- <tOUJ to
UJO h-I- <
Z^ii! JHUJH-enwQ-OUNONNXNa
<i-i oh- a
CO t-H CM CO «* z
oo ->o a.
in
lu inLUX o
O ioO LU_j
LU3 LUO Xz a.
— <Z »-<- <O
Z LUo ut—
i
LU
uD I-cn toh- o«o s:zO to
UJQ
lOZ - 2!« ^ •
v: isi
II H-«fl-ZiflZl-
^ LUm 3
a oa. u
u u u u u
115
m< i
C\J CO CO C* OrHfViW-^invOI^OOO^OHtNjmNfl^vOI^OOOvOHINiCO^ lf\ s0 f- CO vtCO 00 03 CO O^O^OSC*0*O> O^O,s OSO^OOOOOOOOOO»-Hr-• rH i—I h i—t i—i H H 00OOOO OOOOOOOOOOi-l^-t'HrHr-I^^^H^^If-tr-I^Hi-H^t ,-< rH rH rH Ooooo ooooooooooooooooooooooooo ooooo
-~ X> az *-> <-»
l—l o ~>^ f-H «^
*—
»
-—
*
-J a «— r-Z O •—
1
ISJ HH r-* or —
»
h- o » *.-H + «— -> NJ i
~> —II M ~) —
»
* o w •-<
»—
<
O ^~ rvj —» — r-t X o <~-
•> i—
i
M * •—
«
UJ *-^ a -- rH o-~ 0. * ~» —
'
i— w — or CL 1M N «— ~> M < O •— •— * ~»w ~) * * s: + x < *» ->
LU N #> »—
*
^-. »—
i
•— CL SL -) •>
UJ O •—
i
w »"H O k •—« II w »-H orX Z r-« M O Ml r-i i/) «— o >- NJ W o0. o O CL o t—
i
IxJ N UJ UJ rH r- + o r- s:<— iTi -« II z z i-H 0. z < UJ 1 z UJ a — z Z -^ z z f—
(
u • z• 1- o * » a + > V X -~ UJ •> X »— • » l-M w 1 CL UJ rH • —»
• i—
t
< i—
i
i-H * —» rH ~- a v: UJ 1—4 CL o z rH f—t —
»
1—
1
•—I II > II <NJ ^ ^iTi II H- to n II —. *-H II t—
1
+ X II II 1—
1
lO UJ II • II 1— II II*~ Z II ^^ *-* _j
lH -) 3 •—
«
•-h • -> •—
i
w •—
1
w o Q a. •—
<
«~ Cl H Q I—
I
o -> r- •—
«
-> 3 N -> H — Q LU~) CL o —
»
o • o i—
i
N «—
i
1 II II « isl M aH Z z • o II o N II o t-H II fsj II O CsJ w. O z CO «» cn •—
•
<}- >3- •—
>
U. ON f^- *-i ii II ^Z o> o o i-H O rH 1
—
ft + — (1 r-\ •^ II _l z rH uj z o to rH >
—
i rH -1 rH rH •— o 1-^ l-H i»» r- +
•—
<
u •—
•
II or •—
<
< I—
i
o »-* O >— >-^ UJ *—i H ^H '-r ^» o »—
i
_J z ^or o 1- o ii o *— o ii w 1-\ o >— r-i H 1— o X h- 1- o r- O H o o 1—
1
r- o o H *—'*— H
0- a z < or < o e isj C NJ N < o CL < < •", o r- Q r- a O Cl U_ a a _j ISJ M ^t—
I
D o Q D X —
i
CO < o o rH CL <M CL O CL <«^ <r X r-19 UJ rH i-l I) r-< Z> Ul i-H rH U) w-t
u u u u «^»
116
C0COr-4(\JfVJf\J(\)fVJ(SJfV)<VJOO'H'-t'H'Hi—t f~* <-* i-i fiooooooooooo
& O »h f\j c<"\ <t in h- arsj co ro co c> co ro «* ^o o o o o o o
COCO
O -J
OfUJ LU
LUOcco
LU
Q —<s: cc
LUtO CO< s:3 D
LUCO
cO
Z<
• *:
LU O
u.
OLUQ•—
•
U)
IuX3
O
oo
z
o
cc
cccc
ccX
II
iD
LUz> r-z-« o
LULUXQ.
LU CO
D enz
zz•>
CNJ
II
^ I- »- I- I- zw z 2 z —Lu O O O O QT•- u u o vj o.
+zII
o
cD
LULU
•H X+ Q.•— II
I —z *-
z »-
ii or-> x
CD
_Jai—
cc
~> —cccc
"1
0£— z
II
->
oQ.
00 a>in in
vO II
in •—
i
in r\i
COcc<_lr
in< —_i _j•• LUL0 OCO •>
< —a <
CO LU
c~ Z* z
ISJ
«LULU
cc
ct x
< «-*
CC •^ O* ot- o*—
<
*—
-
_i <• <X o< ^
zLUO
< I-o «o
LU —z o
LU _J CO0- CO << - u orKIUJUJ
t-a: a
z oLU Oa .-i
< uH- O< -JO CO
=> <O Ho: uCO Oz>CO OQio <Ii.
CO^ CO «-i
u zO O I-
o s:<o <-• cco
LU OX cc- a>- oCO z
LU COCC LU
-Jo a.
<< COi—< u.o ou. ck
O LUCO
^ zO Z_)CO LUX
cOLUa
LUQOz<r\i
a:o
u orLU LUc x
»-
o ~C LUo<f cO
« LU CO
CC k- LU
O —
•-• »— cO
O _JLU -I»- <-J U0.a luD xcO l-
< h-X CO
_J z z zvOz z
cr: cco. a.
on oCL o
^- aII INJ
cOCO _J< _J0. <
D< LUI- Dctzo— z_l h- LU_l Z< ou u
< CO
OaCD
cO
— L3cO Z ZZ O ~
a:<
ct cc< <
lu s:
— o
Ov O oo
o «o oo
uu uuuuuu
117
(XCL
<o r-or »~
cc UJ• o
LU r LUa
o a.z z —
-
—
»
om o (\i r-o * X or tOII 5: Z
<LUa H-
^*
u_ o < < o too ~ _J I— UJ z 3t- CO < • X Oz • x -~ r- O to -J<-• < • X ^ CM cO CM h- < _l *•
(— ^ UJ *• —
I
CO < 3: oX < cj 0T QT o _J t— x * u. toLU Q o «—
i
O o ^ >—
<
or 00 UJ o Z — 3^ _i oO o O O CM z CNJ <— X to OCJ >- CO UJ ^~ <f >- z 3 00 1—
1
CO —
>
or < cO _)< CO X .—i ~- CO 1—
1
» (— —
.
h- 3 _Ja Q • < u_ O m X o tO r- or o oz o z UJ 51 K- Q sO CM o >- »— t—t X LU _l u.X liJ >—
t
X —
i
< UJ a H 00 or CO _J CD Q _lCO u_ 1— < x U tO V #• CO O N2 < o to
UJ -< H v: z X >t X o CO • Q LU u_ < —»
CO < O CO < UJ Q CC CNJ to LU i—
i
UJ X «»
OT r- X Q z a; Z o 00 O *fl — 1— to c: f-H
O z ^ o UJ < 3 « \- < r-t < r- — U r— < z X(- o < 1—
t
»—
1
u. CO M UJ II r- Z I—i LU < <—
1
<< r- u. or to UJ ^ <r CM Q X CO < UJ »~ o r- CO Q •z.
< _l < LU z a u o 00 Z < o a o o UJ Q z < ^r1- u. X t- UJ < vO * i—
•
< _l •>£ « < r-l o Ul •—I Ul ^< •—
1
h- s: • CO a r-l CM u. r- LL •> * _J «— or o X uo or >- t—t i—
i
z »—
<
z CNJ < NZ h- tn u_ V to or < oO CO a UJ X o 3 UJ CO z a W CO CNJ 1—1 u < X LU UJ _)
UJ or UJ 2: a z •. *—
«
~~ •-> CO k o s CJ X r— coX Qi x _ o 21 2: z QT o < o O 1— _j X _J CJ < ^x
H- UJ LU _) I— < o < CO I— Q UJ < z N^ H- < CO or O X u •cO < z or u CO < r- UJ N2 _J LU • Z SL 1—
.
o LU o «^
r- z X C_ h—
1
to »— z 1— x CNJ or U u. O X "-' — m or or t— _J 1—
1
z < O < tO o cO CO or < •— *—* < or h- "» or o < —< tO (/ 1— or t- •—< —
i
z •> D a •1 * s: o. z CNJ uj or CJ O ^3 i/) i—i >—
1
•—
4
a. < _i tO X •—
4
nO U z X CNJ to ~> LU 1 or o r— u—
1
a _J UJ <M u X < CNJ CNJ • fcA a o >- LU _l oX .—i < CO co ^ t— -J X CO o SL a CO N/ 1—
«
o v: i— Ul _o o r- ^ z X < LU LU UJ » •> U 00 «— c CJ »—i >- UJ _J a. CO>- < l- < u (—1 UJ a. CJ a tO CO or > r- in H- O CNJ H- <f o or r— > CO 21 *-1
_J >—
.
Q < _J a i— LU X t— o < Z CNJ Z _J CO ^1 *- _J < •—
•
< < X »—
< U. z a _J < X < X z or •- X LU CO H-4 CO r- Q < CO 0. or •x X z -i «—
H- iZ 3 U z < _l o h- h- to LU or vO a #> or H I— Z z r— >—< < o X r-l t-H
< 2 X u u < o S LU r-l fr"H *—
*
CNJ a — z LU —
.
< •> < o. r-l — o «a _) •—
t
UJ LU Q \d z z O ~» CM r—l —
«
X u. X < O + r- i—
•
^ • < t— Q LU Q X CJ o •* 1- CO t* + ^ or <—
»
*: r- X X w«$- u Z X r- UJ < 1— »—
*
X X X t—
1
<t CJ 0. » UJ < X X X tn CO <r r-l
»-< X < r- z a z cO CO o m Q — <j- v. < to z z X CO tn en m CO V (—1 zo o u CO 1—
(
z LU CO tf m m z O CNJ xa tft <M UJ •—
1
o V tn <f (*> —
-
«rf X co —a: o < z »—
»
r- X s: < >— — — »—• < CO < z o CO X r- •—
1
•^ «-» -^ << _J UJ ^4 X z l— i— i— u_ _l 51 X II z X cO z i— h- r- r- r— 2 z o or
< o a a LU < < < u. h- ll CD h- t—
.
o z o < < < < < II CO II LU•
h- QT LU ^H or S. 2 2: _l fc—
»
z X _J < Z 1— or LU s: 2 s: 5: 2 51 •-
*
—i or U.< co r- to < or or or _J -
—
f—
1
< _J _J •—
>
z o co 2 jr or or or or or X to < u.X X Ll. tO a. o O o < u_ or 2: < u. or o Z X »—
<
o c o o o o < to 0. X^ CO < < CO LL LL. LL u -* a -> U i^ 0. CJ UJ to a u u. Li- u_ u_ u, 51 < — CO
<!" LH CO CNJ en o •d- en o CO o inCNJ CNJ CNJ CNJ CNJ CNJ r-H i—
i
r-l r-H CM oCO CO 00 00 CO CO CO CO CO CO CO CO
ucjcjcjcjcjcjucjcjcjcj
118
inoCO•>
O COi-t fHCO CO#• *co ino oCO CO —* — CMr f-t
O r-t COco — »
SO U r-»
O O COCO _J «-
CO •— -a• I- D»- z -)»-• LUZOOD *- »-
u. a. o
moCO
o
a
cnj r^-
r-4 OCO CO
z o
CO
O
OoWI—zUJa
co
CO
(X.
a.
in
t-l COCO
CNJ
CO
oo
O
OCO
0Ta
CO
t— r^ coh- O Z «-•
to om to<
Z II
3 <uj a.or —
oro o.o
t—i r-i
co
OhBJHOO O O «H t-i
co co co co co
3 —uj oror aCNJ o*-* oCO CO
-. oa: •-a *o
w (\j
O COII
O H-< z
mt-i
CO
ftCO
COf-l
00*CMr-4
CO
r-A (VJ
CO CO* »O Or* CMCO CO•> *o oO rHco co•> »CO COO —ICO CO* *r» r-O fHCO CO* *
o -tCO CO•>
LUorLUX
LUtoD10oz
<< h-h- << aa^ or• i/)
f-co >-•—> co_j^ d•> UJX _J< -J£ <-> u
u uO <-J Cl
CO z-« 3i/) oro coor 3< to +
< <a az z3 3
X<~>
it
c0
or<
toa2
Oo
<CO
COzo
to—
Z -J-J
• <+^ u.V OX< orsoii ax <<z to1 h-
LU+ O
<o
or za —< Q
LUX i^
< us: <"> ato ^*- uo
-I _J3 CO+ «-i
tooro<<
uo-JCO
u.
oCOora<
u _3 Oor utoz o.— OO
co _j
>- ZCO *
t- oz or3 <
to to
«CM
<<
toor
z x aM < <a -> •-
LU LUJUJO_j or« o ou. - o+ to +
to
CO
<I—<
orou.
a
to
<<
oro
uI- <— 0.X Z iH ^fLU 3 O 3 cm
tO toor ora o
ft < * < ,$
I < * 0QH-)CO CM •-
u. u. or
co r- cjs
—t ft HCO CO CO
vOrHCO lH
II
»- oz <•-• _Jor a.o. »-*
zLUs:zototo<
LU3
ZO
I- >z ctLUh--)33<_JtOoz_i-»-«a<or•-•LUt0t0_l_lt0<
<3<3<3<_J<3<-Jz<ci<o<z<z<z<«-«t0-J»O-JtO»-»tO'-'t0«-, lO
LU
CMco
<a
VJ u
119
«•
XLU
cmcmcmcmcmcmcmcmcmcmcmcmcmooooooooooooo
mQ t-< m
X r-LU •—
•
XI— LU
UJ za: —
i
a:O LUu onO
I- H-UJ tOm +
<U <O Q_JCD m
OLUa: q
z <<
a: ^o^ z
LU Qi> —< U_m +
O XI— LU
a —a: mO *
~ tO-> x 3~ t- ->
u t-O l- X_l Ll OOQ •—•
•—i
— x a:+ m +
m_j o z^ LU»Ol-^ t- X+ Z LU
< CC <~>
I- O to< 3 *Q^ Q t—
Qi inZ cn 3*-• ~>
I—LU LL r-Q£ -• XO X Of- tO •-«/) + a
XX * CM vOCM LU LU CM O * •-« CO
(—I t-t CM i-( CM i—
I
lO LU
_iv: <co O+ h-— Z<< cca o
z o-> z
CMLUa: i-O u.
m X+ I/)
Q ~Z r-LU lOH- •-•
X _JLU ^
mZ CM
o +
lO w<
» I—I- <10 Q~)
ZI— «-<
XO LU•— enat O
I—+ U)
a>—
<
10+
oCM
en
toD->
•
i—
Qo
>- +
rH II
+ •-*
o« CM
Q.O «H
o_J X
LUX r- O< < zs: lu •--) Q.
LU LUz a a:o o
f-< i—^ + mm ~> luw ii a+ -> +
mCD<<
Q. CD _lZ < •>
13 -J —O
UJl/lOz < ~NH _l >-
o 5:I- 3m ->
LUa +
Z> to
O mor <CD Q.D —in *
co->•• i—
<
(M •") CM•• i—l
10 —H- tOa <
a: cm•> f—
i
Ct r-
CM vOf-t CO
CM v0i-l CO
m
< *< *
CM CM CM CM CM rH H (\l
•-•Z)_J»-<i-«<t0t0Ot0t0Ot0t0O<'-'^ -)»-<
z»-—'ZZDaahKctt-ctQii-hZWJZ•-totoLuLu_j_iom_JOto-JOtotO'-''-'inLu
i oo o —
i
_J CM -CL ~ Ofsl CC
a: —LU >fz z I
NH O tOI- -« lO3 m <
_ o z CLLU m LU CC LU m
co s: «-
d — u.
*~ CM
«- *) Q
incco
<M CO-)
mCC t-o -^
<f < X1 00 LU
CO
oCM •II *2 —<a a.
*
<n
X CD
II (ACD< <
II
(A M00 <» •
r~ >3- w» % ~ —CO — O CD CD-» • sO — wi—
i i—i •> U_ Li-
I -HlflZLU tO Lu II O —
r- z> to z < vj mz < < «-<
II II
o -> a. I- — —I- h- —I < 1-1 < <Z — II «-" »H
o o u. — o •- —o U <-h Q. O X >-
o o r- ph
_j*o»
CO•I
CO•>
CM•>
CM*
«
-3-
>*
>-
lA
CD
U.•• ZCO vO >->
• "v tO• — II
CO • -~
• i-( CM• I —r- < >-1-1 w ^* * ~CD •- CM< a —J n x• CD *O —i
•> «r» •
i-H II
* — -~
>— •( •—
•
•> < wX X• WWo —vO (M CD
2 • li_
< «-• mcc ii oo •-> v
ii
_J CM ~-J CM< o ~U Q X
120
•4-mvOf~-COOO»-ICMCO lT>\Of-r-r-r-r-r-r-oocococo oooocoCMfMCMCMCMfMiNjcocMfM cm cm cmoooooooooo ooo
»<>o«--»CMco«3-in\or'-coc>o»-«cMcoin«0r*-coc>CO 09 <M>0>9'0> 9'^(M>^OOOOOOOOOooooooooooooooooooooo
11
*^ *
«- cmi- — ~_J >- 1- /
w* _Jo <- •* •—
i
o *»
00 *- * CM•k •—
4
00 «-H
oo o: • ^^
• ii 00 UJ(M — •> • -1» i-i CM *» h-<M — * O »-*
• > CM ~ O 1—
<f W * -J f-4 —+•> «-» •* UJ «~ mo- ih * o a: —
»
* + •* * •> oen >-• * 15 f o• «— CO < CVJ r-t
* q: • »~ ft «—
CO DC • Z * ^• ii CO « CM >-» ^ • _l T~* •>
l- rsl * UJ «— *»I—i «— H- a •—1 o «•»
* X •—
i
*£ X o-«
w
* * a r-l
CM -* «•. ^ * w a.-> ~ CO Z «-» u•> «^ -> * o X X »«.
— Q£ -» «—
•
s: o •> •—
i
o~1 f-i </> •> o ^» a: *i/) II 1 CO o 4- CM H- •H< ~ — < IM w cH — < II
_J «-H —
'
-i « o * *— 2: — . —•> ^» *—
i
•> UJ NJ » CM CO •>
O X a: CO UJ • •-. •-I • > « •»•> ii m x -* CM «— o O X t—
i
cm w vO —
»
in z a CM •-• Q. — l-t u a. *^* * (NJ •• • «^ «-• • * O UJ «-. UJ>- <M <r «— iT\ >- I) — CM -. O a X X ^M -I• •> > V » < UJ <-* CM r-< — f—•
—
•
4- 4- •Mi h~X to CO & CO CO X i- UJ — i-* — — — «3" » CM 1—
1
UJ D •—
•
• 1— —
'
-— •> ce x a o o -* • X • * « UJ < H-c\j a to i—
(
~» o (NJ a h- CM UJ rH • o ir> o ~ X X • • X h- •—
*
w •-« ^-1 CO 1— *—
'
UJ m hq;m O lH *» r-* CO o o s: 2 -i z a z <—
3c • Q. —i1 2 z Z ~ — U. CM H U. 00 UJ <; r-4 i—
i
• » ii 2 ii • ii * z< *-< I»-I —
'
I/) O < •—
1
O CM h- •> — u. oo «- co *0 ^* ^* i—
t
t-i »— • «» i-» «~ •-« a »a: ii 1 a: 00 O Q •- <—i i-H >- -« M> «>• <M w ii n -> CM —
> • •—• M •—
•
vOo — i—
t
a: < a D lO * »Oh »— 1— »- 1- H- »- - IM -) •>(1 r—1 •> —1 • •—
1
II HllA+ ii Cl bft O Z CM — O < < < < < < < < >-> ">
1 •- z »~II «*
-J CO •—^ ~ —
t
-j a: UJ r-\ MHZ s z s: s: ZZ Z t—
1
i-i •M CM -> w ,0 w f- M ii a_j *- (NJ •* CO -J Q CDZw .-) w q; a a cc a: o: rr or •—
•
II «-• -* •^ Q CO << o U. ^ u_ II < Z Z> ~ o — XOOOOOOOO O O X O — X O X o v: o < uju o >—
i
X •—
«
s: U UJ l/)Ql-H * U. u. u. u. u. u. u. u. o o a o 0. O CL Q XZJIft CM CO
cm oo o ^ ir\ co
mcm co ^- m *o r- 00m i/\ in in m ir« in
CM O
121
o«^c\j<^^in\Of^coc>0'Hrsj(^^-in\£)r^ooo^o»-<c\jc^Nj-in\or^coo^O«-i(\iro>j-if\
oooooooooooooooboooooooooooooooooooo
i/)
Q_J
U.Oo.
X•—I
a: _ —» *—
i- >U ~) ->
< * * •s ~) ^ ^^ *w *—
'
*—
•
C\J z 0. r- a«-H o 1- r- >- »r u * * * S
r- ~- —
.
«-» »» *ii LU ^ V^ ^ 2: r-t
<—
i
1— •> •> * II
r < •—
1
t—
1
-• i-H ~)»~ 1— «—
.
w w II •>
i-H l/> O Q- CL "J ^.M (- r- »- ->
LU >- — + + + — •-1 Q ") *-. «-» •* .»» ") M1— < » ~> ~) ~> ~> »~ * «^•—
*
LU 1—4 •> •> •- » -) »—
1
— Z)r- t- w t—
1
1—
«
>—
1
2: »—
1
1-^ * w X <*—
i
l/> 2: 2: t-x *• 51 21 5: w 2: 2: 27 — 2: * 2: — 2: 2: «- 21 21 1—
1
2: a 2" Ol r-w- ft « • X 1-1 « • VI h- P •« » O * i-H • 9- h- » * t— • « w •> <w • »- z
Z i-H |H o a a 11 1—1 t-t r-H r- r-H r-H r-H H- 1—
•
II O r-H 1— f—t r—
1
»— 1—1 i-H 1-1 * l-H • ** ^^ II II II it H ^ II II II II II II II II II 11 ~>
II II II II II II II II »- r- 11 iTi CD II m i-H
vO < I—
i
~) -~ «— .
—
iZ •- ~> «» ^ «-» •—1 ~) 2»Z »-» 1—
1
—~ iZ *~ •—
*
-> »-« •-. ~>II iTi I-H in in 1—4 in 11
m t— ") -> 2 ~) ~> -) r- ~> ~) ~> <— ^CO o- o- * »- •» m m » in * vO vO v£) # r- (M m h- « O « i-H c—
t
-~> 1— •-H r- h- (M r-Q • o CM (M 1—
1
•—
4
2: «-H CM CM >—
1
(M 1—
<
CM (\J CM <—
<
(M •—
t
(NJ *-x t—
1
i-H •-^ rsj cm •> z CO Z z CO Z 00< i—
i
«-» — >— —, «*• «— O —r •«i w •—I *—
1
r—
t
•—1 *—<
LU ii O O o O 0- O O r- O H- c O c h~ O 1— O 0. O O M a: a a: a:q: a i- a Q t— t- 1- Q o- h- Q r- a O 1- h- a t- »— q a 0. Q. cicao 0. Q
3 ^t m O r~ rH «-< rg
O C\J cm CM fM rH C\J <«» eozu
122
•4- .a* vr «* m if\ inminirvininmirvvOvONONOvOxONOco en m en cr> cr> mcicimmciMcioci^wociwooooooooooooooooooooo
t— H- lOto lO << < -J-I _J »•> * VD r-l
f-l r~* Z ••> ft »—
i
00r» r- 1- «•> « t- vO
lT> m o *• > -J <\j
o o a •>
•> * oo o oc •* * O NOo o u. «•> * oo o UJ •• « u oo o z •• •> UJ oo o 1-4 ••> • z UJLlI UJ UJ -J-I _l > »-1- >- z 1-4
•—
•
*— o t-1- 1— u •—•-4 •— »•> m < _lCO CD — UJ —< < X t/> 03 X-J _J » •-4 < •• «— •> o -1 oo en o —
»
UJ *» • UJ»~ • 1
• —. <NI ec ^ o or") a. 1-4 O O -h u. «— • u.* o * o i « •» X a •
«-H 2 en 2 i—
i
r- o o ow •> + * II ii » z 2: •a ^ -> —
.
—
i
D o » D* >- ** OT » • < O <~» * _i Q •> —
i
—
.
l— z UJ »--) . ^ UJ ISl ^ •—
<
i—
i
Z o or z• X o * X u. *
•—i •> ^ «-> * UJ UJ _J o »/> * _i «*>
•— D • ~) r> -J _i UJ • a: X UJ •i—
•
D < < 3 o w z> < 1— H o .—
!
UJ * f-«
»~ I < \ »- < o Q < »— •—
i
>—
•
NJ 1 > 21 ISl • M 1 to«-l 0. *» 1- -~ z i— o M - < z h- l— >/ 5: >-» z 2: or 2: «*> O• 2 2 + 2 •—I Z J^ *» Z en + z V w >—
i
•—
•
>- • >!< •-4 10 «— • * tol-H «* »«•»•» *—
-
* «— 3: • m ^» « »~ tO 3 •>— — l—t •— fH •^ 3: w ^ ^»— f-H • f-H •— i-H t- .—1 ^ < r-H ft •—
'
i-» •—
•
OT < « •> + Q ii -> z X < O II -) Xa ii o II *- II II *» II > OT 11 • II «-» ii ».» or vD vO D lO ~> «^ o «* or »/> "> <-» •*ii »- ">»-•-• «— .—
i
W II a •—
«
o ~) OT — •—
•
a: IQinm < a X •—* 11 a X II
•^ ii II -4 «— » ii II r-
1
ii en ^ K «3- II 10 i-i _j m II en _|v: <> — CN «~ 00 * ^ CO irf _j <r —» ** «- %« m — li ii _J Q Q Z _J <r — II lu _j _j <r «"» II UJ—
-
»—
i
M l-H > >— _j •—
i
•—
•
or — C 03 _l < < II _J -) UJ o CO -I -J -> ci^ o w o ~ o «-> ii O J^ < o ^* o w ii O w O < < UJ UJ IM < O •_ I o < < < o w O <>- a h- OHO a< a > UOK Cs CL < a or 2 -J U OT OT Z U Q XKZ JUvVOKIJ
o 00 en * in-3-
123 ,
IV
Om
LU
I— • 1— —to O< to < LU_l cci Q 2: u_
i—
(
^i Z »—• X00•1 •~ o »- —vO CM \ ••k i—
(
u to s,CNI ^- ^~
» UJ Ul Ul v£) \ -~ — II
o _J • Ni to v•1 t- to o O a. ^v0 >—
1
to U lT\ U Ul•i h- o z a • — m DCo i—i 1—1 2: II r\i O • u.•l #> II h- 1— 1—
1
LU f\i •—I
o — to 15 3 < UJ or .-^ 1—
<
t- Zk o 3 Ul o H U in ti- < — too —
-
o < _) or _J 11 • *» X 1—
I
h- 0. H- Ul ll >- • -JUJ #• w s: z t— Q — 1—
1
1— rH -~ to_l o 3 h- < Q •- Ll to 1—1 w •~ _l uh- UJ < < to UJ lu a: u X — •> Q. tO u_ 2: — +•—
•
a: h- h- QT Q. O < X U z z u. < 1—1 2:1— u. * —
»
• u_ »—
1
Ul * ^~ m Ul < s * CO «>
1—
1
•> — r—
1
CC K-
1
to X <r 3 • X 1- •— M KJ Csl
•* t— o ~— cc X LU tO CL 1—1 < ro < < CL < * II
_l z c < o to a -* h- • T, _l * w -J ^UJ 1—4 r—
1
u 3 tO 11«— O U. < u + z 1—
1
CO cc -— Q x z U. ce: U. cc • z to 1—
< Q. o to y- z o — 3 r> 1- «* »—
«
21 II < +_J K-t UJ 0. *—t Ul 1—1 O 5: X u to 3 I—
*
2:•1 » cc S Ul 1— X >- c-\ Ul II tO _l u. 2: 2:o 51 u Q£ 3 < a >- X <» X 0- II < z \ 3•» 9- «* "X z h- _J < u. vO m m tO •-< 10 •— CC II
Q \— — tO C Ul LU cc —* Ul X 1—
t
• !-» a. 1—
1
II-~ r-
O < o •—
>
CD CC o CO * • cc z t» 1—
1
U ISJ
2: t— o Qi h- cc i—i v: * m ^~ —1 m Ul w to 2! — Ul 2H _) 1—
1
o < O o Q • u u. • 3 3 1—1 < a.
o Ul — U. _J z u to o 3 r-l CO LU • > 2: M to z 1—
(
to WLU a X LU 1—
1
0. r- u to X u. a < tO _l +q; • » 10 Oi u u. r- — m 11
--» 1—1 * z cc 1—1
u. < — 1— C£ < O U. t- II II 3 #> 11 Ul 2 z tft _l to 2: Ul Zm — O Z O Q. O Ul t- H- < h- CD x 2: Ii. to z II 3 3 tOX a O LU u to cc to z U H • 51 1- u. \ II •~ to _J to II
•> to iH SL — LU Ul LU < 1—
t
O 11—1 _l _J 1 z < 0. ^
M a ~" 3 o Ul CD _l KD O u •—1 u 1 <A 0- z tO 1—1 U3 —1 z NI
2: < O 1— z: 51 r- < i—t 1—1 X u. 1— to 2: u to * 1 to 2:
-^ UJ cc D H^ 3 +* _J X X ^ — O U. « u w m 1»
s z z < < r- z »- u. CO X X CO X X 1—1 • to tA nj + * • «— w CM< t—
I
o in CO CO m 1—1 CO — OO II -J 1—1 Ul _ II
a t— t>—
t
o ii II ii II II X U f—t —I to -J 3 *-r 1—
l
^cc 3 to z »— 1— 1— h- i— »- II to • u to z 11 X to
O z 1—
1
< 1- 2 Ul 1- < < < < < < 11 u <>J r-l II u 1-^ u_j cc UJ «J < -1 z 2: Z 5: 2 5: SI O Si II m w 1—
1
II r- —
.
Q II m_l O CD 51 -J r- r- i-i cc or q: Cr: or CC Z 3 1—1 _J II -J _l z 1—1 Z ^< Z 5 •—
1
< _J »—i cc O •I to to 10 O M 10 10 •s •—
>
to OU UJ to O C UJ i- a u. u u u u. u. u. < u u O < u u u X u. u
o —1 «-H
«—
t
1—1 i—t 1—1 t—
1
GO
1—
I
mrvj
in
COin
u U u u u u u u u u u u u u u
124
cO 1 •a •
u COa. O
X SL u Lc V< D2: a i— O zU_ i—
z LULU»-
LUX
O LU O Z*l 1- CL •> Z D »—
z cO > < O arCO cO i^ a: U 3
t-t * a OJ z uz -J u LU LU X • < LUCO ^ 3 X LU ISI
II CO c O LU z 1—
•
_l r-i U • CL CO Z a. LU _»
z z + < — Lc LU <CO CO
CO r-i •2:
>- Lc. OCO 2!
a: CO*A -1 U Z O O LU LU C 2:
*~ CO * CO LU O QL 1- > zr-i _J U -J * OC LU LU LU < < uCO w + U *£ Lc Z Z (D -J X » +U < t-t z CO 1—
•
LU Z D • »»
II * CO CO U r- < < U CO LU u.-1 N U 11 + < h- LU X _J LU •—
•
•»
CO < * r-4 rH z z u < •—
•
1
U + _J -1 CO >- •—
i
u u 1 LU5: Z *l U t- u Lc »- z 1
r-t or& D CO z * •—
«
LU —
•
— z LU CO + LCCO II CO i^ CO >- O Z «— —
t
D a X 11
v: < i-i z z 13 p-l u II NJ —»
z 11 _l (A CO LU C£ + >- LU isl < ^HCO 2 z II a LU 1 • 5: or • Z N» +II D CO *£. f-H l-t z »— —* a Lc h- O ^-J CO Z _J ^ _i LU 1 • X LU oc M • •—
v: < tf CO ^ z < CC + z -J s •-• Oz * z CO ar O 1 • 21 LU _» O II LUCO M f-1 1—1 _J CO >- Z 3: z> < >- • CO or
z _J ^ II w u t—
«
1 CO _J I— O 2: LC 1-t« _) CO z z _l LU Ll > < < Lc <— ll <
i/ * ifl CO ^ OD OD i—t 1—
t
a. ^ LD * H- 1—4 CO ^. u W h-^ CO _J II 1 z m m —* z ^ CO C + cn O O z •-i _JCO u z _l ^ CO ^ CO Z 1— 51 LU • 1— O LU ^ w >- LUu * CO z CO r- O C 1—
1
* cO a: • z CM • D O OII -~
1 CO U n in l- H »— hZ II LU LU •-1 LU + LC -~ r~i < 2; Z +-J 2: _l r-i * » < z v: 3 -J II «~ O t-H _J »- Lc LU -*
V •> cO CO M -1 1—
1
r» c + CO Z O 3 ^ CO —» 1—
(
— II < 4c x -uCO CM u U i^ _i in O 2 1 CO a. C£ i—
1
^ ••» Z < or M 1- V wu II + * .—
<
CO ^ UJ p D l-i U- — X C "V O »- < ISJ - D_l _J _l U CO D b=» tfl CO CO W LU • — s: X w •—
<
>• _l u O r- ^ <*> • CO CO II u Z in < u X 0- v/ • + * y— O D LU • -J * *- t-
lf> 1—
1
O U rH 11 »—
«
2 5: — * fH 1- < LU w rsi 2 m U z a O LU f-t X II
If w- II II _l _i 1— — UL Lc * v: ^ LU en O X O U D LU • < LU X CO II a II * —• II t—
<
-J v: ^ z N N, \ fvj CO CO 3 co r- z • v: CO z> oc X CO —
»
* ±C r>
M _J cO CO cO O 2: O m Q <J U z •—
•
< 1 O < z 11 Lc II z «-» < +II a < CO U u u u 1 • • K II M >- (X 1 II vO II •—
•
>- r— T— t—
I
•M • CO II ^3l VJ ^ 1—
1
O —
»
i-h II I— «— _J LU Z vO 21 I— O O u u < <n «o *^ »-.
r> »— II 11 V ^ V z *: a. OC r- D D z Z z < < 1— LU 11 ^ r>CO u. M M w CO CO O •« Q- O O CO O CO O LU •—
•
Ll Lc CO a: m ^ << •—
1
a X u u u X < Lu Z < O < u X Lc X X Lc < X t-
* in r«" CO in vO r-iTv in m in m m O <o
u u u-u u u uuuu
125
coIdI-->
o.1
•
-» CO zISI o os: l < 1— t
* Ll. 5-( Ll. LUn * O z <^ O•> vO CO Q CO—
»
1 h- Z . •—*
5-f LL. O <•_• ~ o *—
X on 3 *»
•> O m—
-
00 o H- vO^ Z> < CO 1— co 21 or Ll.
1 o _l •- •—
i
•»-
1 LU CL <1 On h- o CD10 Ll on z 3z *
1 < O <o «— CO z on»—
1
^ e? _J »—
*
o1— w a. < CO i
u < Z LU D toD «
i on aq: «- r\j CO z»- 5^ u a h- i •to •W z z r- luz D i
-- LU ~> z1—
<
< Ll. •— H-4
~» H o u 1 t-1— M «-» *-H O l-H Z>D s: * 1— Ll. a oO — D_ Ll. •- a:
1 o o r- O LU LU CDLU ~ LU <r o 1 Z O 0l DI— r-t C£ •> i-H o •- u O coHH — ll. i—
1
KJ 1-1
CrT < <f y- — D _l O l-5 • » - z O < U Z)
t- CO o •— qd on LU ~ CLs: u s: <r on Z> CD on l-3 < o a. CO ID 3ct: u_ u -— _J to x c 3: O- X •H W Q. »— Q <u * »
1 h- Q »—
<
o cr: XLU LP, CO t- vO ce: a: S X O Cl
LU a o o z o LU < 1- <3 CO f-t «-t o -« l-H 3 LU O _i LU lu ct:
z II q: Z z z < 2 z o•—4 a: 1— 1— t— a k- — I-. < —I •—
I
t— LU z z CO •- z h- H- h- 2: z i- _jz 3 1-^ •—
1
LU «-^ —
»
Z Q D to O o r> <o o en on 1— Ll. On O Z o z CO O a:u Q. a. a ~) -> Q. U LU on <
CD
>-
_jXQ.
q: luCO z
CO f-i O D co O < 3 UJo <t <* tO •"-• 0. on CO KD
u U u u u VJ u U U
126
2UJ1—
< 3 1•
<tO
or UJto
to*—
(
a t—
1
X UJ«— u. -J O r—
t
1 z b 0. oO ••
.* UJ or — »—1 t—
1
u < 00 —o a z to _i or or UJ (NJ
vO tO UJ UJ •-< UJ u 1—
1
rH UJ_J2 UJ
1tO
D_lQ.^ > u.—
X1— UJ or
•>
(M
u a Uj h- _i >: _j or uj or a f-H
Q • 2: > Q. < a < —U < •—
1
U. to < • 00 ^-4 •> •>
CO (/) O _j z to 2: 00 O UJ X — ^»
Q£ 00 < < z _J UJ _j CL «r»
O (\J O UJ z O tO UJ •—
•
0. •1 VtU. m O or z •>—>—• Q or 2 — l-l f-l M
O 1—1 O 1—
1
0- >- h- O 1— < •r^ w —
>
•CO 1—i UJ *—
t
tO •^ 1- < Z 2 00 a 0. «nO m u. OT r—
UJ U— l-OZ _ —i 1—
00z u_ < CO
• II
z in H z X u DUiO Z — • •> rH O «—
< 11 to 3 3 r- X —
•
UJ •—
•
a ~- ^. UJ UJ _ior m z < u. _J Lt_ UJ > or i— or rvj <— to UJi— <J- 1- CD C UJ or UJ •1 • a to
> or >«» •—1 O _J z _i or u. O cc —
»
t—1 t—t _j UJo UJ 1— z < UJ < uj uj a. u_ a 2; rvj •» «— II CO u 2
X u. •v u r—
4
a tO h- 3 O •—
<
z> »• to tor-« tO z to •—
•
UJ < -1 Q or z O _l _J X or UJo z CM 3 3 O or Q < 2: • a O < CO ~. UJ «fr a toz ^^ N* u. z 0. > h- O UJ O » » > « •—
UJ 10 • •—
<
UJ 1- or z 3 to 3 <r ~~ ^« 1- X UJa. z r-H Q < or or 3 UJ D t— => H — rvj <\j 3 21 CO CO zQ. UJ "V O I— a. a or u. < rH r-l a uj •< H- CO O < z < 2: X 1—
«
u. i- «^ w z or tot- *X I a < i— »—
1
DO< >— or < < CO •—
•
-1 H- z—
«
M O UJ •—< < 000 UJ CO 1—
•
or <f _1 to tO UJ tO t- tO 3 > z v »-H 1—
(
< C0 tO —S ^ UJ < • _j X < — X X r- < X UJ 3 fM
^ _i UJ r— Z 3 X < or •—
1
— » * < or «3- _l •2: v. 1—4 r— U) < _l •- Y- 1— r-H ^. *^ V a uj •k 0. O vO< 1—
1
_l 3 Q UJ UJ < < u. u O r\i co u > X 2: O u.or V. a O 2 u 2: > u. Q z O r-l f-l z < CO < O •
•-* to z 2: z UJ to D r—
1
•> « «-» _J r-l > to XUJ 1—
1
It- UJ < i— u. <\J CM CM CO UJ tO »~ •-< II
QT m t- 2 CD X UJ to 3 z ^ r-l 1—
t
rH H-
<
z or _i UJ CM u. -Ja 0. 3 to < uj or z a. < u Mi W w •> » 0. -J • UJ Ow a UJ or >- > < z »—
•
< CO UJ < or -~ — or 2: CL C \i —X 2: —
1
h- • < -• »-^ t- 00 _l _J r- » CM (M or < 2! rH or *H t-h- QT O 1—. a X H- IJ < CO <—t 1—
1
t- < r—
«
-~ • • UJ to < UJ z _J <CO 00 u UJ 0T _l UJ to or UJ z —1 0. a 1-— X O O — — to vO or
UJ tO 0- •—1 > z X r> a * • 1- V a 1—
1
<—
1
rH vO X X 00 » X X3 1- 2: CO I—
1
r- O CL h- u. UJ z »— »» •-H \ V. •—
»
UJ UJ r—
1
rH sO < X <r CM X< UJ >- < UJ h O H CM CM « < CO <J- CM CO O CM CM vO co •—
1
m CM CO-> • 21 D 2 co u 1- or O 1- ^^ f-t r-l — V, s, »* w ~ ~- w. m* — w »* •htf M >—
z < —
.
r- UJ uj u a Z O to •a« w O z z r— 1— V- r— 1— f- I~ H- r— H- r- r-UJ a t— to or NJ z «-^ _J z < CO O oo<<<<<< < < < < < <a. or QT UJ UJ » 3 to tO X Q, UJ UJ UJ f—
1
2: 2: 2: 2: 2: 2: 21 2: 2 21 2 2: 21 Z•>• a 2 z h- >- _J U. 3 tO »—
t
2: UJ UJ *-* 2: 2: or or or or or or or or or or or ora. UJ or O »—
«
e? < UJ _l > •—
>
X X i^OOOOOOOOO 00000» •1 a. tO O < O or u UJ _J XUUIi. u. u. u. u. u. u. b. u. u> u. u.• _j •—1 Z H- 1— UJ or or ^ _J rH CM «oa •1 X < co 00 z < or •—> < O >-> CM CO »* m 1^ CO O »H
z »- Q •— UJ 2 _l W vT> m m in m in m in in iri toi—
u u.
1 1 U U u u u u U KJ u u u
127
QUJtO00UJuoexq.
UJ0000UJuoccQ.
to_) UJUJ CDoo o
tO i/)
h- tO UJCC UJ _J< U 0.t- o s:to or <^ 0. 00
UJ < Oo z•> •-• UJVi tO COz 2:* u. z>z o zO CC tO
Qq a: _i< O UJ
UJ UJa co
I- otOZOD t-
z ^Q «-• Uz z o< — _J< CO
» I—UJ CO u.i- o o<CC Z UJ—
I
_lO 0_Z Q 51.-. UJ <_l 00 to0- Ds: i-< _J toto uj a:o —
tO ^ u.I—UJ u. u.00 O O>- UJ CC_J D UJUJ —I CO> < 51— > 3H- ZV xUJ t— to
oO
z z
UJ cc_J UJ <uj cc t-
^ < ^
z z
UJfsl
— cco
» z
.- z» <
< COUJ UJUJ UJX Xa Q.
•- •— V.
CM CMm in
a a< <UJ UJcc cc
<»— CM< -
II
_i to< --
z to
»—
I
</) oo
Z f-<
o<
<-Iu.
to
X<51
Z rHLU »
< ~
00 _J-> _J_l <
<-Iu.v:
oO UJ<-> Z> II
Z • ^o — o -t— I— II CO >y£
z > ~o o < o o
< —I- o
fsl
+><fsj
II
><
<oII
u. cc
X<X
X >s< >51 <II ISJ
X II
< >T. <X fsj
><
m
>< C\l
fsl #X I #< .
s: ^ ^
occ
• X1—I 1—1
V (X— I—< <
tO X
u. s: ^ u
z o•-• II
cc oCL CC
a oisj isj
II +~ o^ ccw ||
Q Orsl CC
- OI- CC 21Z II oo o —u cc CO
r-t CM en
CC
O tOrsl <w COII
-~ UJ^ I-— <O CCfsl UJz
O UJ
u u u u u u u
128
oN*r
ooCM»ON
vO CM
II r-•-< II
UJ w_J UJ
CO<-H O* •
h- II
<-* CC•> tft
vOi-H
— .—
i
m •
»—
i
X» >*
<J- it
i-H CO•> <
CO _i<-H t*
co•
ii
orw
(M
XII
CD<
r«- vO m •d- CO CM —t• • • • • • •
ii II ii ii II II ii
cc a on cc cc or CC(A w tA w (A w W
CO * m *o r~ CO O^• • • • • • •
X X X X X X X«d- >j- •al- «* «* «<r •d-
II ii ii II II it 11
CO CO CO CO CO CO CO< < < < < < <_J -J _J o _j _j _J(A & W tA <A *» *ft
<o
<
UJxa
•
z z o• * II
r-» r-« —II II "">
CMII
O II Xo »- ao o xo o a.
• »- »-• zii — *-• •«-» » «^ On~> vo • in• in vo
•-< in j—— O a z< < •-UJ uj ccor cc o.
cm
UJ <-\ ONi-H
o•* O O MI— I— • H
ao
in
O O
O COcm co
coO O»— •
li
O <o a
O fM ocnj m cm
o *-* o
in i—
•
CVJ i-H
CM
II II
< o <
en ,$
O O OCM On CMO
O CM O»- • I—
II
o < oO O O
in
mnj•
ii
<O
o m o oCM CM CM «3"
O COO fM O CO
II
o < oo o o
II
<oCO
o oCM O
f-O co•— •
II
o <
UJ3
>- zQ •< <-•
UJ II
1/5
On
zouoCM
_J CM-J< o
129
to<_J
\C »w rHQ. •-J r~
< »* in•_ « "
•—*•
t0_J <—
1
#
< 11
» *-* »oc *-» —
4
+ o —
-
•
"— v: \£1—
1
— X » Om Q «A ^
i—
i
ISJ #* »-«
»- •> UJ O 00 r>-
< < — _i O O • • •
O _l 1—
<
1— rH • 11 11 II
r-( rH <— •—
1
* II DC cc a:0. 1— • a 1— O cnII M -j •—i cn<
<<+ CD
r
00yf> *r> (r> &
kfil Oi—
i
Xm
r-z <S)
<_J
O
en*r-CO i-t CM CO •*
«~ < «— _j •» vO CM • • • • •l—
1
O a < O « rH vO— rH _i 11 O t—
1
• m X X X X< Q. < «-» X II I'- * <J" 4- >r *o * * r-t •> )-4 ll in 11 II 11 11
1—
1
< — ^> "-- U) •> >-^ CO CO CO CO COX
1 < » i—
i
i—z
to_J
_i< 1—
1
» 9> <_J
<_l
<_i
<_l
—» V. < 1«— < » w H-
<
COh- (XI co LU ^ o to <A ^ UJ w *.
ct: * ~-1 UJ .- 1—
(
_j X _J UJ CO w (A (A tr>
< id < ^ X 0. r- < » r- _l COr~ < ~^ r- * a. _i < » r- r- 1—
1
1— r- ml/) _i U. Z m ~ < C£ i— Z Z I— l-H Z CM
• 2*£ iH o •• + a » en z •fc »-» •-H r- COo — r- o (XI i— s: ~-
I* •v rH s — r«4 • *
II c N _J II (X O rH UJ • O i—i r- 00 II UJ < in UJ * ~- O UJ 1—
1
>3" vO O lA-~ M n II ^ < U 1 D rH ^O *o m m 1—
<
3 cc t—
I
<M z> V0 * lT> D CO r- <r <r <r CM ^- O— II—
~
—
-
i— _J ^ 2: II Z Cl + Z lT> sO Z >• <r «* CO— < i—
i
T—{ (M </) w i—i O 1— 1— H- H- CO 1—
•
O •—
1
m r- >—
1
rH O (Nl O t-\ O rH< -J ^< -— (\J ^ -I to i— H—
|
z z Z Z (M r- _J ^ r- 1— O Z rH r- r- • II 1— • II r- • t— •
O rH (/) Cl II _i _l z 1— 1—1 »—
t
r—
«
t—
1
z _J II z < < » 1 11 Z n O 11 a II II
rH »- _J -J o Q < < o < q: a: o; Ct O O < UJ uj q: O O O CD O O CO O CO O COX INl < < G i^ u • u OL Q_ a a. a O U u ^ O u OH ct a V U O O z: O 5:
c-t
f-H <\J CO vO T-* CM CO «3-
CM <\l CMCM
CM CO CO CO co
130
CL-J03»
to_J03
X
to<
»-
mno»o
or a — *+ ^ a o*-» «— _j *t-i o < o•> (SI + *
t-t « —
•
UJ»- CD —
i
-ICD -I >-» h-O i-t lO I—
I
i-H h- _i h-<t a Nl < •-^
• ii » ii •m II ro CM i-i < CO »-» CO• a: • • • o •—
i
<II <A ii 11 it ~- <A l-» «» w -1Oi Q£ a: CL C0 X 1- t0 •M M & W o
1—
•
0. i-<
*COo
z0.
_J<
oam vO r- CO O » w l-l _J W o
• • • • • CO CO a CO 2:X X X X X o o * * —» »<t <t <t <t <t » r-4 CO *-> ~ »—
1
toII M ii II II CO X a ^. »- —
'
_iCD CO CO CD CD UJ 1 < # r-t z a CO< < < < < LU «-. \ CO 1
*— _i »_J _J _l _J -J X 1- (M CO LU * o CO CD ^M w (A (A *A a cr * —.
1 UJ «-• i-H -J + Xw < * < ^ X a »— CO *~ •1- CO i
—
1- * a _i < * »— ->- tO _l U. z CO •»* ca Of >- 1— •IB* za z • ^ r-t O 1 + 0. • Oi z z 00 w.
< » O «-» HOM i- z: «-«1
» 01 * _J 5o o o in o O LJ UJ r-H II o M _l II a: o i-t UJ • o f-H r- CO i-t CD UJ < O UJ<t h- <t <t vi- ^-t <t o ,3- vO Z> »- II —
»
M II II ^ < u 1 D t—
(
vO vO m in II II d or *-• m 3CO 00 eo t^- CM CO Z 03 •—
*
•—
«
II^~ t- -J ^ z II •-^ •~ z + zON ONO co O co O <tf" II >—
i
*- CO i—
t
t—
i
CO to »—• fc—
«
o 1- i- t- h- •—
•
>—
1
00 —1— • - • t- • H- • H- • o l- -J r«- CO _l w «— \i -i to 1- »—
<
z z z z o ^* V- _l i^ *- H-II II ii II II o 2 _J o »-t to a o ti _l -I z 1- •—
t
•—
«
•-H t—
I
to Z -J II Z OO 03 O CD O CD O CD O CO 2: c 1—1 h- JJO a < CO O < or or a: ct o -J O < O O O Zo o o o o o o o o o U u Q X M CD CD ^ u *ut>ra.Q.aQ.O(Cuu^ouuj
in NO r- 00 o O r- 00 0^ Oen en CO en CO <t O
131
<\J
a._jxi—i
to_lX»
a_jx
(NJ Q. »i—
1
o to— to _Jr- Xr- Ll »* o i<i— Z M
rsi X *> »rH H--4 r-H
* a: II _Jf\l r- M rHi-l < r-~- s: in Ma •>
H- > *». O -— -~ O«— « o v: Q -) ~) r-<
o —
»
u <r- •> • X1—1 f\) -> tA ^ ^ •>
a. rH UJ >— •— •^ ot> •» r- a r- 0- Z rH
O CM < z i— Z h- r— » — CLi—
1
t— * -~ z * >« * z z * r-H Z Z »UJ —
•
to —
1
Z fjb «-« f—i ^-* •• > — II •> « oUJ a 1 II * rH ^ II ^ rH r-H ^ ~J i—
i
i—
i
•»
I •i r- >• -) t-t II •> ->II II * II II LU
Cl »» *- Q 11~> »—i •- ~> v: •—
*
o ~) -) LU1—
i-t c\j
-—
*
<LU
1—1 ->
•> >t oiTi
Q. vO •o 0.
1—
1
r-
* XCL
>- •> r-t -~ —
I
(— o — i— O f- r- o ~J wO r\J » z <N X 10 o <~ ") o + o + o o + Q o — « a< —
i
C\J » • a ""> »— » a •^* — Q Q mm, »~ Q ~> — zLJ — f-H cr O X t* •—
«
»—
i
-> -> ~> tA ~> •> -» or- »—
i
•^ >. rH X f—l fee *—
<
— w o tr) »• (A * 4r) M •v •> tr> l—
4
o uCO 1 i—
i
LU <r CO «-» LU 0. 1—1 »—
i
•-* 1—
1
1—
1
— rH -JCi_ i-H X CO *• » «—
i
LU H- • M •^ «« Z — o CLLU a. a 9 X X z • X z X z «— 1—1 z • Z 1— z o z • t- •> r- z H z «-» LUZ z » \ X o o •> o 0_ o • a •* •> II * o i r- •> 1— • o r- i—
i
r- k II •> r Z*—
i
o »• Cj t—
i
<\J 1—
1
1—
(
II II II rH ii r-H i-H O *l r-H II p—
I
II 1—1 II rH II II ll II i-H ^~ (M 1—1 o »—
i
r- »—
i
r\j *s -~ — •w II«» «— — II
«-~ lT\ II m i/ II—
.
II —.II «~ II »". -~ •—
i
«~11
~) tTi II »n D-D in r-l z r- 1— t— *—
1
-> -) Z »-H •—I •—
*
»-* -> ^ -) •—
(
""> H -J ~> ~i r-H *> •—I DO z — o < < < p f> •» 9> r- r- o * •> 1 •> * o 9t —
i
r— r— Oor LU LU •z. s: 5: 5: 1—
I
»-—
t
1—
1
Z CM z Z m Z rH d- 1—
*
<r t—
•
lT\ —• \£) 1—1 •—
i
rH i—
i
r~ — Z 00 Z cacd s: LU s: a: a: a: — — «— «— •—
•
—
H
•M> «^ M ^-- w -~ O —
<
J— o COZ> >—
i
X o o o o o o a. o o CL a: o a: o c r- O r- o o o t- r- O Cl O r-H 0£ O Q z DtO O Q. u u. u_ li_ Q r- r- r- Q t- o_ o Cl o Q r- O r- o 1- o r- t— O i- q a CL Q CL LU to
r—
i
o rH
inrH (\J CO Sj" \C\ vO o
1—
1
r- 00
132
CM1-4
CM
CM
COX
CM
— OCM »iH -»— CMUJ <-•
UJ •X cmCL —« «-»
-~ CLfsj *rH *—•> CMCM i-t Z— O DC»—• «—l »XX —a •> xZtVJSO <-• CO— * >•»
lO CM zZHOUJ-Sso 2:•—
1f—
t o
DC+
1 ~i -J X •» *-• ^- Q. i-W t—
I
^ f* »- a. * — a.ct ~- to * — o -J+ X — Z Z _J fi X^ Z — _J * _l • * * CL +
ft — • H H -• II
I- -> — II II — <IM •> «- ~*> _J I—# — — I- O W—• — X m m + +
X + O O *"> ZO Q. ~ o O * • — ~+ + ~> — Z -~ < <—1 ft •- OO— u. u_— w .— • — «-• o tD Oz zo »z»-zocLa — 00• t»HO*t-»ll II II X J JH O H X II H II H ~| +11 11 11 11 — 11 -» 11 —> ~~> z ^ a_—.-}-»-)_J-^»-. « • *fsj_l
f, _ » » MM Z II X— t- o o — — o o o «- a. a.O-OWOhOhi-Oi-OhhhkjJoaaxoxoxi-Qi-QCLaafsixx
— cm en -4- «r>
• a.
11 -~-> —fM O
I
o ~
w —z o• 1—1
— a.11 11
o1—
»
— a— ^LU -~
UJ —X •
z a. —» II w— —. o— • a — ~ — —z 11 -> —
ft — — fM • fn «~
o x -q a o
to_ix+
< <
z o• I—
I
— XII II
CM*
_l
M rsj
00_JXII II
to to_l -J zX X UJ
133
BIBLIOGRAPHY
1 . Anderson, "W, W. , An Introduction to Multivariate Statistical
Analysis , John Wiley & Sons, New York 1958.
2. Barrett, N. A., Extracting Analog Signals from Noise Using a
Digital Computer, (Appendix I); Unpublished Masters Thesis,
U. S. Naval Postgraduate School, May 1966.
3. Bartlett, M.S., An Introduction to Stochastic Processes , Cam-bridge Univ. Press, Cambridge, 1955.
4. Blackman, R. B. and Tukey, J. W. , The Measurement of PowerSpectra from the Point of View of Communication and Engineering,
Part I, Bell System Tech. Jour ., V37 1958, pp. 185-282; Part II,
V37 1958, pp. 485-569.
5. Bona, B. E., Optimum Reset of Ship's Inertial Navigation System,Advanced Analysis Dept., Autonetics 1965.
6. Bode, H. W. and Shannon, C. E., A Simplified Derivation of
Linear Least Squares Smoothing and Prediction Theory, Proc.
IRE, April 1950, pp. 417-425.
7. Bodewig, E., Matrix Calculus , Interscience Publishers Inc., NewYork, 1959, 2nd Ed.
8. Davenport, W. B. and Root, W. L., Random Signals and Noise,
McGraw Hill Book Co. Inc. , New York, 1958.
9. Deutsch, R., Estimation Theory , Prentice Hall, 1965.
10. Fagin, S. L., Recursive Linear Regression Theory, Optimal
Filter Theory, and Error Analysis of Optimal Systems, Sperry
Gyroscope Co. Inertial Division, Tech. Paper , April 1964.
11. Grenander, U. and Rosenblatt, M. , Statistical Analysis of
Stationary Time Series , John Wiley & Sons, New York, 1957.
12. Grenander, U. and Rosenblatt, M., Statistical Spectral
Analysis of Time Series Arising from Stationary Stochastic
Processes, Ann. Math. Stat. , V24, pp. 537-558, 1953.
13. Hannan, E. J., Time Series Analysis , John Wiley & Sons,
New York, 1960.
14. Helstrom, C. W. , Statistical Theory of Signal Detection , Per-
gammon Press, London, 1960.
134
15. Ho, Y.C., On the Stochastic Approximation Method and OptimalFiltering Theory, J. Math. Anal. & Add. , V6, pp. 152-154, 1962.
16. Ho, Y. C. and Lee, R. C. K. , Identification of Linear DynamicSystems, Cruft Laboratory, Harvard University, Report No. 446,May 1964.
17. Hoel, P. G., Introduction to Mathematical Statistics, John Wiley& Sons, New York, 1962, 3rd ed.
18. Kailath, T. , Correlation Detection of Signals Perturbed by aRandom Channel, IRE Trans, on Information Theory , Vol. IT- 6,
pp. 361-366, Jun 1960.
19. Kalman, R. E. , A New Approach to Linear Filtering and Prediction
Problems, J. Basic Engineering, ASME Trans ., Vol. 82D,
pp. 35-45, Mar. 1960.
20. Kalman, R. E. and Bucy, R. S., New Results in Linear Filtering
and Prediction Theory, J. Basic Engineering, ASME Trans., Vol.
83D, pp. 95-108, Mar. 1961.
21. Lee, R. C. K., Optimal Estimation, Identification, and Control,
MIT Press, Cambridge, 1964.
22. Middleton, D. , An Introduction to Statistical CommunicationTheory
,
McGraw-Hill Book Co. Inc., New York, 1960.
23. Middleton, D. , Topics in Communication Theory, McGraw-HillBook Co. Inc., New York, 1965.
24. O'Donnell, C. F., Inertial Navigation Journal of the Franklin
Institute, Vol. 266, Oct. 1958.
25. Parzen, E. , An Approach to Time Series Analysis, Ann. Math.Stat., V32, pp. 951-989, 1961.
26. Peschon, J., Review and Extension of Real-Time Estimation
Techniques, Stanford Research Inst. Paper, 1965.
27. Price, R., Optimum Detection of Random Signals in Noise with
Application to Scatter-Multipath Communication, I, IRE Trans, on
Information Theory , Vol. IT-2, pp. 125-135, Dec. 1956.
28. Rauch, H. E., Solutions to the Linear Smoothing Problem, IEEE
Trans . On Automatic Control
,
Vol. AC-8, pp. 371-373, Oct. 1963.
29. Rice, S. L., Mathematical Analysis of Random Noise, Bell
System Tech. Jour. , V23, pp. 282-332, July 1944.
135
30. Rosenblatt, M., On Estimation of Regression Coefficients of a
Vector-Valued Time Series with a Stationary Disturbance, Ann.Math. Stat. , V27, pp. 99-121, 1956.
31. Schweppe, F. C, Evaluation of Likelihood Functions for
Gaussian Signals, IEEE Trans, on Information Theory, Vol. IT-11,
pp. 61-70, Jan. 1965.
32. SIMS Reset Report No. 3 , Report prepared for Special Projects
Office, Bureau of Naval Weapons, Dept. of Navy by SystemsManagement, Marine Div., Sperry Gyroscope Co., Mar. 1961.
33. Trench, W. F. , An Algorithm for the Inversion of Finite
Toeplitz Matrics, SIAM Journal, Vol. 12, Sept. 1964.
34. Turin, G. L., An Introduction to Matched Filters, IRE Trans
.
on Information Theory, Vob. IT-6, pp. 311-329, Jun. 1960.
35. Weaver, C. S., Estimating and Detecting the Outputs of Linear
Dynamical Systems, Systems Theory Laboratory, Stanford
University, Dec. 1964.
36. Whittle, P., Hypothesis Testing in Time Series Analysis,
Almqvist & Wiksells, Uppsala, 1951.
37. Woodward, P. M., Probability and Information Theory with
Application to Radar, McGraw-Hill Book Co. Inc., New York, 1953
3 8 . Wold , H . , A Study in the Analysis of Stationary Time Series ,
Uppsala, 1938.
39. Wold, H . , Demand Analysis , John Wiley & Sons, New York, 1953.
40. Wold, H., Ed., Econometric Model Building, North-Holland Pub.
Co. , Amsterdam, 1964.
136
INITIAL DISTRIBUTION LIST
No. Copies
Defense Documentation CenterCameron Station
Alexandria, Virginia 22314 20
Library
U.S. Naval Postgraduate School
Monterey, California 93940
BUSHIPSDepartment of the NavyWashington, D. C.
Director, U.S. Naval Security Group3801 Nebraska Ave. , N.W.Washington 25, D. C.
National Security Agency (Attn: R42)
Fort George A. Meade, Maryland
6. Dr. H. A. Titus
Department of Electrical Engineering
U.S. Naval Postgraduate School
Monterey, California 93940 50
7. LCDR John W. R. Pope, Jr.
NAVSECGRUHQACTY3801 Nebraska Ave. , N.W.Washington 25, D. C.
8. LT. C. E. HeckathornUSS Tiru (SS-416)
c/o FPOSan Francisco, California
137
UnclassifiedSecurity Classification
DOCUMENT CONTROL DATA - R&D(Security ctaeeitication ol title, body ol ebetrect and indexing annotation muat ba entered tehan the overall report la claeellled)
1 ORIGINATING ACTIVITY (Corporate author)
U. S. Naval Postgraduate School
Monterey, California 93940
2a. REPOKT IECUHITY CLARIFICATION
Unclassified2b croup
J. REPORT TITLE
Real-Time Estimation and Random Signal Detection
4. DESCRIPTIVE NOTES (Type ol report and Indue I ve datee)
Doctor of Philosophy Thesis in Electrical Engineering - May 1966S- AUTHORfSJ (Laat name, Hrat nama, Initial)
Pope, Jr. , John W. R, Lieutenant Commander U. S. Navy
6. REPORT DATEMay 1966
7a- TOTAL NO. OF PACES
136
7b. NO. or KIF|
40
8a. CONTRACT OR GRANT NO.
b. PROJECT NO.
9a. ORIGINATOR'S REPORT NUMBERfS.)
9b. OTHER REPORT no<S) (A ny other number, that may be aeelgnedthte report)
10. AVAILABILITY/LIMITATION NOTICES JThis document has been approved for publio
relec on is unlimited!
IMMMBJtitfrtirttftis repai 1 r nir- TV^ %Uo\io
11. SUPPLEMENTARY NOTES 12. SPONSORING MILITARY ACTIVITY
BuShips
13. ABSTRACT
This work investigates some problems arising in application of (Kalman)
linear filter theory to real problems, where practical estimates must replace
exact theoretical quantities in problem formulation. The principle objective is
application of linear filter theory to random signal detection/classification."
Howdver, an example of classical estimation, error estimation in shipboard
inertial navigation systems, is offered to illustrate general points discussed.
A unified treatment of models for random time series is presented, including a
comparative review of models which have been proposed and procedures for
obtaining model coefficients . Correlation detection of deterministic signals
is discussed and the resulting principles extended to the case of random signal
detection. Application of linear filter theory to the problem is indicated.
Finally, an experimental study in random signal detection/classification is
included. Experimental signals used are hydrophone recordings of sea noise
and sea noise plus diesel submarine. Consistency of successful results ob-
tained suggests practical utility of method in certain random signal detection/
classification problems.
DD FORMI JAN 64 1473
139UnclassifiedSecurity Classification
UnclassifiedSecurity Classification
14KEY WORDS
LINK A
ROL(LINK B
ROLI *TLINK C
ROLE
Estimation
Detection
Pattern Recognition
Correlation Detection
Random Signals
Classification
Time Series Analysis
ModelsStochastic ProcessesTime Series ModelsAutoregressive
Moving Average
INSTRUCTIONS
1. ORIGINATING ACTIVITY: Enter the name and addressof the contractor, subcontractor, grantee, Department of De-fense activity or other organization (corporate author) issuingthe report.
2a. REPORT SECURITY CLASSIFICATION: Enter the over-all security classification of the report. Indicate whether"Restricted Data" is included. Marking is to be in accord-ance with appropriate security regulations.
2b. GROUP: Automatic downgrading is specified in DoD Di-rective 5200. 10 and Armed Forces Industrial Manual. Enterthe group number. Also, when applicable, show that optionalmarkings have been used for Group 3 and Group 4 as author-ized.
3. REPORT TITLE: Enter the complete report title in ail
capital letters. Titles in all cases should be unclassified.If a meaningful title cannot be selected without classifica-tion, show title classification in all capitals in parenthesisimmediately following the title.
4. DESCRIPTIVE NOTES: If appropriate, enter the type ofreport, e.g., interim, progress, summary, annual, or final.
Give the inclusive dates when a specific reporting period is
covered.
5. AUTHOR(S): Enter the name(s) of authoKs) as shown onor In the report. Enter last name, first name, middle initial.
If :r.il it ery, show rank and branch of service. The name ofthe principal author is an abaolute minimum requirement
6. REPORT DATE: Enter the date of the report aa day,month, year, or month, year. If more than one date appearaon the report, uae date of publication.
7a. TOTAL NUMBER OF PAGES: The total page countshould follow normal pagination procedures, i-e., enter thenumber of pagea containing information.
7b. NUMBER OF REFERENCES Enter the total number ofreferences cited In the report.
8a. CONTRACT OR GRANT NUMBER: If appropriate, enter
the applicable number of the contract or grant under whichthe report waa written.
8b, 8c, « id. PROJECT NUMBER: Eater the appropriatemilitary department identification, such as project number,subproject number, system numbers, tssk number, etc
9a. ORIGINATOR'S REPORT NUMBER(S): Enter the offi-
cial report number by which the document will be identified
and controlled by the originating activity. This number mustbe unique to this report.
9b. OTHER REPORT NUMBER(S): If the report has beenassigned any other report numbers (either by the originator
or by the eponaor), alao enter this numbers).
10. AVAILABILITY/LIMITATION NOTICES: Enter any lim-
itations on further dissemination of the report, other then those
imposed by security classification, using standard statementssuch aa:
(1) "Qualified requesters may obtain copies of thisreport from DDC"
(2) "Foreign announcement and dissemination of thisreport by DDC is not authorized.
"
(3) "U. S. Government agencies may obtain copies ofthis report directly from DDC Other qualified DDCusers shall request through
(4) "U. 8. military agencies may obtain copies of thisreport directly from DDC Other qualified users *shall request through
(5) "All distribution of this report it controlled. Qual-ified DDC users shall request through
If the report has been furnished to the Office of TechnicalServices, Department of Commerce, for sale to the public, Indi-
cate this feet and enter the price, if known,
11. SUPPLEMENTARY NOTES: Us* for additional •xplaaa-tory notes.
12. SPONSORING MILITARY ACTIVITY: Enter the name ofthe departmental project office or laboratory sponsoring (pay-ing tor) the research and development Include address.
13. ABSTRACT: Enter aa abstract giving a brief and factualsummary of the document Indicative of the report, • ven thoughit may alao appear elsewhere in the body of the technics 1 re-
port If additional apace ia required, a continuation sheet shallbe attached.
It ia highly desirable that the abstract of classified reportsbe unclassified. Each paragraph of the abstract shall end withan indication of the military security classification of the in-
formation in the paragraph, represented as (TS), (S). (C), at (V).
There is no limitation on the length of the abstract. How-ever, the suggested length is from 150 to 225 wens*.
14. KEY WORDS: Key words are technically meaningful termsor short phrases that characterise a report and may be used aaindex entries for cataloging the report. Key words mast beselected so Chat no security classification is required,fiers, such as equipment model designation, trade name, militaryproject code name, geographic location, may be used as keyworda but will be followed by aa indication of technical con-text. The eaaignment of links, rales, and weights ia optional.
DD /,?!., 1473 (BACK) 140 UnclassifiedSecurity Classification
esP741
Real-timeestimation and random sign aid
llliIW»«> v
,2768 000 99283 8
Dudley knox library
''i
KSEffl
WH
nJTOWlWfflm
mmimwki:HHI8§Pmm mm. mHfi
Warn™'.f )''--M U
8Hffi
.; mmII
Mmmmm
MffiR
