Pergamon

0967-0661(94)00047-6

Control Eng. Practice, Vol. 2, No. 5, pp. 889-897, 1994 Printed in Great Britain

0967-0661/94 $7.00 + 0.00

TRACKING AND FUSION USING MULTIPLE SENSORS*

K.C. Chang

Department of Systems Engineering, George Mason University, Fairfax, VA 22030, USA

Abstract. Multisensor tracking and data fusion deals with combining data from various sources to arrive at an accurate assessment of the situation. Difficulties in performing multisensor tracking and fusion include not only ambiguous data, but also disparate data sources. The tracking and data association problem is further complicated by the facts that the target may not be detected by some sensors, dense false alarms and clutters may be present, and the target model may not be known exactly. In this paper, a multitarget tracking problem that involves data obtained from multiple MTI (Moving Target Indicator) sensors is considered. A tracking and fusion algorithm that takes into account the uncertainties in both data origin and target dynamic under the clutter environment is presented.

Key Words. Target tracking, estimation, data fusion, MTI radar

1. INTRODUCTION

Tracking and fusion with multiple sensors has attracted a great deal of attention recently (Blackman, 1986; Bar-Shalom, 1990a; Bar-Shalom, 1992). It deals with integration and correlation of data from various sources to arrive at an overall assessment of the situation. Difficulties in performing multisensor tracking and fusion include not only ambiguous data, but also disparate data sources. First of all, the identity of objects responsible for each individual data set is unknown so there is uncertainty as how to associate data from one sensor which are obtained at one time and location to those of another sensor at another point in time and location. Second, the data sources may include various active and passive sensors such as radar, infrared, and acoustic sensors. The tracking and fusion problem is further complicated by the facts that the target may not be detected by some sensors due to the variation of signals and the sensor characteristics, and dense false alarms and clutters may be present which are not easily distinguishable from the true target measurements.

It is well known that the performance of multiple- hypothesis approaches for multitarget tracking are

*Research partitlly supported by the U.S. Army under the contract DAAB 10-91-C-0170

near-optimal and have gained popularity (Moil et al., 1986; Blackman 1986; Bar-Shalom 1992), since the pioneer work of Reid (1979). In these approaches, all feasible data association hypotheses between measurements and targets are formed, evaluated, and maintained. Although they can handle complex target and sensor models and include track initiating and continuation in one framework, they require huge amounts of computing resource, both time and memory, especially under dense target and clutter environments.

Other approaches for multitarget tracking include target-oriented and track-oriented approaches. The target-oriented approach (Bar-Shalom and Tse, 1975; Bar-Shalom, 1980) assumes that the number of targets is known and combines all data association hypotheses into one for each target. Even though the computational requirement is fixed for this type of approach, it can only handle track continuation, and requires a separate module for track initiation. The track-oriented approach, on the other hand, treats each track individually while they are initiated, updated, and possibly terminated based on the associated measurement history. This approach requires a data association and evaluation scheme to form and evaluate each track. It presents a trade-off between performance and computation and is most suitable for applications with dense target or clutter environment.

889

890 K.C. Chang

In this paper, a multitarget tracking problem under dense clutter environment is considered. Detections from ground vehicles are assumed to be available from multiple airborne MTI radars located at different platforms. Since MTI sensors can only detect targets with significant Doppler (range rate), the cooperation of multiple sensors at different sites will sometimes be necessary in order to track the targets. The goal of this study is to develop a simple, yet comprehensive multiple-target tracking and fusion algorithm suitable for dense clutter environment. It is shown in (Mori et al., 1992) that the optimal data association algorithm is only marginally better when the target or clutter density is either very high or very low compared to the simple nearest-neighbor algorithm. Therefore a simple track-oriented approach is proposed here based on a form of "greedy" nearest-neighbor and multiple model algorithms.

In this approach, a centralized fusion architecture is assumed, i.e., data collected from multiple sensors are pooled together in a central cite where they are combined. The goal here is to be able to pick up a potential track as quickly as possible and to eliminate the false tracks as effectively as possible. To do so, tracks are initiated based on a single measurement and a score is obtained for each track to determine its strength. The track score is calculated based o n associated measurement history as well as the target and sensor models. To eliminate false tracks effectively, tracks with scores below a specified threshold will be pruned. The pruning threshold is one of the system parameters and can be adjusted adaptively based on the scenario and performance requirement.

The algorithm has been implemented in a MATLAB environment. Simulation results with multiple MTI sensors have also been obtained. Extensive analysis with Monte-Carlo simulations shows expected performance. The paper is organized as follows. Section 2 presents the tracking algorithm, Section 3 describes sensor and target models, and Section 4 shows the simulation results

2. TRACKING AND FUSION A L G O R I T H M

The centralized architecture is chosen for tracking and fusion with multiple sensors. In other words, all detections from different sensors are pooled together in the fusion center where they are processed. To handle dense target and clutter environment, a track- oriented approach is proposed together with the multiple-model approach for tracking maneuvering targets.

The tracking and fusion algorithm is summarized as follows (see Fig. 1).

I. T rack clustering: At each scan when new data set (measurements) arrive, first decompose the existing tracks and new measurements into independent groups (clusters) based on gating so that data association can be performed within each cluster. This step is to eliminate unnecessary associations between far apart (in the measurement space) tracks and measurements.

2. Data association: Within each cluster, form and select the most likely data association event to be processed. To speed up the process in dense environments, a "greedy" algorithm is used to associate measurements to tracks.

3. Track scoring/updating: A score is assigned to each track and is obtained based on the association history. The score indicates the "true track probability" (TIP) and is used in the decision of eliminating or confirming tracks.

4. Track initiation: Measurements not associated with any existing track will be used to initiate a new track. A new track is initiated with a single measurement where a gaussian distribution for the target state is created based on that measurement.

5. Track management: To avoid redundant tracks, similar tracks are combined and tracks with scores below a certain threshold are pruned away.

2. I Data Association

In the data association module, a "greedy" nearest- neighbor algorithm is used to associate measurements to tracks. The idea is to start with the track of the least association uncertainty and assign it with the most likely measurement. The association uncertainty of a track is defined as the entropy of the association probabilities between the

For each Clusters

I New Data Set l t

_ I Clustering I - I

Data Associatior~

J ~ ' ~ . o r - - . . m ~ . . For e~t/~g Tracks

I Tmck Management

............................................................................... t ..........................................................................

Fig. 1. Algorithm Summary

Tracking and Fusion Using Multiple Senso.. 891

track "C and all the current "feasible" measurements Yi, i.e.,

where

H(x) = -~ Pi log(p/) (I) !

L(YilX)

Pi - ~. L(Yil x) l

(2)

and L(YilX ) is the association likelihood between

measurement Yi and track "C (for specific example,

see eqn. (16)).

Once the particular track-to-measurement association pair is chosen, they will be removed from the list and the next track with the least association uncertainty will be processed. This process continues until all tracks are considered. Note that measurements which have been chosen earlier will not be included in the current consideration. It is possible that a track may not have any validated 1 measurement; in that case, no measurement will be chosen and the track will not be updated. After processing all tracks, measurements that have not been assigned to any track will then be used to initiate a new track.

algorithm is used to define and evaluate track scores. In this algorithm, two models are used: one for "observable target", designated as Model A, one for "unobservable target", designated as Model D. The notion of "unobservable target" can represent either a true target outside the sensor coverage or an erroneously hypothesized target, that is, it is equivalent to no target. In both models, measurements can originate from the target (with detection probabilityPD) or clutter. However, in the

"no target" model, PD = O.

A Markov chain will model the observable and unobservable situation as follows. Denoting by M x the model x is in effect during the current sampling interval and M x for the previous interval, the

following transition probabilities are assumed:

P(MAIM--A)=I-eA , P (MDIMA)=eA

P ( M D I M % ) = I - e D, P(MAIMD)=eD (3)

That is, transitions between the models are assumed with low probabilities. Practical values for these "designed parameters" are discussed in (Bar-Shalom et al., 1990b).

2.2 Track Initiation

The track initiation module initiates a track based on a single measurement. To initiate a track, we need an algorithm to obtain a gaussian representation (approximation) of the target State distribution of an undetected target. It is reasonable to assume that the information contained in this distribution is very little compared with that contained in the measurements because variances of measurements are typically much smaller than that of the prior distribution. Therefore, ignoring the information contained in the target state distribution of undetected targets, a track will be initiated from a single measurement as follows:

1. First create a gaussian distribution from the three positional measurements, i.e., the range, the azimuth and the elevation.

The initial score for the new track is calculated based on the ratio of new target density ~NT and clutter

13Nr density {~, specifically, PO ( M A ) =

w h e r e I~NT and I~9~ are defined as the expected

numbers of true targets and false alarms per unit surveillance volume per scan. Denoting by P(M x)

and P(M'- x ) the probabilities that track Z and ~ are

in model x respectively, then at each scan, depending on the specific association for the track, the score is updated based on the following formula:

1 P(MA)=-~-P(MAIMA)P(M.A)PD(g)L(flg) (4)

Y I ['I'(M_'M. )e(M. )(l- ~L(~')~] .

P ( M D ) = _ _ / u A ~ X u /ILFA(YJ(5) Cy L+P(MDI~D)P(MD ) J

2. Create a gaussian distribution of the velocity vector with a priori information.

3. Modify the gaussian target state distribution by the Doppler (range rate) measurement.

2.3 Track Scoring/Updating

A score is assigned to each track and is updated based on the association history. A multiple-model

1 measurements inside the validation gate.

for track ~ with associated measurement y where

% = py(M A)+ py(Mo), and

1 FP(MAIMA)P(MA)(I-PD(~) ~ P~(M, ) = ~ / ,, %

"o('o): %

(6)

(7)

892 K.C. Chang

for track ~ with no associated measurement where

C~ = P o ( M A )+ P ~ ( M D ) .

For the tracks with associated measurement, they will updated based on the standard filtering equations such as Kalman filter or Extended Kalman Filter (EKF) (see Section 3 for a specific example). For those tracks with no associated measurement, they will only be extrapolated without updating.

i where PD is the detection function for each

component i in {R, A, E, D} which represents range, azimuth, elevation, and Doppler (range rate) 2 . Each x i represents the true value for each

component i. For i in {R, A, E , D},

121 i 1 1 Yi - xi - - - ~ F O V . exp - - - - - dYidY i P D ( X i ) - ,¢r2"~o i 2 o i (9)

2.4 Track Management

Since many tracks can be initiated under dense clutter environments, to avoid redundant tracks, a test of similarity is carried out to determine whether two tracks represent the same target. This test first checks if the two tracks share the same current associated measurement; if so, a track-to-track association technique (Bar-Shalom et al., 1990b) is then used to determine their similarity. Assuming the tracks are independent, this test will accept the hypothesis that the two tracks represent the same

target if (~ i -~ i ) ' (V. ^ - 1 ,, +Vj) (x i - ~ j ) < V where xi ' x j and ~/, ~y are the state means and covariances

of the two tracks and ~ is the decision threshold.

To judge if a track is real or not at any moment of time, the scoring threshold is used. A track is judged to be false and will be pruned if the score of the track is below the given threshold. This is to effectively eliminate unwanted tracks under dense clutter environments. The pruning threshold is one of the system parameters and should be adjusted adaptively based on the scenario and performance requirement.

3. SIMULATION MODELS

The target and sensor models used for the simulation are first described. Targets of interest are generally assumed to follow constant velocity trajectories. To study the effects of randomness in target dynamics (target maneuvering), white accelerations are assumed. The target motion is modeled in two- dimensional Cartesian coordinates.

The MTI radar observes targets in a low signai-to- noise ratio environment. Assume the MTI radar is modeled by the two functions, measurement model p u ( y l x ) and detection model p o ( x ) . The detection model can be described as

i PD(X) = l"I PD(Xi)

ie{R,A,E,D} (8)

where FOV i is the field of view 3 for the

component i, and (~i is the standard deviation for the component i, which will be discussed later. The measurement value model p M ( y l x ) is likewise

decomposed as

PM (yl x) = I'I PM (Yilxi) i ~ {R,A,E, D}

(10)

F o r i i n { R , A , E , D } ,

p M ( Y i l X i ) -

] FOV. exp t

where (~/ is the

ox , Iy i -x i l 2 2L °i J

_ _Iyi-xi] 2" 2L )

measurement

deviation given by

(11)

dy.' l

error standard

(12)

with ~i being the sensor resolution for the

component i and SNR is the signal-to-noise ratio.

The false alarm probability is determined by

P fa = exp(-SNRTH ) (13)

from which the expected number of false alarms can be calculated as

I-t(FOV i ) l I (14) Vfa = P

f a i e { R , A , E , D } 8.z

where SNRTH is the SNR threshold for detection

and ~t(FO~ ) is the volume of the field of view for

the measurement component i.

2 signal strength can also be explicitly modeled, see (Chang et al., 1994) for details. 3 It is generally an interval or a union of intervals so that the integral in (9) can be expressed by error functions.

Tracking and Fusion Using Multiple Sensors 893

The target state is modeled as a 4-dimensional vector, i.e., 2-D position and 2-D velocity as described earlier. It is also assumed that the target state distributions of all the old tracks ~ have gaussian representations, as

that update the mean vector to J and the estimation

error variance matrix to I~'. When no measurement is assigned, a track is simply extrapolated based on the target dynamic model.

p(xl~) = (det(2~V))-l / 2 e x p ( - l [[x - x-J[2_l ) (15)

Then it follows from the linearization of nonlinear measurement equations that the measurement to track association likelihood is

l, oxp( any o o 1 (16)

where YRAED = (R, A, E, D) is the measurement

vector, Y R A E D = ( R , A , E , D ) is the nonlinear

projection of target state estimate • onto the sensor's measurement space, S is the innovations variance matrix defined by

S = HVH T + Y~ with

[i ° cos

H= 0

0

0 01-,F 0 0 / / d

° / / 4 0 RJ L(V_vs)T

g R cosA'cosE

g -s X _

/ T/ L -cosAsinE LgeJ

sin A" cos

cosA

- sin A" sin

(17)

o/L-% (18)

0//':/(19' °°sg,fj

and 2 2 2

~, = diag.(o2,O A ,g E,O D) (20)

where V is the velocity component of 2 , v s is the

sensor velocity vector, I 3 is a 3x3 identity matrix,

and (gi)iEIR,A,g¿ is the sensor pointing system

defining azimuth and elevation directions.

The initial state estimate is obtained as described in Section 2. After a track is initiated, it is a rather straightforward application of the Extended Kalman filter to update a track by a measurement. Namely,

= ~ + K(YRAED - YRAED ) (21)

with

K = v H T s -1 (22)

and

~" = ( I - KH)V (23)

4. SIMULATION RESULTS

This section describes the results of simulations. The simulation is implemented in a MATLAB environment. A two-sensor, four-target scenario shown in Fig. 2 is created and simulated measurements based on the models described in Section 3 are generated. The two MTI sensors are located at approximately 2000 feet above the locations (0,0) and (50 kin,0). The nominal set of key parameters used are:

Azimuth FOV

Range FOV

Resolutions

Doppler threshold

Samplinl~ Interval

Mean SNR

sensor 1 (0 - 90°), sensor 2 (-90 - 0 ° )

15 - 55 km

Range:300m, Doppler: 1 m/s, Azimuth: 0.7 degree

2.5 rrgs

2 minutes

14dB

SNR threshold 10 dB

Track pruning threshold PrH = O. O1

Fig. 3 shows the receiver operating characteristics (ROC) of the system. With SNR=14dB, SNRth =10dB corresponds to the false alarm

density of, approximately PFA = 4.5x10 -5 , which

translates into the average number of 6 false alarms in the sensor FOV (field of view). Fig. 4 shows the typical sensor detections from two MTI sensors for the entire 80 minutes of simulation. It can be seen that the clutter density is relatively high and targets are often mis-detected especially when their relative velocities (range rate) are low. Fig. 5 shows the sample run results with single MTI sensor only and Fig. 6 shows the results with both sensors. Figs 7 - 8 show the performance curve with 50 Monte-Carlo simulations. It is clear that with two sensors, not only are all four targets tracked (versus two or three targets tracked with a single sensor) but the track probabilities are also higher while the average false track probabilities remain low. Since a track is initiated with a single detection, there will be more tracks created in the fusion case than in the single- sensor cases as shown in Fig. 7. Fortunately, the track life of most of the false tracks is relatively short as shown in Fig. 9. In other words, the false tracks are pruned rather effectively.

894 K.C. Chang

A more ~tressing examl~le with S N R h = 8dB was

also simulated which corresponds to the false alarm

density, PFA = 1.2x10 --4, that generates about 16

false detections per scan in the surveillance region. The simulation results given in Figs 10 and 11 show that even though the average number of false tracks increases significantly, the tracking performance does not degrade too much. Specifically, most of the four targets are tracked with reasonably high probabilities. The average SNR is also varied to study its impact on the tracking performance. Since the detection probability is a function of the SNR (see Fig. 3), the tracking performance, particularly the average good track probabilities, decreases from 0.9 to about 0.7 when average SNR decreases from 14 dB to 10 dB (see Figs 8 and 12). The pruning threshold is the last factor studied in the simulation. It turns out that the tracking performance (not shown here) is relatively insensitive to the pruning threshold, particularly for the threshold ranging from 0.01 to 0.1. However, the performance degrades notably when the threshold goes beyond that.

5. SUMMARY AND CONCLUSION

This paper considers the MTI tracking and fusion problem for scenarios with high false alarm density. In order to develop a simple, yet comprehensive algorithm suitable for densely cluttered environments the track-oriented approach is adopted where a form of "greedy" nearest-neighbor association algorithm is used. In this approach, a centralized fusion architecture is assumed. To establish good tracks and eliminate false tracks effectively, tracks are initiated based on a single measurement and a score is assigned for each track to determine its strength, The track score is computed based on the association history using a Markov chain multiple-model algorithm. Tracks with scores below a certain threshold are then discarded and similar tracks are combined.

The algorithm has been implemented in a MATLAB environment. Tracking results with multiple MTI sensors using extensive Monte-Carlo simulations have also been obtained. Several key parameters such as SNR and detection threshold were varied to demonstrate the feasibility of the algorithm. Future work include the development of algorithms for incorporating terrain information and integrating data from other intelligent sources such as infrared and COMINT.

5C

~c

2C

1C

~0.8 ~6

~ 0.6 n

0.4

0.2

$1 S;

0 10 20 30 40 50 x (km~

Fig. 2. Tracking Scenario

"3NR = 18 DB - - - - - - - - - ~ /

10 10 10 10 10 False Alarm Density

Fig. 3. Receiver Operating Characteristic

50

4~

2~

4(

2(

0

5(

1C

Detections (1) o ' o o ' ' 1 ' '

0(3O° O0~_nO0 0 0 o ° 0 O % ~ ~ o

J O~B~ 0 o 0(~ ° 0 0 0 o 't~O0 OOo_ 0 o o =° ~=~ °o~o o

o ooo~o ~=~o

~ o O o c ~

~ - ° o %°o-_o° i o ^ 0 O uOo 0 O U O0 0 u ~o~ o~ oooo;~o o ~ 0

%1o , Oo O~o~ o ~o%o 0% o~

~1 , oa o , o n , ~ 10 20 30 40 50

x (kin) Detections (2)

o, o f _ o %0 o o

Oo ~ °O o o ~

o o o

oO o oO

o°O ~ = °

o %goOo

~ o o # ~o ~° o

.2o o ° % ° ° ° , 10 20 30 40 50

x (krn)

Fig. 4. Sensor 1 and Sensor 2 Detections

Tracking and Fusion Using Multiple Sensors 895

5C

4C

l(

Estimated Trajectories

=-4--.=__ I

10 20 30 40 x 0an)

50

5(

4(

21

I0

0 0

E~imated Trajecmrics

X X

I0 20 30 40 x Oun)

50

Fig 5 Tracking Results Single Sensor (Sensor 1 and Sensor 2)

15

i i 0

I ' -

Sensor 1 only =:=-~=::~.-- Sensor 2 only --x--x-- Bo~ Sermors --o--o--

10 20 3o "r'wnm

4

jt 3.5 J. ~ 2 . 5

~ 2

1.,5

1

0.5

10 20 30 40 Time

Fig. 7. Monte-Carlo Performance Curves (I) - Average Total Track and Good Track

Numbers (SNR = 14 dB)

5(

4(

>,

2(

1(

Estimated Trajectories

0 10 20 30 40 50 x (km)

Fig 6 Tracking Results - With Both Sensors

1

'0.~

~0,6

~ 0.7

0.e

0.~

0.4

0.~

~' Sw~or 2 ordy - x - x - ÷ Both Semmrs --o---o-1

10 20 30 40

~ 0 . 5 ~ . 'Sonso r l on l y - : : I

I o 4511~ ~ = o~y - x - x , "-°--°'t

0,3

0.2~ 10 20 30 40

Time

Fig. 8. Monte-Carlo Performance Curves (II) - Average Good Track and False Track

Probabilities (SNR = 14 dB).

Average FaIN Track Age 6

5

I- 3

2 i

Sensor 2 only - -x- -x- - Both ,~msom --o--o--

10 20 30 40 Time

Sensor 2 only --x---x--- Both Sensors ---o---o--

Fig. 9. Average Life of False Tracks

25

1-

10

Sensor 1 only - -+ - -+ - - Sensor 2 only - - x - - x - - Both Sentmm --o--o---

10 20 30 40 Time

4

~ 2.5

I- 2

1.5

1

0.5 Senior I only - : ~ : : : ~ - Sensor 2 only - - x - - x - - Both , s e m o m - - - o - - o - -

.

10 20 30 , 4 0 Time

Fig. 10. Monte-Car lo Results - Average Total and Good Track Numbers ( SNRth = 8 dB )

1

~ 0.9

~ 0.8

~ 0.7

0.6

0.5

0.4

0.3

896 K.C. Chang

0 10 20 30 40 Time

0.5[~ Sensor 1 only ---+--+--- "~ ._1~ Sensor 2 only --x---x--- ~ o-,,~r ~ Both Sensors ---o---o-~-

0 . 3 5 ~

0.25 0 10 20 30 40

Time

Fig. 11. Monte-Carlo Results - Average Good Track and False Track Probabilities (SNR~h = 8 dB)

4 - , -

!3 z ~ 2 . 5

_. 1.,, .... 0.5~-' Sensor 2 only --x--x---

Of , Both ,Sensors ---o:,--o-- 0 10 20 30 40

Time

1

-~0.9

~ 0.8

~0.7 ) -

0 . 6

0 . 5

0 .4

0 .3

Sensor 2 only --x--x--- Both Sensors ---o--o---

10 20 30 40 T~e

Fig. 12. Monte-Carlo Results - Average Good Track

Numbers and Good Track Probabilities (SNR = 10 dB)

Tracking and Fusion Using Multiple Sensors

REFERENCES

897

Bar-Shalom, Y. and E. Tse (1975). Tracking in a Cluttered Environment with Probabilistic Data Association, Automatica, Vol. 11.

Bar-Shalom, Y. T.E. Fortmann, and M. Scheffe (1980). Joint Probabilistic Data Association for Multiple Targets in Clutter, Proc. the 1980 Conf. on Information Science and Systems, Princeton University.

Bar-Shalom, Y. (1990a). Multitarget-Multisensor Tracking: Advanced Applications, Vol. I, Artech House, Inc., 1990.

Bar-Shalom, Y., K. C. Chang, and H. A. P. Blom (1990b). Automatic Track Formation in Clutter with a Recursive Algorithm, Multitarget- Multisensor Tracking: Advanced Applications, Vol. I, Chapter 2, Artech House.

Bar-Shalom, Y. (1992). Multitarget-Multisensor Tracking: Applications and Advances, Vol. 11, Artech House, Inc., 1992.

Blackman, S. S. (1986). Multiple-Target Tracking with Radar Applications, Artech House, Inc.

Chang, K. C., S. Mori, and C. Y. Chong (1994). Evaluating a Multiple-Hypothesis MultiTarget Tracking Algorithm, IEEE Trans. on Aerospace and Electronic Systems.

Mori, S., C. Y. Chong, E. Tse, and R. P. Wishner (1986). Tracking and Classifying Multiple Targets without A Priori Identification, IEEE Trans. on Automat. Contr., Vol. AC-31, No. 5, pp. 401 - 409.

Mori, S., K. C. Chang, and C. Y. Chong (1992). Performance Analysis of Optimal Data Association with Application to Multiple Target Tracking, Multitarget-Multisensor Tracking: Applications and Advances, Vol. II, Chapter 7, Artech House.

Reid, D. B. (1979). An Algorithm for Tracking Multiple Targets, IEEE Trans. on Automat. Contr., Vol. AC-24, pp. 843 - 854.