Performance Analysis of Modern Sensor Systems

Ake Andersson and Thomas R. KronhamnAirborne Radar Division

Ericsson Microwave Systems AB, Sweden.ake.andersson@emw.ericsson.se and thomas.kronhamn@emw.ericsson.se

Abstract – Traditionally, radar or sensor performance isreviewed in terms of detection ranges and measurement ac-curacy. Today, when the radar often is a part of a locallyintegrated sensor system, which shall cooperate with othersensor systems, other performance parameters are requiredfor evaluation and comparison of various solutions. Suchperformance parameters shall focus on the more complexrequirements on target track data as stated by the user. Ex-amples of such performance parameters are track accuracy,track continuity, probability of track acquisition, track load,etc, which all can be compiled into one single performanceparameter if required.

The first step in such an analysis is to get a picture of thepossible performance, given a set of scenarios. This canand should be done without detailed knowledge of a spe-cific system design of the sensors. This analysis can then beused either for comparison with the actual performance ofavailable designs, or for the determination of requirementsof a new, not yet designed system. The present paper isa step towards an analytical computation of sensor systemperformance at the track level.

1 Introduction

The present paper describes a first step towards a methodfor performance analysis of sensor or systems of sensorsgiven a scenario or a set of scenarios. The ambition isto handle systems of sensors of arbitrary type, includingadaptive sensors. The general idea is to calculate the per-formance that can be achieved in principle, and such thatvarious system solutions can be compared for system eval-uation.

Typically, such an analysis shall focus on performanceparameters important as stated by the user. Such perfor-mance parameters are tracking accuracy, track continuity,probability of track acquisition, track load, etc, which allcan be compiled into one single performance parameter ifrequired.

The method of analysis is based on analytical modelsof the sensor search and track performance in a commonframework called ALCATRAS, analytical calculations oftrack and search, thus avoiding time-consuming Monte-Carlo simulations and detailed sensor definitions. For ear-lier work on the single and adaptive surveillance sensor casesee references [4] and [5]. The models and some introduc-tory examples are described below.

2 Sensor Measurement Model

2.1 Detection Model

The detection model is based on a generic form of the radarequation. The model is as follows: the signal to noise ratio(SNR) is computed as

SNR = SNR50σ

σ50

(R50

R

)4

(1)

whereσ is theradar cross section (RCS) of the target, andR is the range to the target. This simple form modelsthe SNR dependence of the fundamental target parametersradar cross section and range. The SNR is normalized to theSNR at the 50% detection range for a reference radar crosssection ofσ50. These quantities can be calculated from thetechnical parameters for a sensor, given a specific sensordesign. Furthermore, a model is required that describes theRCS fluctuations. An example of such a model is the so-called Swerling case 1 model, see [6]. The probability ofdetection for this model is

Pd = P1

1+SNRf (2)

wherePf is the probability of false alarms, which in prin-ciple settles the detection threshold. Here, this particularmodel is selected for its simplicity. Other models can ofcourse be used when required.

2.2 Single Sensor Measurement Model andTrack Load

The single sensor model consists of a covariance matrix,R, of the measurement vector,z, and a few parameters thatcontrols the time between measurement attempts, includ-ing the scan rate of the search volume. The sensor mea-sures position in spherical coordinates and the covariance ofthe measurement is assumed fixed and known. This modelcould of course be more elaborate, i.e. a function of SNRwhich in turn is a function of, among other quantities, rangeand RCS. Again, here a simple and straight forward modelis selected such that the principles are not diffused by com-plicating matters more than necessary. Then, for a givenprobability of detection,Pd, and a given time between mea-surement attempts,T0, the effective time between success-ful measurements is computed as

Te =T0

Pd(3)

whereT0 is the nominal time between measurement at-tempts.

In case of an adaptive sensor, the time between measure-ment attempts for thekth target is controlled via a priorityparameter,pk. This priority parameter is a continuous vari-able between zero and one, which can be set either man-ually by the user, or automatically by some sensor controllaws. The nominal time between measurement attempts isthen computed by

T0 =Tc

1 + ceil (pkNe)(4)

for targets inside the search volume, and by

T0 =Tc

ceil (pkNe)(5)

for targets outside. Here, the parameter,Ne, is the maxi-mum number of additional measurement attempts allowedfor each tracked target. Note, the productpkNe is zero fortargets not yet detected. The function ceil() rounds towardsinfinity. Furthermore, the cycle time,Tc, is the time,Ts,that it takes to scan the search volume, and to perform theadditional measurements for one cycle, i.e.

Tc = Ts + Tt + Tf (6)

whereTf is the additional time generated by false alarms,andTt is the time it takes to perform the additional mea-surement attempts on tracked targets, which is

Tt = Tl

Nt∑k=1

ceil (pkNe) (7)

whereTl is the required time for one additional measure-ment attempt, andNt is the number of tracked targets. Fi-nally, the resulting track load is computed by

Lt =TtTc

(8)

2.3 Multi Sensor Measurement Model

For the analysis in the multi sensor case, the single sensormodel of the various sensors is compiled into one common“measurement” model consisting of a measurement vectorand its corresponding covariance matrix, and an effectivetime between measurements. This is done as follows: themeasurements are converted into a common Cartesian co-ordinate system where the tracking filter operates. The var-ious sensor measurements are then “fused” by

z = R

Ns∑k=1

µkR−1k zk (9)

R =

{Ns∑k=1

µkR−1k

}−1

(10)

whereNs is the number of sensors,zk andRk are themeasurement vector and its corresponding covariance ma-trix obtained from the involved sensors. Furthermore, the

weights,µk, are computed as

µk =Te

Tk/Pdk(11)

whereTe is referredto as the effective time between mea-surements, which is computed as

Te =1∑Ns

k=1

PdkTk

(12)

wherePdk is theprobability of detection computed by (2),andTk is the nominal time between measurements for sen-sork.

3 Target Acquisition and Tracking

3.1 The Tracking Filter

The target tracking filter is a standard Kalman filter [1] withthe addition of the decomposition, shown in [3], of the co-variance matrix,P , as computed by the Kalman filter. Thisdecomposition of the covariance matrix,P , is motivated asfollows: consider the target model

x(k + 1) = Φx(k) + u(k) (13)

whereΦ is the system matrix, e.g.

Φ =[1 T0 1

](14)

for the 1-dimensional case and a constant velocity target,and the vectoru is a deterministic but unknown vector pro-cess that describes the target maneuvers. Now, we intro-duce the error vector of the estimated state vector, whichis decomposed into the stochastic errors,xM , and the biaserrors,xS , as

x(k|n) = x(k)− x(k|n) = xM (k|n) + xS(k|n) (15)

By combining (13) and (15), the bias error of a given targettrajectory is

xS(k + 1|k) = ΦxS(k|k) + u(k) (16)

xS(k + 1|k + 1) = (I −KH) xS(k + 1|k) (17)

where I is the identity matrix with suitable dimensions.Then, the corresponding decomposition of the covariancematrix,P , as for the error vector, is as follows

P (k|n) = PM (k|n) + PS(k|n) (18)

PM (k + 1|k) = ΦPM (k|k)ΦT (19)

PS(k + 1|k) = ΦPS(k|k)ΦT +Q (20)

PM (k + 1|k + 1) =

(I −KH)PM (k + 1|k)(I −KH)T +KRKT (21)

PS(k + 1|k + 1) =

(I −KH)PS(k + 1|k)(I −KH)T (22)

whereQ is the “covariance” of the system noise,R is thecovariance of the measurement noise,H is the observationmatrix, andK is the Kalman gain matrix. From this, it fol-lows that the covariance ofxM isPM for a given given pro-cessu. Thus, the covariance of the estimated state vector,x, is alsoPM . Furthermore, from the decomposition ofP ,we observe that the covariancePM originate only from thecovariance of the measurement noise, and the “covariance”matrixPS originate only from the covariance of the systemnoise,Q, which we consider as a design “parameter” of thetracking filter. Note, the Singer model [7] is applied for themodeling of the system noise covariance,Q.

In conclusion, we can now analytically compute thetracking accuracy performance in terms of bias and covari-ance, given the filter structure above and the deterministicvector process,u. Furthermore, this analysis is done with-out costly Monte-Carlo iterations. It should here be notedthat this is not the “optimal” filter performance that can beachieved. However, for a system analysis and/or a systemreview of typically performance, the method is adequate.

3.2 Target Detection and Acquisition

The analysis of detection and tracking ranges is based onMarkov chain analysis, see e.g. [2]. The detection range ofa given target trajectory is computed via the accumulatedprobability of detection,Pa. This is done for a nose aspecttarget with a given constant velocity and radar cross sec-tion. The Markov chain for the accumulated probability ofdetection is

pk+1 =[qk 0pk 1

]pk (23)

p0 = [1, 0]T (24)

wherepk is computed by (2), andqk = 1 − pk. The accu-mulated probability of detection is obtained as

Pa(k) = pk[2]. (25)

A target is then considered detected whenPa is greater thanone half, which gives the detection range for this particulartarget.

The corresponding tracking range is computed by the ac-cumulated probability of target tracking,Pt. For this pur-pose, criteria are required for target acquisition and dele-tion. For target acquisition, we require that the first mea-surement of a candidate target is followed by two additionalmeasurements of that target, each within a given time,Ta.The resulting criterion is a 2 out ofN and 1 out ofN crite-rion, where the parameterN depends on the measurementrate of the sensor and the acquisition time,Ta. Similarly, atarget track is deleted when the sensors fail to produce mea-surements of a given target within a given time,Td, whichgives the deletion criterion. The probability of target track-ing is then computed by Markov chain analysis and the tar-get is declared tracked when this probability is greater thanone half. The Markov chain used for computation of theprobability of target tracking is illustrated in figure 1. Here,the scalar integersK, L, andM are computed from the pa-rameters,Ta andTd. The probability of target tracking,Pt,

is then computed as the sum of probabilities obtained fromthe statesL+ 1 toM of the Markov chain at each iterationof the chain for a given target trajectory.

1 K−1

M M−1 L+3 L+2

L+1LL−1K+2K+1K32

q

p q q p

q

p p p p p

q

p

q

q

pp

q

pppp

qq q

qq

q

Figure 1: Illustration of the Markov chain used for the com-putation ofthe probability of target tracking.

4 Examples

4.1 Analytical vs. Monte Carlo Analysis

First, we compare the analytical computation of the track-ing accuracy and the corresponding accuracy as computedby Monte-Carlo simulations. In this example, we have asingle target measured by a single sensor with a constanttime between measurements of 2 s. The sensor is located atthe origin of the scenario and the target, starting in the up-per left corner, approaches the sensor with a speed of 270m/s and makes a 3g turn (clockwise) after 60 s, and an ad-ditional 6g turn (counter-clockwise) at 200 s, see figure 2.

−10 −5 0 5 10 15 20

−10

−5

0

5

10

15

20

25

30

x [km]

y [k

m]

Figure 2: Illustration of the scenario used for comparisonof analyticalcomputation of tracking accuracy and Monto-Carlo simulations.

The tracking accuracy as computed by the proposed ana-lytical method is compared by the method of Monte-Carlosimulation iterated 500 times. In figure 3, the total RMS

error and the standard deviation is shown for the trackingaccuracyin the x coordinate direction. The accuracy ascomputed by the analytical method is shown in the upperinsert of the figure, and that of the Monte-Carlo method isshown in the lower insert. The dashed lines illustrate thetotal RMS error, while the solid lines illustrate the standarddeviation.

First, we note that we have bias at three occurrences ofthe tracking trajectory; at the track initiation and at the twotarget maneuvers. The bias is caused by the non-stochasticbehavior of both the initiation and the maneuvers, and bythe fact that this particular filter solution models these track-ing events as stochastic. Furthermore, we can clearly seethat the deviation between the two inserts of the figure isminor, and that this deviation is caused by stochastic vari-ations of the Monte-Carlo method. Thus, this confirms thevalidity of the proposed method.

0 50 100 150 200 250 3000

200

400

600

800

1000

1200

t [s]

rms

& s

td [

m]

0 50 100 150 200 250 3000

200

400

600

800

1000

1200

t [s]

rms

& s

td [

m]

Figure 3: Comparison of analytical computation of trackingaccuracyand Monte-Carlo simulations.

4.2 Analysis of Adaptive Single Sensor

In this example, we illustrate the behavior of an adaptivesensor by an tracking example as shown by a target indi-cator, see figure 4. In the figure, the search volume of thesensor is illustrated by dashed-dotted lines, and the detec-

tion ranges for various RCS are indicated by dashed lines.Furthermore, we have two tracked targets and the currenttrack load is shown in the track load indicator at the upperright corner of the figure. The dotted lines at the tails ofthe target symbols, illustrate the history of the target tracks.The search volume and the target priority can be controlledby the user via the sliders to the left, and the result thereofis seen directly in the target and track load indicators.

This application of the analysis tool is used to illustrateand/or study the behavior of an adaptive sensor. In par-ticular, the user can study the consequences of allocatingadditional sensor resources for a particular target track, i.e.prioritizing the target. For example, the user can see howthe search and tracking performance is affected when pri-oritizing one or several targets, and thus determine whetheror not the taken measure does the trick or fails.

0

100 %

RCS:20 m2

RAcc,50

:154.6667 km

RCS:5 m2

RAcc,50

:104 km

RCS:1 m2

RAcc,50

:66.6667 km

Figure 4: Illustration of track load applying an adaptive sen-sor.

4.3 Analytical Multi Sensor Performance

In this section, we give an example of a scenario wherethree sensors cooperate when tracking a single target. Infigure 5, the scenario is illustrated where the positions ofthe sensors and their detection ranges are indicated by smalland large circles. Furthermore, the target trajectory is il-lustrated by the solid line starting in the upper left corner.The target makes two maneuvers during the scenario: one3g turn (clockwise) and one 1g turn (counter clockwise).Finally, the target has a speed of 270 m/s and the total sim-ulation time is 700 s.

For comparison, we first show the tracking accuracy asobtained from the sensors when tracking the target individ-ually. The tracking accuracy is illustrated in figure 6, wherered lines show the accuracy of the sensor located at (0,50)km, blue lines that of the sensor located at the origin, andblack lines that of the sensor located at (70,25) km. Fur-thermore, the upper insert shows the accuracy obtained inthex coordinate direction, and the lower insert that of they coordinate direction. Finally, dashed lines show the totalRMS error, and solid lines the standard deviation. Note thatnone of the sensors have enough capacity to track the tar-get over the whole of the target trajectory. Thus, we have

−60 −40 −20 0 20 40 60 80 100 120 140

−60

−40

−20

0

20

40

60

80

100

120

x [km]

y [k

m]

Figure 5: The multi-sensor scenario.

regionswhere only one of the sensors gets measurementsof the target and that we have only smaller overlapping re-gions.

0 100 200 300 400 500 600 7000

200

400

600

800

1000

1200

1400

t [s]

rms

& s

td [

m]

0 100 200 300 400 500 600 7000

200

400

600

800

1000

1200

1400

t [s]

rms

& s

td [

m]

Figure 6: Tracking accuracy of the individual sensors.

In figure 7, the accuracy of the fused track is shown as

computed by the proposed analytical method. In the upperinsert of the figure, the accuracy of the track is shown inthex coordinate direction, and the lower insert shows thatin the y coordinate direction, c.f. figure 5. Dashed linesshow the total RMS error, and solid lines show the standarddeviation. Note, the deviation between total RMS error andstandard deviation at the target maneuvers.

0 100 200 300 400 500 600 7000

200

400

600

800

1000

1200

1400

t [s]

rms

& s

td [

m]

0 100 200 300 400 500 600 7000

200

400

600

800

1000

1200

1400

t [s]

rms

& s

td [

m]

Figure 7: Tracking accuracy of the multi-sensor scenario.

Finally, we show an illustration of the probability of de-tection and target tracking (object report) in combinationwith a so-called accuracy factor, see figure 8. The accuracyfactor is computed as

fa = min(

1,σ0√λmax

)(26)

whereσ0 is therequired accuracy stated by the user (here,σ0 is selected as 300 m), andλmax is the maximum eigen-value of the matrix

C = W (PM + xSxTS )WT (27)

where the weight matrix,W , is selected as

W = HΦ (28)

Here, the matrix,H, is used to select the coordinates con-sidered important, and the “system matrix”,Φ is computed

for some suitable prediction time (here, the prediction timeis selectedas 2 s). Thus, the accuracy factor can be setproperly for whatever requirements the user might have. Inthe figure, the dashed-dotted, dashed, and dotted lines showthe probability of detection of the various sensors, the blueline shows the probability of target tracking, and finally thesolid black line shows the accuracy factor. Note that theprobability of target tracking is close to one, even at the endof the scenario, where the probability of single scan detec-tion is below one half for all three sensors. Furthermore, wenote the dip in the accuracy factor,fa, at the time intervalof 270-300 s. This dip in the accuracy factor is caused bybias error at the target maneuver.

0 100 200 300 400 500 600 7000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

t [s]

[−]

Figure 8: Probability of detection of the various sensorsand theresulting probability of object report. In addition,the accuracy factor is given.

5 Discussion

The present paper gives a method of analytical calculationof search and track for analysis of the performance of sin-gle or multiple sensor systems. The method avoids time-consuming Monte-Carlo simulation and provides a frame-work for evaluation of a specific system solution, compari-son between various system solutions, and computation ofthe in principle achievable system performance.

References

[1] Y. Bar-Shalom and T.E. Fortmann.Tracking and DataAssociation. Academic Press Inc., New York and Lon-don, 1988.

[2] S. S. Blackman.Multiple-Target Tracking with RadarApplications. Artech House Inc., Dedham, MA, 1986.

[3] Thomas R. Kronhamn. Adaptive target tracking withserial Kalman filters. InProc. of 24th Conferenceon Decision and Control, pages 1288–1293, Ft. Laud-erdale, 1985.

[4] Thomas R. Kronhamn. AEW performance improve-ments with the ERIEYE phased array radar. InRadar93, IEEE 1993 National Radar Conference, Boston,USA, 1993.

[5] Thomas R. Kronhamn. Surveillance performance.In Radar 95, IEEE International Radar Conference,Washington, USA, 1995.

[6] D. P. Meyer and H. A. Mayer.Radar Target Detection.Academic Press Inc., New York and London, 1973.

[7] R. A. Singer. Estimating optimal tracking filter perfor-mance for manned maneuvering targets.IEEE Trans.AES., AES-6(4):473–483, July 1970.