An Algorithm to Ascertain Critical Regions of Human Tracking Ability

183IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-9, NO. 4, APRIL 1979

An Algorithm to Ascertain Critical Regionsof Human Tracking Ability

DANIEL W. REPPERGER, MEMBER, IEEE, SHARON L. WARD, EARL J. HARTZELL,BETTY C. GLASS, AND WALTER C. SUMMERS

Abstract-A statistical algorithm is developed to study human FORCING FUNCTION VISUAL (DISPLAYED) CRANK SIGNALtracking behavior in a precognitive tracking task. The algorithm T MAN PLANTpresented here determines the point in time when a tracking task =H(s)becomes too difficult for thehuman to follow. Consequently, different J

behavior responses are observed to occur. A decision rule based on astatistical test of normality is used to delineate the two regions oftracking behavior. The proof of convergence of this algorithm to aunique solution is given. Data from a good and poor tracker areanalyzed using this algorithm to illustrate how to utilize the approach Fig. 1. One operator.presented here.

I. INTRODUCTIONV [HtEN THE interaction of a human with a machine is

studied within the context of system theory, the totalclosed-loop performance capability is dependent on limita-tions of both the human and the machine. In a tracking task,if the controlled element remains invariant, then a problemof interest is to determine or quantify upper limits oftracking capability of the human. A well-known example ofthis approach is the critical tracking task devised by Jex et al.[1]. The concept behind the critical tracking task can be bestunderstood by referring to Fig. 1. In the critical trackingtask, the plant takes on the form:

H(s)= K (1)

where K2 > 0 and the plant being controlled is unstablewith a pole at K2 rad/s. As K2 iS increased in value from zero,the tracking task becomes more difficult to control. Thisdifficulty occurs due to the problem of trying to maintainclosed-loop stability. A critical value for K2 is defined as thatvalue for which the closed-loop man-machine system be-comes unstable and the error signal diverges. A critical valueof 6.6 rad/s was obtained from empirical data [1] and can berelated to the inverse value of an effective estimate of thehuman's time delay.

Manuscript received May 16, 1977; revised November 30, 1977, July 24,1978, and November 20, 1978. This work was supported by the AerospaceMedical Research Laboratory, Aerospace Medical Division, Air ForceSystems Command, Wright-Patterson Air Force Base, OH 45433. Thispaper has been identified by the Aerospace Medical Research Laboratoryas AMRL-TR-78-76.

D. W. Repperger, E. J. Hartzell, and W. C. Summers are with theAerospace Medical Research Laboratory, Wright-Patterson Air ForceBase, OH 45433.

S. L. Ward and B. C. Glass are with Systems Research Laboratories,Inc., Dayton, OH 45440.

From an intuitive point ofview, one would expect to test ahuman's limitations by gradually increasing the difficulty ofthe tracking task until a breaking point is observed to occur.For the case ofa randomly appearing tracking task, a secondmethod [2] of establishing critical tracking regions is ob-tained by keeping the plant constant but gradually increas-ing the bandwidth of the input forcing function (sum ofsines). Using this procedure, the input becomes more andmore difficult to track until a point is reached where thebandwidth of the input forcing function becomes too wide-band for the human to control. As a consequence, the humanlowers his gain, which results in a reduction in the crossoverfrequency of the open-loop man-machine system. This hasbeen termed "crossover regression" [2]. The critical regionsof tracking capability can now be quantified by the inputforcing function bandwidth, and the event of regression orlow-gain tracking strategy is the result of a human drivenpast his limitations for this manual control task.

In this paper the tracking task considered was deter-ministic and had an extensive learning period for thesubjects involved in the experiment. To develop a criticalregion of tracking capability for this case, a differentapproach will be taken here. Using the definition of precog-nitive tracking task [3] as "any system in which the humanhas foreknowledge of the input in terms of other than adirect and true view," the term precognitive can be employedto describe a forcing function which has been memorized tosome extent. One example of this type of target input occursin AAA (antiaircraft artillery) simulators which mimicair-to-ground military engagements. The details describingthe learning period for the subjects in this experiment andthe characteristics of the deterministic forcing function willbe presented in the sequel.

Studying man-in-the-loop problems for this type ofprecognitive tracking task is important because a human isthe weakest element in this man-machine interaction. If the

00 18-9472/79/0400-0183$00.75 ©) 1979 IEEE

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-9, NO. 4, APRIL 1979

man-in-the-loop is subjected to stress, the effectiveness ofthetotal man-machine system is impaired depending to whatextent the human is affected by the stress. This motivatesthe study of the human limitations for this type of manualcontrol problem. In the approach presented here, the humanlimitations are defined in terms of the velocity and accelera-tion properties of the target forcing function. This definesthe effective capability of the total man-machine system.

In this paper a phase plane study of the closed-looptracking error will be conducted. Such methods have beenconsidered previously in the literature, e.g., by Costello [4],Young and Meiry [5], and Phatak and Bekey [6]. In thisapproach the tracking error is studied in the phase plane,and the distinction between adequate tracking performanceand inadequate tracking performance is obtained on thebasis ofthe statistical distribution ofthe system's error. Ifthedistribution of the error signal is normal, the tracking isconsidered to be satisfactory; if the distribution of the errorsignal is not normal, then more than one type of trackingbehavior has occurred. It is noted that the assumption ofnormality gives rise to a decision rule for this deterministictracking task. These results, however, do not extend readilyto the random tracking task, as can be shown.'To dichotomize human tracking response into normal

tracking and other types of tracking response may beconsidered a behavior model [7] approach to the problem.The concept of a behavior model [7]-[9] is a useful mannerof describing human responses when the desired propertiesto model are not numerical values of output responses, butare modes of response behavior. The approach used in thispaper to characterize human responses or behavior as afunction of properties of the target forcing function is notunlike that considered by Baron et al. [10] by applyingdifferential game concepts to combat games. In [10], as isconsidered here, regions oftracking capability and unattain-able regions of tracking capability can be determined andquantified depending on characteristics of the target forcingfunction or tracking maneuver.

II. THE EXPERIMENT

Seven teams of two subjects each, both male and female(one operator tracking azimuth (or horizontal axis) and theother tracking elevation (or vertical axis)) were trainedapproximately 20 days with 16 runs per day prior to thecollection of the data used in this paper. The target forcingfunctions (or flyby trajectories) simulated an aircraftapproaching a stationary ground point and were con-structed by an arc-tangent function in the azimuth axis and acosine bell function in the elevation axis. Fig. 2(a) and (b)illustrates the arc-tangent forcing function in the azimuthaxis. For simplicity, only the azimuth trackers are studiedhere using four forcing functions similar to Fig. 2(a) and (b)but with slightly different velocity and acceleration profiles.From the seven teams in the study, three teams were selectedfor this analysis procedure based on the performance resultsdiscussed in Section V of this paper.

These aspects of the problem were pointed out by one of the reviewers.

III. Two MODES OF TRACKING BEHAVIOR

With reference to Fig. 1 it is noted that the closed-looperror signal e(t) satisfies:

e(t) - DT(t) -X(t) (2)

where $D(t) is the target forcing function in Fig. 1 and x(t) isthe controlled element output.

Ideally, if the human were a perfect tracker, then e(t) = 0and x(t) = 'dT(zj. Obviously this rarely happens, and twodefinitions are developed here to illustrate the two possibleextremes. Defining 1je(t)jW as follows:

e(t)l- e2(t) + e2(t) + Z,2(t) (3)

where the dot indicates time differentiation, two definitionsof tracking behavior are considered as follows.

Definition 1 (Normal Tracking): In this case the outputvariable x(t) reduces the magnitude ofthe error signal e(t) in(2). Consequently

Ile(t)ll < FI1T(t)ll (4)

Definition 2 (Regressive Tracking): In this case the outputvariable x(t) is not that helpful in reducing the magnitude ofthe error signal in (2). Consequently,

(5)Ile(t)ll JIDT(t) IIDefine a ratio A(t) as follows:

e(t)- l(t)A

1

(t

(t)LID 11

The two regions of tracking behavior can now be definedquantitatively.

Definition 1 (Normal Tracking): The loop is tightly closedwith 0 < A(t) 4 1. In other words, the error in the loop is

much smaller (by an appropriate norm) than the inputforcing function.

Definition 2 (Regression Tracking): The loop has openedup with A(t) > 1.0. In this case, the input forcing functiondominates the closed-loop error signal, implying that thehuman's response u(t) has little effect in reducing themagnitude of e(t) with respect to T(t).

Of course, there exists an intermediate region whichneither satisfies definition 1 nor definition 2. It will beimportant to discriminate between definitions 1 and 2 forprecognitive tasks and thus develop a more explicitdefinition of response behavior.

IV. DETECTION OF DIFFERENT MODESOF BEHAVIOR

During the experiment the subjects exhibited responsebehavior which can be conceptually described in Fig. 3.Normal tracking behavior occurred during the time periods[to, t l] and [t2, tF] when the velocity and acceleration of the

forcing function were relatively small. In the time period [t 1,t2], however, the largest value of the velocity of the forcingfunction occurred, and also C- reached its peak values. It is

during this middle segment of the flyby where regressiontracking is most likely to occur due to the velocity and

(6)

184

185REPPERGER et al.: HUMAN TRACKING ABILITY

ZR

NIC -20

I_-

-lsi

-8s4-10B1 1i i -

a s to Is 2a 2STIME (SEC)

(a)

P s Is ICs 23 2

TINE CSEC)

38 3S 40

23 3S I40 M4

(b)

Fig. 2. (a) Target forcing function DT(t). (b) Velocity and acceleration of (DT(t)-

INITIAL REACQUISITION

-TRACKING -REGRESSION__OF THE TARGET

PHASE PERIOD AND FINAL

TRACKING

t0 t1

ERROR DOT

tf

Fig. 3. Segmentation of tracking task.

acceleration of the target forcing function. From Fig. 3 it isnoted that the time values t1 and t2 are unknowns whichcharacterize, in the time domain, the changes in trackingresponse behavior. Fig. 4 illustrates typical phase planes forthis type of tracking task. During the time period (to, t1),when the human has tracking response characteristics typi-cal of definition 1, the error signal oscillates about the originin a small ellipse. At the time t1, when the forcing functionstarts to dominate the closed-loop error signal (and con-

t2

Fig. 4. Closed-loop error phase plane.

03

U)

4)4)0

c0C.)4)C/)

U)4)4)

0

n3

ERROR

._ I.

.T(t)a0-

6B-

t2

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. SMC-9, No. 4, APRIL 1979

dependent on performance during a previous time segment.This aspect is discussed in [13] as well as in Poulton [14].Although Fig. 6 has symmetrical properties, (as does theforcing function), this result was not always typical of allflybys and teams. Ephrath and Kleinman [15], [16] have alsoreported results using a visual display with perceptual cues.In their experiment, as in some cases in this experiment, theerror signal did not always have the symmetrical property.One method of evaluating performance in this type of

tracking task is to study measures of the closed-loop errorsignal using different error metrics. In [17], [18]. as isdemonstrated here, two measures of performance arechosen, and the tracking task is studied by consideringdifferent metrics of the closed-loop error signal during thethree modes of behavior in Fig. 3. The first measure ofhuman tracking performance considered was the root mean-square error metric eRMS which is defined by

sequently when the tracking behavior changes during (t,t2)), the phase plane trajectories diverge significantly fromthe origin. To define Fig. 4 in a quantitative manner, Fig. 5 isconstructed. Inside of region I, the human's output isconstructively being used to reduce the error vector [e(t),e(t)] by matching up the plant's output with the forcingfunction. For simplicity, it will be assumed that region I canbe bounded by an ellipse described by

e2(t) + e(t)+ <1 (7)

for R1 > 0, R2 > 0. Thus, if the human can keep theclosed-loop error trajectory within region I, this will corre-

spond to definition 1 ofhuman tracking. Region II is definedin such a manner to include all responses not in region I.

The statistical algorithm in Section VI will estimate R 1,

R2, t1, and t2. To validate the algorithm, the results will becompared between a good and poor tracker. It is necessaryto now discuss the criteria for distinguishing between a goodand poor tracker.

V. PERFORMANCE DISTINCTION BETWEENA GOOD AND A POOR TRACKER

In this type of tracking task, the manner of distinguishingbetween a good and poor tracker is difficult. Fig. 6 illustrates

plots of the ensemble mean tracking error based on 16

replications of this tracking task for what is to be defined as a

good and poor tracker. A plot of the variance of the error

about the mean indicates that using mean error values to

distinguish between two trackers is not accurate due to the

large variance in the middle segment of the tracking task.

(Kleinman has pointed this out [11, 12].) It is noted that a

large value of the variance ofthe error signal near the middlesegment of the tracking task is due to a few unusual runs out

of the 16 trials. Also, the problem ofevaluating performanceusing this type of temporal measure is not desired because

task performance during one time segment is implicitly

CRMS

e2(ti)i= I

N(8)

where N is the number of time frames considered.As an approximation, values oft I and t2 were chosen to be

20 and 30 s, respectively, for all teams in the study. eRMS was

then calculated for phases I, II, and III defined in Fig. 3.Table I illustrates these performance results from the azi-

muth (horizontal tracker) operator of teams 2. 5, and 7chosen from the seven teams in the data base.As a comparison to this approach, it was decided to

consider a second measure ofhuman tracking performancedenoted as e hit score defined as follows:

Definitioni: e hit score implies

e(t)I < K3 (9)

for one sample of data; i.e., each time the error e(t) was

smaller than a window size K3, the numerical value of e hitscore increased one more integer unit. Using the data fromall 16 replications, Table II illustrates the performanceresults using this error metric for the azimuth operators ofthe three teams considered here.

It is observed that the results in Table I when compared to

Table II are not always consistent. Table III summarizes a

total comparison between the two measures ofperformanceof the azimuth operators considered here.

Table III shows consistent ranking of the trackers exceptfor phase III using time on target. With this one exception,the good tracker appears in team 7 and the poor tracker

appears in team 2. From this performance analysis, the

choice is made to call team 7 azimuth the good tracker and

team 2 azimuth the poor tracker. This describes the manner

in which it is possible to distinguish between a good and

poor tracker for this type of tracking task.

VI. THE ALGORITHM

The ensuing algorithm determines the boundaries of the

elliptical region of Fig. 5. In this paper the analysisprocedure will be applied to each subject individually. The

ERROR DOT

Fig. 5. Behavior regions I and II.

186

REPPERGER et al.: HUMAN TRACKING ABILITY

TABLE IAZIMUTH OPERATORS-eRMS VALUES

Phase I Phase II I Phase IIIAe5 Ae ARM

Team 2 2.6135 Differences 51.436 Differences 2.619 IDifferencesTeam 5 2.269 +.3445 39.126 +12.31 2.462 +.1570

Team 7 1.0506 +1.2184 28.8097 +10.3163 2.4423 +.0197

AZIMUTH TRACKERS-TABLE II

-HIT SCORE FOR WINDOW SIZE K3

PerformanceHierarchy Window Size-- (0°-1°)- (0°-2°)~(00-30) '0°3° > (0--9 (0°-15 (0-1°) (0°-30) (00-50'

Team 2 Team 2 Teas 2 Team 2 Team 2 Team 2 Team 7 Teas 2 Team 2Worst-- 508 823 931 510 860 1078 647 1].19 1172

Team S Team 5 Team 5 TeamS Team 5 Team 5 Team 5 Team 5 Team 7Mediocrc 554 847 933 606 944 1114 658 1121 1175

Team 7 Team 7 Team 7 Team 7 Team 7 Team 7 Team 2 1 eam 7 Teani 5Best- 6668 939 970 647 1009 1128 692 1129 1176

TABLE IIIPERFORMANCE HIERARCHY OF AZIMUTH OPERATORS (BY TEAM NUMBER)

ERS Hit Score7 7

Phase I 5 52 27 7

Phase II 5 52 27

Phase III 5 mixed2 __ _ _ _ _ __ _ _ _ _

A = Good Tracker0 = Poor Tracker

IT Zs2E

TIME CSEC)Fig. 6. Ensemble mean error.

3B 3S 4B 4E

4 -.

I

,E B.:

N:

B.

aWEU

Z -a.a;

E-B.:U-I

K -B.3awzhi -Bf.I

-B.

187

IEEE TRANSACTIONS ON SYSTEMS. MAN. AND CYBERNETICS, VOL. SMC-9. NO. 4, APRIL 1979

THE NUMBER OF POINTSWITHIN A SPECIFIEDWINDOW SIZE

-Ri -!e o &e RIFig. 7. Histogram of points.

need to proceed in this manner was due to intrasubjectvariability in this type of tracking task. As the tracking taskbecame more difficult to follow, the subjects respondeddifferently. The need to study individuals separately ismotivated by some of the work done in the earlier remnantstudies in tracking [23], [24].The desire to use an algorithm can be seen from Fig. 7

which illustrates a heuristic idea ofwhat one could expect ifahistogram of e(t) or e(t) were constructed when an extremeseparation existed between regions I and II. The horizontalaxis includes the range ofobserved values for e(t) or e(t). Fig.7 is essentially a side view distribution of the number ofpoints found in the phase plane in either the e(t) or the e(t)'axis based on a grid with a window size of Ae. Equal time(and consequently an equal number of points), however, wasnot spent in each mode of behavior during any one trial.Hence, a histogram of the actual data may differ from thedistribution of e(t). A histogram of I e(t) obtained using theempirical data from forcing function 1 is shown in Fig. 8.

It is noted from Fig. 8 that starting from zero, the numberof points within a window dropped substantially and thenleveled off as if a noise level was reached. The algorithm isdesigned to separate the noise at the extremes of thedistribution from the normal distribution in the center.

The AlgorithmStep 1: Starting from e = 0, go out + windows of width

Ae until all points are included in the sample.Step 2: Calculate the mean of the distribution (which is

approximately zero) and denote it as .N* Calculate thestandard deviation of all points and denote this as UN.

Step 3: Check if the number of points within x ± 3UN iSat least 99.7 percent of N.

Step 4: If the answer to step 3 is no, then replace N byN - 1, exclude the points in the window N, and recalculateXN-1 and UNI using only points in the first N - 1windows. Now return to step 3 and replace N by N - 1.

Step 5: If the answer to step 3 is yes, then stop because theboundary R1 has been determined. Similarly, the boundaryR2 is determined for e.Appendix A discusses the decision rule in Step 3 and

contains the proof of convergence of this algorithm inprobability.

TABLE IVGOOD TRACKER (TEAM 7-AZIMUTH)

Forcing Forcing Forcing ForcingFunction Function Function Function

.1 2 3 4

Rl .15 .40 .30 .28

R2IIZ.54 1.18 .86 .84

(t2 -t1)mean 0.0 13.587 10.648 8.8185

(t2 -t1) S.D. 0.0 4.006 2.8547 2.7235

$T mean(tl) - -9.095 -8.367 -10.80

§T S.D. (t ) - 1.979 1.708 2.783

3T mean(t1) - -.9938 -.5068 -1.8735

PT S.D.(t1) - .5738 1.032 .6495

t mean - 17.8667 23.9323 20.2277

t1S.D. _ 1.2836 2.1265 1.4537

t2 mean - 31.453 32.1169 29.046f2

t S. D. 3.6205 1.2700 1.99832_

§T mean(t2) - -2.9384 -1.2999 -4.583

iT S.D.(t2) 1.8579 1.1124 2.6871T mean(t2) .9q45 .688 1.006

T S.D.(t 2) - .562 .755 .601N - 9 13 13

Determiniation of t1 and t2Step 6: Using the values of the boundaries R1 and R2

from steps 1-5, the value of t1 of Fig. 3 will first bedetermined. The definition of t1 is that point in time whenthe error trajectory diverged outward from this ellipse andstayed outside the ellipse for at least 285 ms (20 datasamples). If the trajectory returned to region 1, it had toremain inside of the ellipse defining region I for at least 285ms.

Step 7: t2 was defined to be that point in time when theerror trajectory had returned inside region I from region IIfor at least 285 ms.

Step 8: To determine the critical boundaries, 4 T andi attime t were defined to be the velocity and acceleration oftheforcing function for t = t1 and t = t2.

VII. A STUDY ON THE CRITICAL BOUNDARIESFROM THE ALGORITHM

Tables IV and V illustrate the values obtained by applyingthe algorithm described in Section VI to the empirical datainvolving the four flybys. R l and R2 were computed over alltrials for the same subject for each trajectory. t l and t2 werecomputed for each trial and averaged for each subject andeach trajectory. P 7 andT 7 were then obtained at t, and t2 foreach trial and then averaged. It is now desired to quantify thecritical regions of tracking ability.

Fig. 9 illustrates a heuristic idea of a special phase plane ofall possible target functions that are studied here. Near theorigin of the &.-T plot, it is expected that normial tracking

188

.05 .10 .15 .20 .25 .30 .35 .40 .45

Window Size Ae = .01

Fig. 8. Actual histogram of distribution of e from data.

TABLE VPOOR TRACKER (TEAM 2-AZIMUTH)

Forcing Forcing Forcing ForcingFunction Function Function Funiction

1 . 2 3 4

R .20 .43 .34 .271

R2 .72 1.40 1.12 .92

(t2-tl)mean 11.80 13.194 10.942 10.056

(t2-t1)S.D. 4.140 3.166 4.723 2.844

iT mean(t1) -4.289 -5.798 -5.723 -7.672

_T S.D.(t 1) 1.250 2.863 3.183 2.761

{T mean(tl) -.2641 -1.6148 -1.0828 -1.746

T S.D.(t ) .1057 .4534 .3869 .4946

t mean 14.50 15.67 20.209 18.573

ti S.D. 3.932 1.877 3.649 1.592

t2 mean 26.288 28.863 131.151 28.629

t2 S.D. 1.455 3.847 2.086 2.264

mmean(t,) -3.8416 -4.3456 -2.499 -4.889T

FT S.D.(rt) .1869 3.62 3 2.764 2.078

§T mean(t2) .3582 .8460 .9998 1.1575

BT S.D.(t2) .01171 .3225 .7831 .5634

N 5 7 13 15

lEGIO

EGION

Fig. 9. Target phase plane.

REPPERGER et al.: HUMAN TRACKING ABILITY 189

0

C

.0

.C

.04

z

1700

1600

1500

1400

1300

1200

1100

1000

900

800

700

600

500

400

300

200

100

0.0

iTI~~~~~~~~I ! 7 - ----91tiII

!


1..

-r

'b (r)

/'

-to -5 a )T (t)

VELOCITY ( DEGREES/SEC)

AZ I mmTH TRRS.CTOR I ES

Fig. 10. 4T versus (DT for all four flybys.

will occur due to the fact that the forcing functions in thiscase are slowly moving. Region II ofFig. 9 should be outsideof some closed region and displaced from the origin.

Fig. 9 is only a heuristic guess on the shape of some

unknown region. Here only an approximation (rectangular)to this region will be obtained via the statistical algorithm.To make a guess on the shape of Fig. 9, a plot is madeparametrically with time in Fig. 10 using the (bT- 4T)curvesfor the four flyby functions in the data base. It is noted fromFig. 10 that forcing function 1 was the easiest to track (sinceit was closest to the origin) and had the least dispersion. Inforcing function 1, no regression tracking occurred for thegood tracker. Forcing functions 2, 3, and 4 were successivelymore difficult to track, and their distances from the originwere correspondingly larger. One may use as a measure ofdifficulty of the forcing function the distance from the originin this target phase plane.

Using step 8 of the algorithm, Fig. 11 is constructed byusing the smallest values of T and «T where regressiontracking occurred. This approximate region is a best guesson Fig. 9 based on the assumption that the forcing functionis equally difficult in either the right or left direction. Sincethe two boundaries are plotted for both the good and poortracker, this procedure allows a sensitive distinction be-tween the two types of trackers.From observation of Fig. 10 several additional points can

be made. One first notices in Fig. 10 that the good trackerreaches a threshold barrier region (time is clockwise in thefigure) for three out of the four forcing functions. The poortracker reaches this threshold region in all four forcing

functions but for smaller values of velocity and accelerationthan the good tracker. This is to be expected, since the goodtracker should follow a wider class offorcing function inputsas compared to the poor tracker. Also in Fig. 10 it appearsthat the threshold region is essentially characterized by a

straight-line barrier region. One also notices a hysteresiseffect in which the reacquisition region occurs for smallervalues of target velocity and acceleration than the regressionregion. This may be because the acquisition task requiresdifferent human tracking behavior as compared to the eventof regression from the tracking task. It is noted that Fig. 10also completely defines the critical region of possible reac-

quisitions of precognitive inputs. The lines of GH and EFdefine how slow a target trajectory must be before an

acquisition can occur. Fig. 10, therefore, gives a measure ofthe human threshold values for regression and reacquisitionfor all possible families of flyby targets for this fixed plantand display. It is also noted that in Fig. 10 the mean values oftarget velocity and acceleration may not fall on the curves.

This is because the coordinatewise average of points on a

curved line need not fall on the line; this does not effect anyof the results presented here. Table VI summarizes the fourstraight lines obtained from Fig. 10.

It is emphasized that the lines AB, CD, and GH have a

high correlation which is based on a straight line throughthree or four data points. The line EF has such a smallcorrelation that conclusions about this boundary cannot bemade.To study the critical boundaries for the good and poor

tracker, it is noted that Fig. 11 defines quite explicitly the

3

0

O

G

1,0

af

in-

ssC

Lbi-rsrJ

2

- I

- oF w -

190


40/Sec2jD (t)T

20/Sec2

90 2-2 /Sec_

Region II - Good Tracker

Degr

I

;T(t)rees/Sec

-40/sec21.Fig. 11. Comparison of regions of tracking capability (good tracker

versus poor tracker).

TABLE VICRITICAL TRACKING BOUNDARIES (FIG. 10)

Line Region Equation of Line Correlation Coefficient

AB Regressiu.i Barrier .T- *55T> 4.10/sec .99

CD Regression Barrier .; .424 > 1.3 /sec2 .882

EF Reacquisition Barrier XT <.80/sec .11

GH Reacquisition Barrier iT + .0970T <*6'/sec

differences that exist between the two trackers. Using Fig.11, one could design a target trajectory to cause a humantracker to employ regressive tracking if the target is outsideof region I and into region II. Also, the family of all knownforcing functions that can be tracked should fall within theinner rectangle. Thus a methodology or technique is estab-lished by which the characteristics of trajectories are knownwhich give rise to regression tracking behavior. In addition,the family of possible forcing functions which could beaccurately tracked is specified by region I in Fig. 11. Theseresults hold for a variety of precognitive tasks and can easilybe used as a predictive tool for evaluation of both taskdifficulty and human tracking capability. Finally, it is notedthat the good tracker has a larger region of trackingcapability and a smaller regression region than the poortracker.

Statistical tests are now conducted to study the impor-tance of the variables XT and qT during tracking regressionand for reacquisition of the target.

VIII. STATISTICAL TESTS ON THE

VARIABLES XT AND bT

It is desired here to conduct statistical tests on thevariables XT and qT at the times t1 and t2 over eachtrajectory and for each subject. The process ofusing statisti-cal tests in the study of human responses has been appliedpreviously in the area of manual control, e.g., in Wald testsstudying human monitoring [19], [20] in the failure-detection-type problems, and also in studies of the human'sability to estimate statistical properties of visual time series[21]. Recently [22], the use of identification and discriminantanalysis can be used to detect human shifts of attention in a

control and monitoring situation.The first test conducted here will study the effect of the

trajectory on the mean values of (PT and kT at regression andreacquisition for each subject. Appendix B describes indetail how the F-ratio tests were conducted. The purpose ofthese tests, summarized in Table VII, is to investigate if a

Region I - Good Tracker

Region I Poor Tracker

9 -7 5 4 -3-2-1 2 3 5 6 7 8 9

[

i

i

I

r l

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNFTICS. VOL. SMC-9, No. 4, APRIL 1979

TABLE VIITESTS OF EQUALITY OF MEANS ACROSS DIFFERENT TRAJECTORIES

Degrees ofTest Tracker Time Variable F Ratio Freedom Significant P

I Good t1 4.04 2,23 P < ,05

II Good tI fT 9.69 2,23 P < .01

III Poor t1 fT 2.31 3,36 P = .09

IV Poor t1 bT 17.79 3,36 P < .01

V Good 12 PT 8.59 2,31 p < .01

VI Goodt .82 2,31 m~~~~~~~ieans are notVI Good t 2 $T .82 2,31 significantly

_____ .____ ._____ .___ .__ different

VII Poor t2 PT 2.16 3,36 P = .11

. _ ._S.r __. _ ._ ___. _~~~~~~~~~~~Viii Poor t2 2.42 3,36 P = .08

variable is the same across all forcing functions. If this is true,then one would suspect this consistent variable as theprimary factor in causing regression tracking. Conversely, ifa variable showed wide variation across the different forcingfunctions, then one would not suspect this variable as theprimary factor causing the regression or reacquisition.

In these tables, P is the probability that the tested variablehas equal mean for all forcing functions. The conclusionsfrom Table VII can be stated as follows.

1) For both the good and poor tracker, neither T or «Tby itself causes regression tracking. This conclusion issupported by Fig. 10 in which straight lines (with highcorrelation and nonzero slopes) characterize the regressionregion. For the poor tracker, T does not seem to be thecause of regression. Since P = 0.09 for test III, this conclu-sion is not very strong.

2) For the good tracker, qT may be the primary variableof importance in the reacquisition ofthe target. This result issupported by test VI in Table VII.

3) Tests VII and VIII are inconclusive because of largedifferences in the variance.

It is noted that in the F-ratio tests conducted in Table VII,an implicit assumption is that the variances of the variables(0Tor qT) were equal across forcing functions. This was notthe case in four of eight tests. Appendix B studies in detailthis assumption of equality of variances and conductsfurther tests on the four cases where the variances showedwide variation. These further tests did not change theconclusions above.

Tests of Variances of The Variables XT and «TFrom Table VII, a study was conducted to see if a given

variable has the same mean across forcing functions. It isalso of interest to study if the good or poor tracker showeddifferences in variation in their tracking behavior. TableVIII summarizes a comparison of variability between thetwo trackers.

In Table VIII, if one tracker shows less consistency thanthe second tracker, this will manifest itself in a largerstandard deviation. If the standard deviations differ between

trackers, insight can then be obtained on the trackingstrategy of each tracker. The results of Table VIII indicatethat there exists some difference between the good and poortracker as far as consistency of behavior is concerned. Inthree of the 12 cases, the poor tracker shows significantlylarger standard deviations. In only one case does the goodtracker show a larger variability. Therefore, the conclusionis that the poor tracker is slightly more inconsistent in histracking behavior (across the same trajectory) then the goodtracker. Recall that Table VII is constructed across alltrajectories, but Table VIII is constructed for the replica-tions of one trajectory.

IX. SUMMARY AND CONCLUSIONS

Data from a precognitive tracking experiment areanalyzed using a statistical algorithm to delineate regions oftracking behavior. The tracking behavior is dichotomizedinto different regions (normal tracking and regressive track-ing) using target phase planes. A phase plane is constructedwhich defines responses to all possible families of precogni-tive tracking tasks for the simulation considered here. In thevalidation of the modeling procedure presented here, acomparison between a good tracker and a poor tracker isconducted. The algorithm is shown to converge in probabil-ity, and it gives rise to lines in the target phase plane(dependent on only properties ofthe target forcing function)which define limits of human tracking ability.

APPENDIX ADISCUSSION OF STEPS 1-5 OF THE ALGORITHM

AND PROOF OF CONVERGENCE

In this Appendix, a discussion of steps 1-5 of the algor-ithm will be conducted, and a proof of convergence inprobability will be given. The explicit assumption is madethat the points in region I are normally distributed withmean zero and standard deviation a1 . The probabilitydensity of the points in region I can be described by

f(= [s2x2o>e (Al)127o

192


TABLE VIIICOMPARISON BETWEEN TRACKERS OF STANDARD DEVIATIONS OF TARGET

VARIABLES

Time Forcing Staiidard n = Standard n DegreePeriod Variable Function Deviation obser. Deviation obser. F-Ratic of P

Nuniber Good Tracker Poor Tracker Freedom

t1 I T 2 1.979 9 2.863 7 2.06 6,8_

t1 ;TS 3 1.708 13 3.183 13 3.47 12,12 *

t1 T 4 2.783 9 2.761 15 1.02 12,14 -

t1 FT 2 .5738 9 .4534 7 1.60 8,6t1 _¢T 3 1.032 13 .3869 13 7.11 12,12 **

t1 IT 4 .649 13 .4946 15 1.72 12,14t2 ;T 2 1.8579 9 3.623 7 3.80 6,8 *

t2 FT 3 1.1124 13 2.764 13 6.11 12 ,1^__t2 XT 4 2.6876 13 2.078 15 1.67 12 ,14 _

t2 FT 2 .562 9 .3225 7 3.04 8,6 _

t2 FT 3 .755 13 .7831 13 1.08 12,12

t2 FT _4 601 13 .5634 15 1.14 12,14

The points in region II may have any distribution such thatregion I + region II does not appear normal. In step I ofthealgorithm, the procedure uses +N windows of width Aeuntil all points are included in the sample. The effectivemean is

N

XN= Xi -0° (A2)it=

and the effective variance 62 can be written:

N = E(XiXN- (A3)it=I N

Proceeding to step 3, the decision rule becomes

is (99.7 percent) of

N < the number of points within XN ± 3oN? (A4)If (A4) is satisfied, then the distribution of points is such thatthe proportion of points in region I + 1I is not above theexpected proportion from a normal distribution. If,however, (A4) is not satisfied, then the tail distribution ofregion I + region II has outliers which contribute to thefailure of the total distribution to appear normal. To correctthis situation, step 4 eliminates some of the outliers whichare due to points in region II. Replacing N by N - 1, the testfor normality is repeated K times until the mean andvariance become XN-K and U2NK' respectively, where

2 NK (X,i XN-K) (AS)itN-K N K I-i(5and XN -K defined similarly. The test (A4) is applied K timesuntil the inequality is finally satisfied. For K sufficiently

large, the outliers are eliminated, and the remaining pointssatisfy this normality test. R1 now becomes the windowvalue corresponding to XN - K + 3UN -K. Theorem A. 1 dem-onstrates that this procedure converges in probability be-cause of the central limit theorem.

Theorem A.l: Given a random sample {X1, ..., XN} froma normal distribution with mean 0 and variance a', thereexists (in probability) a radius R1 such that

3Sxo > R1 (A6)

where

(mxo1I )IXil<R1(A7)

where mxo is the number ofXi satisfying Xi < R 1, and SXOis the sample standard deviation. The definition of conver-gence and convergence in probability is as follows.

Definition A.1: The sequence of numbers X,n converges toa limit X (in probability) [25], [26] if, given any E > 0, thereexists an N such that

P{IXN XI >E}-°, forN -* oo. (A8)ProofJof Theorem A.l: For the independent samples Xi,

it can be shown [25] that

(mxo-1 s2\mxo ao

converges in probability to E{X2}, which is a consequence ofthe central limit theorem. By definition it is known that

RIE{X2} = 2 F x2g(x) dx (A9)

*0


/

/

I 1.X 2 2K.

TMCJNT Iam raPIUT

Fig. 12. Plot of E(X3) versus R2/9.

2

ar Units

'4

where the density function g(x) is of the form

hRl f(X),

0,

if |XI <R

otherwise.

(AlOa)

(AlOb)

f (x) is the probability density function of N(O, a2) distribu-tion, and the constant hRl satisfies:

hR~= 2 4f(x) dx. (All)RlR

For hRl specified by (A11), g(x) given by (Al0) satisfies theproperties of a probability density function. One can plotE{X } and R' /9 using (A9) and (A1) as shown in Fig. 12. Itis demonstrated here numerically that

E{X2}> R2/9 (A12)

whenever R1 < 3. This result can also be shown by seriesexpansion [25]. Using the inequality (A12), it follows that

S >Q Sk0- P-- E(X) > R19 (A13)

for R 1 < 3. Therefore, there exists (in probability) R 1 suchthat 3Sxo > R 1, and the proofofTheorem A. 1 is complete.

Q.E.D.

APPENDIX BANALYSIS OF VARIANCE RESULTS

OF TABLE VII

In order to compare properties of the target forcingfunction FT(t), which may be the important effect in regres-sion or reacquisition of the target, Table VII is constructedusing F-ratios. It is desired to test various hypothesesconcerning the good and poor tracker at the time periods t1

and t2. To illustrate the tests conducted here, test I will be

explained in detail. Tests II-VIII follow in the same manner;the only variables that change are the tracker (good or

poor), the time period (t1 or t2), and the target characteristic(4T or I). Since there are three conditions to consider withtwo choices of each condition, it is required to conduct23 = 8 tests.

Test I will consider the good tracker at time t1. The target

characteristic of interest in this test is whether XT iS the causeof regression at time t I. For the good tracker there are threevelocities at which regression occurred. The test is made on

the hypothesis Ho versus the alternative hypothesis HI:Ho: 02= ¢)3= 104H l: 41, i, j, 8[2, 4].

(BI)

(B2)

The hypothesis Ho is testing if the velocities at whichregression occurred are equal versus the alternate hypoth-esis H1 that there is a significant difference in these targetvelocities at time t1. To test this hypothesis, the grand mean41G is computed as follows:

(B3)4G = E E 4iji 3

where

i trajectory number = 2, 3, and 4,J 1 2, Ii,Ii the number of replications of trajectory i.

It is known that associated with each velocity is a

standard deviation ui of this velocity over the replicationsof the experiment. Fig. 13 illustrates a comparison of+, with

the grand mean 41, for the three trajectories considered here.

The problem of interest is whether there is more variation in

the individual trajectories across replications (measured bystandard deviations) as compared to differences betweenand 41G-

1 .1

I *'

U.'

O.

B.g

a O.S

194

leI-0, I

I

REPPERGER et al.: HUMAN TRACKING ABILITY

4i.2.-10 1-8

-6

-4

-2

0

4- - - .- - - - - - - - - -(D

= -9.4210 . G(D2 A.3

1L

2 3 4

TRAJECTORY NUMBER

Fig. 13. «i for each trajectory.

To test the hypothesis, a F-ratio is computed as the ratioof two variances:

F u/2V/N

where

(B5)U = E E ((/ij XG)2i i

i= 2, 3, 4

j= 1, 2, ., Ii

(B6)

(B7)

and Ii is the number of replications of trajectory i. Thenumber of degrees of freedom in the numerator of (B4) is 2,which is the number of trajectories minus one (3 - 1). Forthe denominator, the variable V satisfies

V= E (+ij-_i)2

where ij is the velocity for trajectory i, replication j, andN = Ji Ii-3 is the number of degrees of freedom in thiscase.One assumption in this statistical analysis is that the

variable (/ij - /i) is distributed normally. A secondassumption is that a = a 2 = (Ta3. In some of the tests thisassumption is violated, and the tests of the hypothesis areinconclusive. To test the assumption of equality ofvariances,use is made of Bartlett's x2 test [27]. The results are displayedin Table IX. From Table IX it is shown that the assumptionof equal variances is tenable for four ofthe eight tests used inTable VII. This implies that the conclusions are the same asbefore and are listed in Section VIII of this paper.For the four tests in which this assumption is acceptable,

conclusions can then be drawn from Table VII. For theremaining four tests in which this assumption is not tenable,another test is performed for equality of medians acrosstrajectories using a median test [28]. The results of this lasttest are shown in Table X. The results from Table X indicatethat the conclusions for tests IV and V are the same as inTable VII. Therefore, the results of Tables VII and VIII canbe accepted for these two tests. In each case there is a

TABLE IXTESTS FOR EQUALITY OF VARIANCES

Degrees of Analysis ofTest Tracker Time Variable X2 Freedom P Varianice

I Good t1 %T 2.90 2 NS _

II Good tl fT 4.03 2 NS _

III Poor tl fT 3.48 3 NS

IV Poor t1 *T 7.87 3 <.05 inconclusive

V Good t2 ;T 8.10 2 <.025 inconcl-usive

VI Good t2 T 1.00 2 NS

VII Poor t2 *T 18.87 3 <.005 inconclusive

VIIIj Poor t2 j T 30.93 3 <.005 inconclusive

TABLE X(B4) TEST FOR EQUALITY OF MEDIANS WHEN ANALYSIS OF VARIANCE (TEST FOR

EQUALITY OF MEANS) IS INCONCLUSIVE: P > 0.05 IS NONSIGNIFICANTDegrees of

Test Tracker Time Variable %f Freedom P

IV Poor t1 *T 11.48 3 <.01

V Good t2 *T 10.23 2 <.01I~~~~~~~~~~~~

VII

VIII

Poor t2 3.93 nonsigni ficaut

nonsignificant

3

i _ _- - i

Poor t2 6.44 3

significant difference in the variable across trajectories.Therefore the variable considered is not a cause of reacquisi-tion or regression.For tests VII and VIII in Table X, there is no significant

difference in medians across trajectories. This indicates forthe poor tracker that both median XT and kT are importantin reacquisition. This conclusion is not as solid as it wouldhave been if it were possible to use analysis ofvariance ratherthan a median test. The reason for this is that the median testhas less power of a test as compared to the analysis ofvariance. The power of a test is used in this context as theprobability of finding a significant difference when one trulyexists. The lack of a significant difference in tests VII andVIII could be genuine or possibly a true difference that wasnot discovered.

ACKNOWLEDGMENT

The authors would like to thank the reviewers for muchhelpful criticism which improved and corrected earlierversions of this paper. The authors also gratefully acknow-ledge the support, research facilities, and discussions withDr. C. R. Replogle and Dr. C. N. Day, which stimulatedmuch of this research.

-

195

I I


REFERENCES[1] H. R. Jex, J. E. McDonnell, and A. V. Phatak, "A critical tracking

task for man-machine related to the operator's effective delay time I:theory and experiments with a first-order divergent controlledelement," NASA CR-616, Oct. 1966.

[2] D. McRuer, D. Graham, E. Krendel, and W. Reisener, "Human pilotdynamics in compensatory systems-theory, models, and experimentswith controlled element and forcing function variations," AFFDL-TR-65-15, July 1965.

[3] T. B. Sheridan and W. R. Ferrell, Man-Machine Systems-Information, Control, and Decision Models of Human Performance.Boston: MIT Press, 1974.

[4] R. G. Costello, "The surge model of the well-trained human operatorin simple manual control," IEEE Trans. Man-Machine Syst., vol.MMS-9, no. 1, pp. 2-9, Mar. 1968.

[5] L. R. Young and J. L. Meiry, "Bang-bang aspects of manual controlin high-order systems," IEEE Trans. Automat. Contr., vol. AC-10, no.3, pp. 336-341, July 1965.

[6] A. V. Phatak and G. A. Bekey, "Model of the adaptive behavior ofthe human operator in response to a sudden change in the controlsituation," IEEE Trans. Man-Machine Syst., vol. MMS-10, no. 3, pp.72-80, Sept. 1969.

[7] T. Nishikawa, "Analysis of the behavioral system by using controltheory," First Int. Conf. Mathematical Modeling, University ofMissouri-Rolla, Aug. 1977.

[8] J. W. Senders and M. J. M. Posner, "A queueing model of monitoringand supervisor," in Monitoring Behavior and Supervisory Control, T.B. Sheridan and G. Johannsen, Eds. New York: Plenum Press, 1976,pp. 245-259.

[9] T. B. Sheridan, "Review of the symposium on monitoring behaviorand supervisory control," 12th Conf. Manual Control, NASATMX-73, 170, Univ. Illinois, May 1976.

[10] S. Baron, K. C. Chu, Y. C. Ho, and D. L. Kleinman, "A new approachto aerial combat games," NASA-CR-1626, Oct. 1970.

[11] D. L. Kleinman and B. Glass, "Modeling AAA tracking data usingthe optimal control model," 13th Ann. Conf. Manual Control, MIT,June 15-17, 1977.

[12] D. L. Kleinman and T. R. Perkins, "Modeling human performance ina time-varying anti-aircraft tracking loop," IEEE Trans. Automat.Contr., vol. AC-19, no. 4, pp. 297-306, Aug. 1974.

[13] D. W. Repperger, K. A. Smiles, J. A. Neff, and W. C. Summers, "Afeature selection approach in evaluation of parameter changes on the

human operator under thermal stress," Ergonomics, vol. 21, no. 1, pp.35-48, 1978.

[14] E. C. Poulton, Tracking Skill and Manual Control. New York:Academic Press, 1974.

[15] D. L. Kleinman and A. R. Ephrath, "Effects of target motion andimage on AAA tracking," 1977 IEEE Conf. Decision and Control,New Orleans, LA, Dec. 1977.

[16] A. R. Ephrath and B. S. Chernoff, "Effects of uncertainty on manualtracking performance," 14th Ann. Conf. Manual Control, Los An-geles, CA, Apr. 1978.

[17] D. W. Repperger, W. C. Summers, E. J. Hartzell, and G. D. Callin, "Aphase plane approach to study the adaptive nature of a human per-forming a tracking task," 1976 IEEE Conf. Decision and Control,Clearwater, FL, Dec. 1976.

[18] D. W. Repperger and E. J. Hartzell, "Performance measures ofhuman tracking utilizing PID modeling and a closed loop errormetric," 1976 NAECON Conf., Dayton, OH, 1976.

[19] E. Gai and R. E. Curry, "A model of the human observer in failuredetection tasks," IEEE Trans. Syst., Man, Cybern., Feb. 1976.

[20] R. E. Curry and T. Govindaray, "The human as a detector of changesin variance and bandwidth," Proc. 1977 Int. Conf Cybernetics andSociety, Washington DC, Sept. 1977.

[21] W. B. Rouse and K. D. Enstrom, "Human perception of the statisti-cal properties of discrete time series: Effects of interpolationmethods," IEEE Trans. Syst., Man, Cybern., vol. SMC-6, no. 7, pp.466-472, July 1976.

[22] *, "Real-time determination of how a human has allocated hisattention between control and monitoring tasks," IEEE Trans. Syst.,Man, Cybern., vol. SMC-7, no. 3, Mar. 1977.

[23] H. R. Jex, R. W. Allen, and R. E. Magdaleno, "Display format effectson precision tracking performance, describing functions, and rem-nant," AMRL-TR-71-63, Aug. 1971.

[24] W. H. Levison, S. Baron, and D. L. Kleinman, "A model for humancontroller remnant," IEEE Trans. Man-Machine Syst., vol. MMS-10,no. 4, pp. 101-108, Dec. 1969.

[25] A. J. Thomasian, The Structure oJ Probability Theory with Applica-tions. New York: McGraw-Hill, 1969.

[26] A. Papoulis, Probability, Random Variables, and Stochastic Process.New York: McGraw-Hill, 1965.

[27] C. R. Rao, Advanced Statistical Methods in Biometric Research.Hafner Publishing Co., 1970.

[28] B. W. Lindgren, Statistical Theory. New York: MacMillan, 1968, pp.458-460.

196

Documents

An Algorithm to Ascertain Critical Regions of Human Tracking Ability