Embed Size (px)
Citation preview
This article was downloaded by: [Moskow State Univ Bibliote]On: 22 November 2013, At: 12:21Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
The International Journal of AviationPsychologyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/hiap20
Psychological and PsychophysiologicalModels of Pilot Performance for SystemsDevelopment and Mission EvaluationErland A. I. Svensson & Glenn F. WilsonPublished online: 13 Nov 2009.
To cite this article: Erland A. I. Svensson & Glenn F. Wilson (2002) Psychological andPsychophysiological Models of Pilot Performance for Systems Development and Mission Evaluation, TheInternational Journal of Aviation Psychology, 12:1, 95-110
To link to this article: http://dx.doi.org/10.1207/S15327108IJAP1201_8
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Psychological and PsychophysiologicalModels of Pilot Performance
for Systems Development and Mission Evaluation
Erland A. I. SvenssonSwedish Defense Research Agency
Glenn F. WilsonU.S. Air Force Research Laboratory
Wright-Patterson Air Force Base, OH
The purpose of our study was to analyze the effects of mission complexity on pilotmental workload, situational awareness, and pilot performance and to develop mod-els by means of structural equation modeling. Earlier studies indicate that missioncomplexity affects mental workload and that mental workload affects situationalawareness, which, in turn, affects pilot performance. In the first phase of this study,20 fighter pilots performed 150 missions. In the second phase, 15 pilots performed40 simulated missions. The pilots answered questionnaires on mission complexity,mental workload, mental capacity, situational awareness, and performance. Duringthe simulated missions we measured eye fixation rate, heart rate, and blink rate.Model analyses show that mission complexity affects workload and that workloadaffects situational awareness and performance. Significant relationships occurbetween heart rate and rated workload, mental capacity, situational awareness, andperformance. Model analyses show a workload factor combining psychological andphysiological aspects and a performance factor combining situational awarenessand pilot performance. Significant relationships occur among heart rate, eye fixa-tion rate, and blink rate.
As a consequence of modern warfare and technological development, aircraft andtheir subsystems are more complex than ever before. This greatly increases the
THE INTERNATIONAL JOURNAL OF AVIATION PSYCHOLOGY, 12(1), 95–110
Requests for reprints should be sent to Erland A. I. Svensson, Swedish Defense Research Agency,Man-System Interaction, P.O. Box 1165, SE-581 11, Linköping, Sweden. E-mail: [email protected]
2451-c07.qxd 4/3/02 2:55 PM Page 95
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
amount of information pilots have to process before decision and action, and dur-ing complex missions pilots are reaching and surpassing their capacities toprocess information.
To analyze the pilots’ actual cognitive needs and to evaluate systems andmission developments, we must create reliable methods to assess pilot mentalworkload (PMWL), situational awareness (SA), and pilot performance (PP). Inaddition we must develop psychological and psychophysiological models ofthe pilots.
Researchers in the field have developed techniques for assessing PP and alsoconstructed different psychological indexes (Svensson, Angelborg-Thanderz,Sjöberg, & Olsson, 1997). Modeling by means of structural equation modeling(ad modum LISREL; Jöreskog, & Sörbom, 1984, 1993) indicates that missioncomplexity affects mental workload and that mental workload affects SA, which,in turn, affects performance. Svensson, Angelborg-Thanderz, and Sjöberg (1993),and Svensson (1997) presented models of how military pilots cope with increas-ing mission task loads.
The study consists of two phases: missions in the air and missions in the sim-ulator. The aims were to analyze the effects of mission task complexity on men-tal workload, SA, and performance; to develop causal models for systems devel-opment and mission evaluation; and to compare real and simulated missions.During the simulations the aims were to analyze the relationship between psy-chophysiological and psychological measures of mental workload and to inte-grate the psychophysiological measures into the psychological model developedfrom the missions in the air.
MISSIONS IN THE AIR
Method
Scenarios. During the first period one of the squadrons was acting as afighter interceptor group and the other as the enemy in a ground attack escortedby fighters. During the second period the enemy aircraft, now without fighterescort, maneuvered to escape. The Swedish multirole aircraft Viggen (JA 37) wasthe system we used.
Participants. Twenty active fighter pilots from two squadrons at theBlekinge Air Force Base (F17) participated; they performed 150 missions, ofwhich we have analyzed 144. The pilots’ mean time on aircraft JA 37 was 395 hr(SD � 137 hr).
96 SVENSSON AND WILSON
2451-c07.qxd 4/3/02 2:55 PM Page 96
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
Measures used. The pilots answered checklists before and after each mission.Before they rated their motivation, expected performance, perceived mental andphysical workload, and form (Svensson et al., 1993; Svensson, Angelborg-Thanderz,Sjöberg, & Gillberg, 1988; Svensson, Angelborg-Thanderz, & van Avermaete, 1997).
After the missions the pilots answered 90 assorted items from the NASA TaskLoad Index (NASA-TLX; Hart & Staveland, 1988) and the Bedford Rating Scale(BFRS; Roscoe & Ellis, 1990). The postmission questionnaire covered aspects ofmission difficulty, perceived performance, motivation, control, vigilance, mentalcapacity, mental and physical effort, SA, concentration, information load (tacticalsituation display [TSD] and target indicator [TI]), priority of tasks, interferencebetween tasks, availability, and complexity of information.
By means of factor analyses the large number of items was reduced to asmaller number of indexes. Their reliability was tested with Cronbach’s alpha.
Statistics. We used primarily correction statistics. Linear causal modelanalyses were performed by means of LISREL VI and LISREL VIII (Jöreskog &Sörbom, 1984, 1993). This procedure makes possible a statistical test of thevalidity of different causal flow models and their goodness of fit to a specificmodel in the population from which the sample has been drawn. The method usescorrelational and nonexperimental as well as experimental data to determine theplausibility of theoretical models hypothesized by the researcher. Hypothesizedin the models are causal structures among a set of unobservable constructs or fac-tors, each measured by a set of observed variables.
Results
By means of principal factor analysis, the items of the postmission questionnaireshave been reduced to nine indexes. Table 1 presents the indexes and the reliabil-ity values. The indexes have an acceptable to high reliability.
To get an initial opinion of the changes in PMWL, SA, and PP as a functionof the missions’ complexity, we divided the missions into five groups of increas-ing complexity. They range from Group A, which consists of simple training mis-sions, to Group E, which consists of applied missions of very high complexity.Figure 1 presents the changes of means of PMWL, SA, and PP as a function ofincreasing mission complexity.
First, a linear increase in PMWL over the five groups can be seen. One canalso see that SA and PP decrease over the groups. At first the decreases are small,but the changes of Groups D and E indicate critical decreases in SA and PP.
The conclusions based on the relative changes of the indexes in Figure 1 forman embryo to a model indicating that mission complexity affects PMWL and thatPMWL in turn affects SA, which, finally, affects PP.
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 97
2451-c07.qxd 4/3/02 2:55 PM Page 97
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
To test the credibility of this causal model we used structural equation mod-eling ad modum LISREL. In the analyses we used the following factors orindexes from Table 1: mission difficulty (DIFFIC), complexity of informationtactical situation display (COMP TSD), complexity of information target indi-cator (COMP TI), mental capacity reduction (CAPAC), situational awareness
98 SVENSSON AND WILSON
TABLE 1Reliability Values
Index Cronbach’s �
Pilot Performance (PP) .74Situational Awareness (SA) .80Difficulty (DIFFIC) .84Mental Effort (EFF) .86Pilot Mental Workload (PMWL) .87Ment. Capacity Red. (CAPAC) .77Motivation (MOTIV) .84Inform. Comp. TSD (COMPTSD) .92Inform. Comp. IT (COMP TI) .93
Group
High
Low A C D E
PMWL
SA
PP
B
FIGURE 1 Changes in PMWL, SA, and PP as a function of mission complexity. Group Aconsists of simple missions, and Group E consists of applied missions of very high complex-ity. Each group represents about 30 missions.
2451-c07.qxd 4/3/02 2:55 PM Page 98
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
(SA), and pilot performance (PP). We measured PMWL with the BFRS. Fig-ure 2 presents the final model.
The circles represent the different indexes or factors and the arrows representthe directions of the effects. The fit of the model is acceptable, and the model canbe generalized to the pilot population (goodness of fit index � .95, adjusted good-ness of fit index � .85, and root mean square � .053).
As Figure 2 shows, the model has its starting point in the difficulty of the mis-sions and its terminal point in the performance of the pilots. From the model wefind that increases in general mental workload (BFRS) in turn reduce mentalcapacity (CAPAC). Increases in information complexity on TSD and TI producea reduction of mental capacity of about the same size. Regression analyses showthat the common effect of COMP TSD, COMP TI, and BFRS accounts for 65%of the variance of the mental capacity index. The model indicates a strong con-nection between the information load on the displays and mental workload.
It is also evident from the model that increases in general workload (BFRS)and information complexity on TSD and TI both decrease SA. From analyses ofthe indexes COMP TSD and COMP TI, by means of multidimensional scaling,we find that the markers of each index form two clusters representing perceptualand cognitive aspects, respectively. From the clusters of the TSD and TI indexeswe formed two perceptual indexes (PERCTSD, PERCTI) and two cognitiveindexes (COGTSD, COGTI).
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 99
DIFFIC
BFRS
COMPTI
COMPTSD
CAPAC
.62
.30.34 .51
.37 .44
-.35
-.29
.17
.36
.64SA
A B C
PP
FIGURE 2 The final structural model of the relationships between six of the indexes andthe BFRS workload scale. The circles represent indexes or factors and the arrows representdirections of effects. All effects are significant (p < .05).
2451-c07.qxd 4/3/02 2:55 PM Page 99
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
Multiple regression analyses of the subindexes on the workload measureBFRS show that only the beta weights of the cognitive aspects are significant(.52 for COGTI and .49 for COGTSD, respectively, p < .001). The model (Fig-ure 2) can be divided into three consecutive parts: A, B, and C. Part A consistsof aspects of missions and systems demands, Part B comprises aspects of men-tal workload, information load, and mental capacity, and Part C includes SA andperformance aspects. The way pilots cope with the demands of the missionsforms an intermediary and compensating process affecting their performance.
Discussion
Missions in the air. We found that PMWL is comparatively sensitive toincreased information load. Increases in workload turn up earlier than decreasesin SA and performance. These differences reflect how the pilots cope with theload of the situation. They try to maintain their performance and SA. However,the performance decreases we found show that this compensation does not standfirmly to the end.
From model analyses we find that mission complexity affects differentaspects of mental workload and that these aspects in turn affect SA and PP. Themodel indicates a strong connection between the information load on the dis-plays and a reduced mental reserve capacity. That the pilots’ SA grew worse asa function of high information complexity on the displays indicates a bottleneckin the system.
It is interesting to note that the workload measures are more related to the cog-nitive aspects of the information complexity indexes than to the perceptualaspects. These findings support the position that cognitive aspects of informationhandling play a dominant role in modern aircraft cockpits.
The structural relationships among the indexes in the air form a framework forthe modeling in the study of missions in the simulator. In the simulation study weadded psychophysiological variables to the psychological indexes.
MISSIONS IN THE SIMULATOR
Unfortunately, it is hard to observe and measure cognitive processes directly;indirect measures are called for. Furthermore, military aviation represents a(hyper)dynamic situation, and dynamic situations need dynamic measures.Psychophysiological measures are genuinely dynamic, and accordingly theycan reflect the dynamics of the missions. In former studies (Berggren, 2000;Svensson, Angelborg-Thanderz, & van Avermaete, 1997) we used subjectiveratings repetitively as quasi-dynamic measures of PMWL, SA, and PP. By
100 SVENSSON AND WILSON
2451-c07.qxd 4/3/02 2:55 PM Page 100
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
means of this technique we can correlate psychological and psychophysiolog-ical changes reflecting the dynamics of a mission.
Methods
Scenarios. In Simulation Period 1 the observed pilot served as a flight leaderand the simulator instructor as his wingman. Together, their two aircraft formed anair defense fighter unit. Their task was to distract escorting enemy fighters from alarge enemy attack column so that other fighters could engage this attack column.
During Simulation Period 2 the participant was pilot in a solitary fighter. Anenemy attack column approached at low level and took no evasive action. Duringeach sortie, the mission was carried out twice. The scenarios were of very highcomplexity.
Participants. Fifteen fighter pilots from two squadrons at Blekinge AirForce Base performed a total of 40 simulated missions. We analyzed 35 of thosemissions. The pilots’ mean time on system JA 37 was 420 hr (SD � 121 hr).
Measures used. Before and after each mission the pilots answered the samequestionnaires as in the air study, and the same indexes were used. After each ofthe two intercepts during Simulation Period 2 the pilots responded to questionsabout mental workload (using BFRS), SA, and performance.
During the simulated missions the pilots’ heart rate (HR) and blink rate (BR)were recorded. The HR and BR data were analyzed with the Workload Assess-ment Monitor at the U.S. Air Force Research Laboratory.
The mean HR for 2-min periods, centered on the maximum HR during eachperiod of interest, was estimated. The periods of most interest were the interceptphases. In Simulation Period 2 we also used approach phases as comparisons tothe intercept phases. The pilots’ eye-point-of-gaze was videotaped. The durationsand frequencies of eye fixations on seven different areas of the instrument panelwere recorded manually. We used fixation rate (FR) as a general index of thepilots’ visual search behavior. This measure indicates how often the eye fixationchanges from one area to another per time unit (30 sec). Analyses of where thepilot is looking and the sequence of the fixations are of great interest with respectto training and cockpit design (Svensson, Angelborg-Thanderz, Sjöberg, et al.,1997; Svensson, Angelborg-Thanderz, & van Avermaete, 1997). Scanning behav-ior and fixation times are related to different aspects of PMWL (Harris &Christhilf, 1980; Itoh, Hayashi, Tsukui, & Saito, 1990; Kennedy, Braun, &Massey, 1995; May, Kennedy, Williams, Dunlap, & Brannan, 1990).
Statistics. The same statistics were used as in the study of the air missions.
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 101
2451-c07.qxd 4/3/02 2:55 PM Page 101
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
Results
The missions of Simulation Phase 2 consisted of two intercepts (A and B) ofequal complexity. To test the sensitivity of the 2-min HR means we compared themeans of Intercept A with the corresponding means of a preceding approachperiod. The means of Intercept B were compared with the means of a precedingperiod during which the pilots answered questions.
Paired sample t tests show that HR increases significantly from the firstapproach phase to Intercept Phase A (t � –4.39, p < .001) and from the questionperiod to Intercept Phase B (t � –3.51, p � .004). We found no significant differ-ences in HR between the two intercept phases or between the approach and ques-tion periods, respectively. Figure 3 represents a lucid example of how a pilot’sHR can change as a function of the phases of a sortie.
We have used the correlation between the two intercept phases as an estimate ofthe reliability of the 2-min mean HR measure. The correlation is .82 ( p < .001),meaning that 68% of the HR variance in the first intercept is common to the vari-ance in the second. The differences between the mean HR during the approach andintercept phases for each pilot provide crude measures of the HR reactivity. To ana-lyze the stability of this reactivity measure we compared the HR reactivity duringIntercept A and Intercept B. A significant covariation was found (r � .52, p � .05),which indicates that the reactivity measure is reliable. It also shows that there areinterindividual differences in HR reactivity.
The FR (number of fixations/30 sec) was analyzed in the same way as the HRmeasure. When comparing the mean of FR for Intercept A with that of Intercept B
102 SVENSSON AND WILSON
FIGURE 3 Illustration of how a pilot’s HR changes as a function of the different phases of asortie. Periods of special interest are in black. Each bar represents the HR mean for 10 sec.
2451-c07.qxd 4/3/02 2:55 PM Page 102
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
we found a significant covariation (r � .73, p < .01). This means that 53% of thevariance in FR of Intercept B is common to the variance in FR in Intercept A, and,accordingly, the reliable degree of variance is 53%. This shows that the FR mea-sure is reliable and that interindividual differences exist in the visual searchbehavior of the pilots.
The same indexes (see Table 1) were used during the simulated missions as inthe missions in the air. We found several significant correlations between theindexes and mean HR. The HR measure correlated positively with reduced men-tal capacity (CAPAC; .67, p < .001) and motivation (MOTIV; r = .41, p � .05) butcorrelated negatively with PP (r = –.45, p < .032), and HR tended to correlatenegatively with SA (r � –.38, p � .073).
To permit the testing of a model representing the simulated missions we com-bined the data from Phases 1 and 2 of the simulations. In the model analyses weused the indexes COMP TSD, COMP TI, CAPAC, SA, and PP from Table 1. Weadded the mean HR measure to the psychological indexes. When missions con-tained two intercepts, we derived a mean of the HR measures.
The fit of the model is acceptable (goodness of fit � .85, adjusted goodness offit index � .68, and root mean square � .110). The fit of this model is not as goodas that of the model from the air. The main statistical reason is the smaller num-ber of cases (35) of the correlations used in the analyses. This increases the ran-dom variation of the correlations and reduces the stability of the model.
As Figure 4 indicates, the model has its starting point in COMP TI and COMPTSD and its terminal point in PP. The complexity of information from the dis-plays TI and TSD has a strong effect on the mental capacity index. This meansthat high information complexity on the displays has a deteriorating effect onmental capacity. The markers of the mental capacity index deal with difficultiesin evaluating the synthetic information and the necessity to reduce the flow ofinformation. The mental capacity index has a strong effect on HR, and 45% of the
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 103
CAPAC SA
COMPTSD HR PP
.55
.65
.67
-.33
-.45COMPTI
.34
FIGURE 4 The final structural model of the relationships among five of the psychologicalindexes and the HR measure. All effects are significant (p < .05).
2451-c07.qxd 4/3/02 2:55 PM Page 103
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
variance in HR is explained by the variance in mental capacity. Thus, a decreasedmental capacity results in an increased psychophysiological activation. Thismeans that the effect of increased information complexity on HR is mediated bya reduced mental capacity.
Mental capacity has an effect on SA, which in turn affects PP. This means that areduced mental capacity restricts the pilot’s SA, which in turn reduces the pilot’s per-formance. Finally, HR has an effect on performance. The higher the HR, the worsethe performance.
The effects found in this model are the same as those found in the model fromthe air. In the analyses of that model we find that the effects can be divided intothree consecutive parts: one consisting of aspects of missions and systemsdemands, one composing aspects of mental workload, information load, andmental capacity, and one including SA and performance aspects. These three con-secutive parts are also found in the simulation model. The way pilots cope withthe demands of the mission forms an intermediary and compensating processaffecting their SA and performance.
Figure 4 shows a relationship between PP, SA, mental capacity reduction, andHR. These four aspects were used to derive two new second-order factors toexamine the relationship between aspects of PMWL and PP on a more generallevel.
Figure 5 presents a LISREL analysis of the indexes CAPAC, SA, PP, andHR. The fit of the model is almost perfect. Adjusted goodness of fit is .93, androot mean square is .017. The mental capacity reduction index and HR wereoptimally combined to form a second-order factor called general workload(GENWL). The SA index and the PP index were combined to form a new second-order factor called general performance (GENP). As Figure 5 shows, the fac-tor loadings of the second-order factors are very high. We also find a strong
104 SVENSSON AND WILSON
HR
SACAPAC .83
.80
.67
PP.68
-.77 Gen.Perform.
Gen.Workload
FIGURE 5 A structural model based on the relationships among mental capacity reduction(CAPAC), heart rate (HR), situational awareness (SA), and pilot performance (PP). Factorsare denoted by ellipses and manifest variables by squares. Factor loadings are presented in ital-ics. All coefficients are significant (p < .01).
2451-c07.qxd 4/3/02 2:55 PM Page 104
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
negative effect of GENWL on GENP. The former factor explains 60% of thevariance of the latter.
The general workload factor combines psychological and psychophysiologicalaspects. The factor is based on seven manifest variables (six items on mentalcapacity and one on HR).
Both SA and PP are equally weighted in the GENP factor, which is based on10 manifest variables. The GENP factor shows that SA and PP are differentaspects of the same concept (Angelborg-Thanderz, 1997).
To test for a curve-linear relationship between GENWL and GENP, we per-formed separate regression analyses for workload values below and above themean, respectively. For workload values below the mean the correlation betweenGENP and GENWL is –.02 (p � .93), and for workload values over the mean itis –.59 ( p = .013). Thus, we found a negative curve-linear relationship. Thecurved relationship means that the rate of performance impairment increases as afunction of increases in workload. This empirical result is in accordance with the-ories of the relationship between mental workload and performance (O’Donnell& Eggemeier, 1986; cf. Lysaght et al., 1989).
After each of the two intercepts during Simulation Period 2 the pilots wereasked to respond to BFRS and to a 7-point performance scale. Thus, the pilotswere asked to rate workload, SA, and performance directly after each interceptduring the mission.
Except means of r-minutes HR periods, we also used eye FR (changes in fix-ations per 30 sec). BR was excluded because of missing data.
For the postintercept measures we find significant correlations between per-formance ratings and the ratings of SA (r � .52, p < .01), between mental work-load and HR (r � .49, p < .01), between HR and FR (r � .45, p < .01), andbetween mental workload and SA (r � –.46, p < .01). Figure 6 presents the LIS-REL model that was the input for this correlation matrix.
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 105
HR
SABFRS .84
.57
.90
PERF.56
-.45 PerformanceWorkloadFIXRATE.28
FIGURE 6 A structural model based on the relationships among mental workload (BFRS),fixation rate (FR), heart rate (HR), situational awareness (SA), and performance ratings(PERF). Factors are denoted by ellipses and manifest variables by squares. Factor loadings arepresented in italics. All coefficients are significant (p < .01).
2451-c07.qxd 4/3/02 2:55 PM Page 105
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
The fit of the model is acceptable (adjusted goodness of fit index � .75 androot mean square � .112). The ratings of mental workload by means of BFRS,the FR, and HR are significant markers of the workload factor. This means thatan increased activity in the pilot’s visual search behavior, an increase in the HR,and an increase in the pilot’s perceived mental workload go together in a work-load factor. It is of special interest that two psychophysiological variables corre-late with a psychological variable. The ratings of performance and SA are sig-nificant markers of a performance factor.
It is notable that the same structure as in Figure 5 was derived even though dif-ferent input variables were used here. This adds credibility to the notion that theunderlying structure and the relationships between the factors are valid androbust.
Inspection of the dynamic changes of FR, BR, and HR of all missions, asa function of mission time, shows that the measures change systematicallyover time. Mission phases with higher cognitive demands and higher work-load (the intercept phases) cause increases in HR and FR but cause decreasesin BR. Even if there are differences between the pilots with respect to thechanges in the three measures, we conclude that these relationships recur dur-ing the intercepts of our scenarios. Figure 7 presents a generic model of therelationships.
106 SVENSSON AND WILSON
HR
BR
FR
WEAPONSDELIVERY
FIGURE 7 A generic model of the relationships among blink rate (BR), fixation rate (FR),and heart rate (HR) during air-to-air intercepts.
2451-c07.qxd 4/3/02 2:55 PM Page 106
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
Changes (in terms of increased FR and decreased BR) observed prior toweapons delivery indicate that the pilots are searching the visual environment andreducing their blinking in order not to miss significant target information.Decreased blinking has the effect of reducing the probability of missing signifi-cant visual stimuli because one is temporally blind during the eye closure.
HR was found to be increasing, reaching a peak at the time of weapons deliv-ery (cf. Angelborg-Thanderz, 1989). These observations indicate that eye andheart activity are tied to the activities required for successful performance duringthe engagement.
Discussion
Simulated missions. The same questionnaires were used in both the sim-ulation study and the study in the air. In addition to these measures psychophys-iological variables were used. Furthermore, in the second period of the simula-tion study, the pilots responded to questions about mental workload,performance, and SA immediately after each of two consecutive intercepts.
A wealth of empirical data shows that HR is a sensitive measure under dif-ferent circumstances in both real and simulated missions (Angelborg-Thanderz,1989; Wilson, Purvis, Skelly, Fullenkamp, & Davis, 1987; Wilson, Skelley, &Purvis, 1988). The reactivity of HR to the changes in cognitive load over themission was also documented in this study. We find significant differences inHR (means over 2 min) between approach and intercept phases.
When comparing the mean of the FR for the consecutive intercepts, we find asignificant covariation. This shows that the FR measure is reliable and that thereare interindividual differences in the pilots’ visual search behavior.
When comparing the model in the air (Figure 2) with the corresponding modelfrom the simulation (Figure 4), we find that the sequential effects of informationload on mental workload and the effects of workload on PP are the same. As in themodel from the air study, the complexity of information from the interaction of thedisplays TI and TSD has a strong effect on mental capacity. Several significant cor-relations occur between the psychological indexes and HR. Of special interest is thehigh correlation between mental capacity and HR; the variance in mental capacityexplains 45% of the variance in HR. Thus, a decreased mental capacity results inan increased psychophysiological activation. This means that the effect of increasedinformation complexity on HR is mediated by a reduced mental capacity.
The models in Figures 5 and 6 present a general workload factor, which combinesboth psychological and psychophysiological aspects. When the pilot’s mental capac-ity was reduced, there was an increase in his psychophysiological reactions.
We also found that SA and PP were equally weighted in a performance factor.This result shows that SA and performance are different aspects of the same concept.
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 107
2451-c07.qxd 4/3/02 2:55 PM Page 107
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
It is important to establish the relationships among SA, mental workload, andPP. According to Endsley (1995) SA is considered to be a precursor to the pilot’sdecision making and a stage separate from performance. In our opinion, SA isoften so closely related to the pilot’s performance that it is more logical to con-sider it a part of performance. In our research, PP includes SA aspects such as tar-get detection and target identification.
In ongoing research we develop algorithms that permit the detection of highinformation input (bottlenecks) prior to weapons delivery during (simulated andreal) attack missions. In the first phase we analyzed the relationships among HR,HR variability, and FR over mission phases. (For a presentation of these studiessee Magnusson, this issue). In the second phase we include electroencephalogrammeasures. The algorithms could be useful for cockpit design and mission analy-sis. They could be used in training of optimal visual search behavior. Further-more, data from the algorithm could be used as input to the flight and weaponssystems (adaptive aiding; Hankins & Wilson, 1998). If the aircraft systems canadapt to the pilot’s workload and performance levels by means of this technique,the total pilot and machine performance will be improved.
GENERAL CONCLUSIONS
We have found and verified the internal relationships among the central aspects ofmental workload, SA, and performance. We have demonstrated how they changeas a function of the complexity of the missions performed. From the model analy-ses we concluded that mission task complexity affects workload and that workloadaffects SA and performance. The similarity between models from real and simu-lated missions was established. We were successful in compiling psychophysiolog-ical and psychological variables into factors. In the same way SA variables and per-formance variables were compiled to a performance factor. This illustrates themultifaceted nature of the concepts of mental workload and PP. The dynamicchanges of HR, FR, and BR during mission phases of high complexity show impor-tant relationships in analyses of mental and systems bottlenecks. We have demon-strated that the interaction between pilot and aircraft can be analyzed by means ofreliable and valid psychological and psychophysiological measures and models.
ACKNOWLEDGMENTS
The article is based on the technical report “Models of Pilot Performance for Sys-tems and Mission Evaluation: Psychological and Psychophysiological Aspects”(Svensson, Angelborg-Thanderz, & Wilson, 1999).
108 SVENSSON AND WILSON
2451-c07.qxd 4/3/02 2:55 PM Page 108
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
REFERENCES
Angelborg-Thanderz, M. (1989). Assessing pilot performance in training simulators: A structuralanalysis. In Proceedings of the 1989 Spring Convention: Flight simulation: Assessing the benefitsand economics. London: The Royal Aeronautical Society.
Angelborg-Thanderz, M. (1997). Military pilot performance: Dynamic decision making in itsextreme. In R. Flin, E. Salas, M. Strub, & L. Martin (Eds.), Decision making under stress: Emerg-ing themes and applications (pp. 225–232). Aldershot, England: Ashgate.
Berggren, P. (2000). Situational awareness, mental workload, and pilot performance: Relationshipsand conceptual aspects (FOA-R-00-01438-706-SE).
Endsley, M. R. (1995). Theoretical underpinnings of situational awareness: A critical review. InD. Garland & M. R. Endsley (Eds.), Experimental analysis and measurement of situational aware-ness. Daytona Beach, FL: Embry-Riddle Aeronautical University Press.
Hankins, T. C., & Wilson, G. F. (1998). A comparison of heart rate, eye activity, EEG and subjectivemeasures of pilot mental workload during flight. Aviation, Space, & Environmental Medicine, 69,360–367.
Harris, R. L., Sr., & Christhilf, D. M. (1980). What do pilots see in displays? In Proceedings of theHuman Factors Society Meeting. Santa Monica, CA: Human Factors Society.
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results ofempirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental work-load (pp. 139–183). Amsterdam: Elsevier.
Itoh, Y., Hayashi, Y., Tsukui, I., & Saito, S. (1990). The ergonomic evaluation of eye movements andmental workload in aircraft pilots. Ergonomics, 33, 719–733.
Jöreskog, K. G., & Sörbom, D. (1984). LISREL VI: Analysis of linear structural relationships by max-imum likelihood, instrumental variables, and least squares methods. Unpublished manuscript,Department of Statistics, University of Uppsala, Sweden.
Jöreskog, K. G., & Sörbom, D. (1993). LISREL VIII: Structural equation modeling with the SIMPLIScommand language. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Kennedy, R. S., Braun, C. C., & Massey, C. J. (1995). Performance, workload, and eye movementactivity (Rep. No. SAE Paper 951570). San Diego, CA: SAE.
Lysaght, R. J., Hill, S. G., Dick, A. O., Plamondon, B. D., Linton, P. M., Wierwille, W. W., Zaklad,A. L., Bittner, Jr., A. C., & Wherry, R. J. (1989). Operator workload: Comprehensive review andevaluation of operator workload methodologies (Tech. Rep. No. 851, MDA 903-86-C-0384).United States Army Research Institute for the Behavioral Sciences.
May, J. G., Kennedy, R. S., Williams, M. C., Dunlap, W. P., & Brannan, J. R. (1990). Eye movementsindices of mental workload. Acta Psychologica, 75, 75–89.
O’Donnell, R. D., & Eggemeier, F. T. (1986). Workload assessment methodology. In K. R. Boff,L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (Vol. II,pp. 42-1–42-49). New York: Wiley.
Roscoe, A. H., & Ellis, G. A. (1990). A subjective rating scale for assessing pilot workload in flight:A decade of practical use (Tech. Rep. No. 90019). Royal Aerospace Establishment.
Svensson, E. (1997). Pilot mental workload and situational awareness: Psychological models of thepilot. In R. Flin, E. Salas, M. Strub, & L. Martin (Eds.), Decision making under stress: Emergingthemes and applications (pp. 225–232). Aldershot, England: Ashgate.
Svensson, E., Angelborg-Thanderz, M., & Sjöberg, L. (1993). Mission challenge, mental workloadand performance in military aviation. Aviation, Space, and Environmental Medicine, 64, 985–991.
Svensson, E., Angelborg-Thanderz, M., Sjöberg, L., & Gillberg, M. (1988). Military flight experienceand sympatho-adrenal activity. Aviation, Space, and Environmental Medicine, 59, 411–416.
PSYCHOLOGICAL AND PSYCHOPHYSIOLOGICAL MODELS 109
2451-c07.qxd 4/3/02 2:55 PM Page 109
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
Svensson, E., Angelborg-Thanderz, M., Sjöberg, L., & Olsson, S. (1997). Information complexity:Mental workload and performance in combat aircraft. Ergonomics, 40, 362–380.
Svensson, E., Angelborg-Thanderz, M., & van Avermaete, J. (1997). Dynamic measures of pilot men-tal workload, pilot performance, and situational awareness (Tech. Rep. No. VINTHEC-WP3-TR01). Amsterdam: NLR.
Svensson, E., Angelborg-Thanderz, M., & Wilson, G. F. (1999). Models of pilot performance for sys-tems and mission evaluation: Psychological and psychophysiological aspects (AFRL/HE-WP-TR-199-0215). Wright-Patterson AFB, OH: Air Force Research Laboratory.
Wilson, G. F., Purvis, B., Skelly, J., Fullenkamp, P., & Davis, I. (1987). Physiological data used tomeasure pilot workload and actual flight and simulator conditions. In Proceedings of the HumanFactors Society, Thirty-First Annual Meeting. Santa Monica, CA: HFES.
Wilson, G. F., Skelley, J., & Purvis, B. (1988). Reactions to emergency situations in actual and simu-lated flight. In Proceedings of the AGARD AMP Symposium on Human Behavior in High StressSituations in Aerospace Operations. Paris: AGARD.
Manuscript first received May 2001
110 SVENSSON AND WILSON
2451-c07.qxd 4/3/02 2:55 PM Page 110
Dow
nloa
ded
by [
Mos
kow
Sta
te U
niv
Bib
liote
] at
12:
21 2
2 N
ovem
ber
2013
