A Comparison of a Brain-Based Adaptive System and a Manual Adaptable System for Invoking Automation

Nathan R. Bailey Mark W. Scerbo Frederick G. Freeman

Old Dominion University Peter J. Mikulka Lorissa A. Scott

Abstract

Two experiments are presented that examine alternative methods for invoking automation. In each experiment, participants were asked to perform simultaneously a monitoring task and a resource management task as well as a tracking task that changed between automatic and manual modes. The monitoring task required participants to detect failures of an automated system to correct aberrant conditions under either high or low system reliability. Performance on each task was assessed as well as situation awareness and subjective workload.

&at used theii EEG signals to switch the tracking task between automatic and manual modes. The remaining participants were yoked to participants from the adaptive condition and received the same schedule of mode switches, but their EEG had no effect on the automation. Within each group, half of the participants were assigned to either the low or high reliability monitoring task. In addition, within each combination of automation invocation and system reliability, participants were separated into high and low complacency potential groups. The results revealed no significant effects of automation invocation on the performance measures; however, the high complacency individuals demonstrated better situation awareness when working with the adaptive automation system.

The second experiment was the same as the first with one important exception. Automation was invoked manually. Thus, half of the participants pressed a button to invoke automation for 10 s. The remaining participants were yoked to participants from the adaptable condition and received the same schedule of mode switches, but they had no control over the automation. The results showed that participants who could invoke automation performed more poorly on the resource management task and reported higher levels of subjective workload. Further, those who invoked automation more frequently performed more poorly on the tracking task and reported higher levels of subjective workload.

and the adaptable condition in the second experiment revealed only one sigaificant difference: the subjective workload was higher in the adaptable condition. Overall, the results show that a brain-based, adaptive automation system may facilitate situation awareness for those individuals who are more complacent toward automation. By contrast, requiring operators to invoke automation manually may have some detrimental impact on performance but does appear to increases subjective workload relative to an adaptive system.

Ir? the first experhent, half of Lhe participants worked with a brain-based system

A comparison of participants from the adaptive condition in the first experiment

https://ntrs.nasa.gov/search.jsp?R=20040084079 2020-03-19T19:28:29+00:00Z

INTRODUCFION

Automation can be characterized as the execution by a machine of a function previously carried out by a human (Parasuraman & Riley, 1997). The widespread implementation of automation in complex systems such as transportation? process control, decision support systems, and quality control and maintenance has been the result of anticipated improvements in system performance, efficiency? and safety. These improvements have been generally realized. Within the context of commercial aviation, automated systems have made it possible to reduce flight times, improve fuel efficiency and passenger comfort, navigate more effectively, and to improve the perceptual and cognitive abilities of the crew (Wiener, 1988). However, the increase in both quantity and complexity of advanced automated systems has raised a number of potential concerns including increased operator workload, degraded monitoring skills, loss of task proficiency, and reduced situation awareness (Endsley, 1996; Parasuraman, Molloy, & Singh, 1993; Wiener & Curry, 1980; Wiener, 1988). In response to these concerns, some researchers have suggested that one potential way of abating the negative effects 2sswiated with mm~mti~zd aatomation is &-mi@ tlic iise of adaptive automatim (Hancock, Chigicfi, & 'awenthall, 1985; Parasuraman, Bahri, Deaton, Morrison, & Barnes, 1992; Scerbo, 1996).

Adaptive automation refers to systems where the level or̂ automation or the number of systems operating under automation can be modified in real time (Scerbo, 1996). These systems can adjust their methods of operation, allowing for a restructuring of the task environment based on evolving situational demands. It has also been suggested that the dynamic allocation of functions that o m s as part of an adaptive automation paradigm may represent the best match between task demands and the cognitive resources available to an operator (Parasuraman et al., 1992; Rouse, 1976). As a result, some researchers have argued that adaptive automation can be used to enhance performance (Parasuraman, Mouloua, Molloy, & Hilburn, 1996), reduce workload (Scerbo, 1996), and improve situation awareness (Kaber, Riley, Tan, & Endsley, 2001).

Review of Previous Research

Empirical investigations of the efficacy of adaptive automation have focused primarily on the methods used to invoke the automation or the performance implications of using adaptive systems. With respect to methods of invocation, one of the most critical challenges facing designers of adaptive systems is how to appropriately switch between the levels and/or modes of automation (Scerbo, 1996). There are several potential candidate triggering mechanisms in adaptive automation systems including critical events, operator performance measurement, operator modeling, physiological assessment, and hybrid methods (Parasuraman et al., 1992).

The critical events method uses specific tactical events to trigger changes in the mode or level of automation. This method has been characterized as flexible? assuming adequate planning by the designers, and provides for integration with conventional tactics and doctrine (Parasuraman & Byme, 2003). However, the critical events methodology has the inherent disadvantage of being insensitive to the performance of the system and the human operator. Hancock and Chignell(1987,1988) have discussed real-time

assessment of operator performance. They argue that deviations from expected performance can be used as a measure of workload for invoking task allocations. Although this approach is sensitive to operator performance, it has been criticized as being too general and information intensive, requiring a massive database of operator performance (Scerbo, 1996). As such, an alternative method uses models of operator performance (Morrison & Gluckman, 1994; Rouse, Geddes, & Curry, 1988). Under this method, automation is invoked based on models of human performance that take into account current system states, external events, expected operator actions, and models of past behavior.

In contrast with critical events and performance-based approaches, some researchers have suggested the use of physiological measures for initiating system changes (Parasuraman et al., 1992; Scerbo, Freeman, & Mikulka, 2OOO). These systems are predicated on the notion that psychological states are associated with changes in operator physiology. By measuring the physiological signals that represent these states, it is possible to allocate system functions dynamically, maintaining an optimal level of physiological arousal. Consequently, an operator’s attentional and cognitive resources for perfemkg +ah are impraved (Gaillard, 1393).

There are several a&vantages to using psychophysioiogid measures for triggering changes in an adaptive system. Psychophysiological measures can be recorded continuously in a relatively unobtrusive way (Byme & Parasuraman, 1996). In addition, for many complex operating environments, users have very little opportunity for making overt responses despite a considerable amount of cognitive activity. Under these circumstances, behavioral measures are insensitive and provide a poor sample of an operator’s mental activity. Psychophysiological measures, by contrast, can be recorded without respect to overt responses and may provide a better measure of underlying cognitive activities (Parasuraman et al., 1992).

using physiological measures for triggering dynamic changes in adaptive systems. Specifically, both pupillometric and heart rate variability (Hilbum, Joma, Byme, & Parasuraman, 1997) as well as EEG (Freeman, Mikulka, Scerbo, Prinzel, & Clouatre, 2OOO; Pope, Bogart, & Bartolome, 1995; Prinzel, Freeman, Scerbo, Mikulka, & Pope, 2OOO) have been studied. Both types of systems have yielded improvements in operator

The most promising work on adaptive systems that use EEG signals stems from

Accordingly, some researchers have examined the performance implications of

pelfOIIllance.

research on a biocybemetic or brain-based system developed by Pope, Bogart, and Bartolome (1 995). Their system uses the operator’s EEG to derive an engagement index from a ratio of EEG powerbands including alpha, beta, and theta. Deviations in the engagement index are then used to trigger task allocations in the adaptive system. A system such as this can be used to maintain moderate levels of workload and arousal, optimizing the relationship between task demands and an operator’s immediate ability to complete a task.

A number of recent studies have shown that performance can be enhanced using the system developed by Pope et al. (1995). For example, Freeman, Mikulka, Prinzel, and Scerbo (1999) had participants operate a modified version of the Multi-task Attribute Battery (MAT Comstock & Amegard, 1992). The MAT is comprised of several tasks that represent activities commonly performed by pilots. Freeman et al. recorded a

baseline index of operator engagement and used deviations fiom that baseline to guide task allocations for the compensatory tracking portion of the MAT. They found that manual tracking performance was improved using the adaptive system. Subsequent studies by Freeman et al. (2OOO) and Prinzel et al. (2OOO) have replicated these results. Taken together, these findings demonstrate the viability of a brain-based adaptive automation system for facilitating performance.

Choosing to Use Automation

Although there is evidence that a brain-based adaptive automation system can facilitate human performance, at present, there is no research addressing whether the performance benefits associated with a brain-based system exceed those of systems where the user has control over task allocations. Recently, Scerbo (2001) described a twofold taxonomy of adaptive technology. The first dimension concerns how changes among states or modes of automation are invoked. For adaptable systems, the user initiates changes among presentation modes or functionality. By contrast, in truly adaptive systems, both the user and the system can initiate changes among system states or modes. Scerbo’s taxonomy makes a qualitative distinction between adaptive and adaptable technology suggesting that operator performance may vary between the two types of systems. However, rather than contrasting operator performance between adaptive and adaptable systems, previous research has focused on the underlying factors influencing operator decisions to use automation such as subjective trust, operator self- confidence, and task complexity (Lee & Moray, 1994; Muir, 1987,1994; Muir & Moray, 1996; Parasuraman et ai., 1993). In addition, Lee and Moray (1992) and Riley (1994a) have argued that operators show an overwhelming preference for retaining manual control even when a task can be automated. This inclination demonstrates an operator’s reluctance toward using adaptable systems, despite their potential benefits. As such, given the lack of research comparing the efficacy of adaptive versus adaptable technology and the preference for manual control described by Lee and Moray and Riley, it is critical to compare systems that employ task allocation strategies under the control of the operator versus those where changes occur automatically.

The primary purpose of the present investigation was to compare operator performance with adaptive and adaptable interfaces, (i.e., where task allocations occurred automatically resulting fiom changes in an operator’s physiological engagement versus allocations that were initiated manually). It was expected that performance with an adaptive system would exceed that of the adaptable system. Specifically, participants using the brain-based system would demonstrate improved performance over those operators who were required to invoke automation themselves because the adaptive system provides for a tighter coupling between task demands and the available cognitive resources of the operator. A second goal of the present study was to determine whether using adaptive automation for one task would generalize to improved performance on another task. Adaptive systems have been shown to improve primary task performance. However, can adaptive systems improve performance on additional concurrent tasks? Kaber and Riley (1999), using an adaptive system proposed to regulate mental workload, found that in addition to enhanced primary task performance, operators demonstrated increased performance on a secondary monitoring task. Therefore, within the present

study, it was expected that use of an adaptive system would enhance performance on another task. Consistent with Kaber and Riley, those operators using an adaptive system would demonstrate enhanced performance on a secondary system monitoring task relative to those individuals in the adaptable, user-initiated system.

Automation-induced Complacency

The increase in both quantity and complexity of advanced automated systems has also resulted in an increased demand for operators to monitor systems for failures or malfimctions (Wiener & Curry, 1980). One potentially negative consequence resulting from increased monitoring demands has been referred to as automation-induced complacency (hnwa.man et al., 1993; Thackray & Touchstone, 1989). Automation- induced complacency refers to the decline in monitoring performance that often follows the shift from performing a task manually to monitoring the automation, particularly with highly reliable, automated systems where operators serve in a backup role (Farrell & Lewandowsky, 20oO; Parasuraman et al., 1993). Some researchers have suggested that ~ ~ h g adaptive task Zllocaticc m q improve the degaded monitoring performance assockkc? With automation-induced complacency (Parasuraman, Modoua, & Molloy, 1996).

A number of factors contribute to automation-induced complacency including trust and system reliability (Lee & Moray, 1992; Muir & Moray, 1996). In addition, Singh, Molloy, and Parasurman (1993) have argued that operators may also exhibit relatively persistent attitudes that contribute to the style and effectiveness of their ability to monitor automation. Singh et al. refer to these attitudes as comphcencypotentiul.

attentional resources, an adaptive automation system that uses operator engagement may improve monitoring performance. By examining an operator’s physiological level of engagement, it may be possible to determine when that individual is experiencing a level of engagement that is either too high or low. Pope et al. (1995) argued that physiologically-based adaptive systems provide a means of maintaining optimal states of arousal; thus, improving performance for flight-related tasks. It is therefore conceivable that the use of a brain-based adaptive automation system can ameliorate the effects of automation-induced complacency by helping to maintain an operator’s most advantageous level of engagement. Thus, as noted earlier, individuals in the present study who used the adaptive system were expected to have better monitoring performance compared with those who used the adaptable system. An additional goal was to assess the benefits of adaptive automation for those individuals who differed in complacency potential. Overall, it was expected that individuals who had a higher potential for automation-induced complacency would focus more on the manual aspects of the task and less on automated components and therefore demonstrate poorer monitoring performance. Consequently, the adaptive system was expected to be of greater benefit to individuals with high complacency potential because changes between task modes would be optimized for the individual and further, decisions about when to implement mode changes would be assigned to the adaptive system.

Because automation-induced complacency is characterized by less than optimal

Workload and Situation Awareness

As noted earlier, a major concern for the operation of complex systems is the level of mental workload. Hart and Wickens (1990) defined workload as the cost of accomplishing task requirements for the human element of a man-machine system that may result in subjective discomfort and reductions in performance or physiological reactions. Because performance has been shown to suffer in a variety of contexts when workload is high (Hancock & Desmond, 2001), understanding factors that can mitigate excessive levels of workload may improve operator performance. One potential benefit of adaptive automation is to abate the effects of high workload by using dynamic task allocation. Thus, tasks can be modified or offloaded in real time depending on the current workload of the operator. Therefore, during periods of operation where workload is high, the operator’s responsibility for completing other less critical tasks is eliminated so that he or she may focus on more immediate and critical demands.

seems obvious, but the potential benefits for maintaining situation awareness (SA) are not as well understood. At present, there is very little empirical work examining the impact of adaptive automation on situation awareness ( S A hdsley, 2995). SA has k e n dsfiaed s ‘The perception of the elements h the environment with a vokwne of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Endsley, 1988, pg. 97). For operators in complex environments such as commercial aviation or nuclear power plant control, maintaining accurate knowledge of current system status as well as anticipating future states is critical. It has been argued that when operators are required to perform manual tasks while remaining vigilant in complex environments, monitoring performance and SA may become degraded (Paraswaman et al., 1993). Consequently, several investigators have suggested that SA might be improved through the use of adaptive automation (Kaber et al., 2001; Prinzel et al., 2000). These improvements result from providing a better match between operator resources and current task demands when systems can dynamically alter their mode and/or level of automation. Through dynamic task allocation, an operator’s optimal level of arousal is maintained yielding improved and more consistent levels of SA.

In addition to looking at the impact of adaptive automation on workload and SA, it is also important to examine these constructs in overall comparisons between adaptive and adaptable systems, (i.e., whether one system elicits higher or lower levels on one construct than another). According to Vidulich (2003), both workload and SA can be characterized as meta-measures. Meta-measures are defined as properties of human information processing that emerge while performing demanding and complex tasks. Because decreased workload and increased SA can be associated with improved resources for cognitive processing, using systems that reduce wcrkload and/or improve SA may result in better task performance (Vidulich, 203). The final goal of the present investigation was to assess workload and SA in an adaptive and adaptable context. Regarding workload, it was hypothesized that operators in the adaptive, brain-based system would report lower levels of workload resulting fiom the better match between available cognitive resources and the immediate demands of the task. With respect to SA, the findings of Paraswaman et al. (1996) suggest that adaptive automation may be used to bolster SA. Therefore, in the present investigation, it was expected that operators who used an adaptive system would demonstrate better SA compared to operators using an

The idea that an adaptive system would be beneficial for moderating workload

adaptable interface because of the improved attentional resources associated with systems that dynamidy alter their mode and/or level of automation.

EXPERIMENT 1

The first experiment was designed to examine the efficacy of adaptive automation using a brain-based system. Participants performed a suite of flight-related tasks includmg a compensatory tracking task that was shifted between automatic and manual modes by the adaptive system. Their performance was compared to a yoked group who received the same pattern of shifts between modes, but whose EEG had no effect on system operation. In addition, the role of complacency potential and the performance consequences of using the adaptive system on subjective workload and SA were assessed.

Method

Participants. The participants were 40 undergraduate students whose ages ranged from 18 tr, 32 j m r s (M = 21.1). "he sample iiiduiled 15 women a d 25 ffien with a mmpaabk distrhtion of women mcl men in each of the experimental conciitions. All participants had normal or corrected-to-normal visual acuity.

EEG recording and engagement indk. EEG activity was recorded using a montage of four sites consisting of CZ, PZ, P3 and P4. A ground site was located near FP1 with each of the sites referenced to the left mastoid. The amplified signals from each of the sites were digitized at a rate of 200 samples/s into a circular buffer array with samples taken from the buffers in four vectors (representing each of the sites). Each vector contained 512 data points resulting in 2.56 s of data for each channel. Each vector was then processed using a standard Hanning windowing procedure in order to remove the frequency characteristics that are inappropriately incorporated into the power spectrum resulting from the sudden onset and offset of epochs.

The power spectrum was derived using fast Fourier transformations. Bin powers were combined to compute total power in three bandwidths including theta, alpha, and beta. Bin powers represent estimates of the power spectrum within bins between Fourier frequencies of 0 to 256 Hz. The resulting values were divided by the total power to produce percentage power with the array of percentage power for each of the sites comprising the engagement index: 20 beta/(alpha + theta). This engagement index has been demonstrated to vary between 2 and 20 (with higher values representing higher levels of engagement) and is regarded as the most effective measure of engagement (Freeman et al., 1999; Pope et al., 1995). This index was initially computed over a 20 s interval and was subsequently updated every 2 s using a sliding 20 s window.

epoch from the four digitized EEG channels and compared them with pretrial tests with eyes open and eyes closed. From these data, a power spectral distribution was derived and if the voltage in any channel exceeded the threshold by more than 25%, the epoch was excluded when computing the engagement index and in subsequent analyses. Previous research has shown that as a result of this algorithm, less than 1% of any participants' data files are typically rejected.

In addition, an artifact rejection subroutine examined the artifacts from each

Apparatus. EEG activity was recorded from electrodes attached to the scalp via NuPrep Ten20 conductive EEG paste. The EEG signals were amplified using a Neuroscan SynAmps differential amplifier module consisting of four high-gain, differential input, biopotential filters, with the low and high pass filters set to 100 and 1 Hz, respectively. The amplifier was connected to a Labview Virtual Instrument (VI) and the supporting software computed the power in the theta (4 Hz to 8 Hz), alpha (8 Hz to 13 Hz), and beta (13 Hz to 22 Hz) bands. The VI also computed the engagement index calculations that were used to drive the task mode switches through a serial port connection to the computer that displayed the experimental tasks. The experimental task was displayed using a Pentium III personal computer with a Dell E550 15 in monitor. Participants used a standard computer keyboard along with a Microsoft Sidewinder joystick with the gain set to 60% of the maximum value.

Attribute Battery (MAR Comstock & Amegard, 1992). The MAT consists of a suite of tasks that are similar to activities performed by pilots in the cockpit including system monitoring, compensatory tracking, and resource management. The MAT was modified to include a numher of ksks.

System monitoring. The system monitoring task consisted of four vertical bars with moving pointers (see Figure 1). Under normal conditions, the pointers randomly fluctuated around the center of each gauge without moving more than two pointer marks above or below the center value. Occasionally, the pointers would deviate by more than 5 marks fbm the gauge’s central value, after which, a red warning light, placed directly above the four vertical gauges indicated that the system had detected a critical deviation and would take action to correct it.

Under most circumstances, critical deviations in the monitoring task were accompanied by the red warning light and no action was required from the participants. The system automatically corrected the deviation and returned to its normal state after four seconds. Occasionally, a critical deviation occurred that was unaccompanied by the red warning light. This was considered an automation failure. If the participants detected an automation failure, they were asked to press the corresponding function key (Le., F1, €2, F3, and F4) indicated under each vertical bar in the display. Following a correct detection, the pointer immediately moved back to the center of the gauge. If the participant failed to detect an automation failure within 10 s, it was scored as a miss and the pointer would revert back to its normal position. If at any time the participant pressed a function key when the red light was not illuminated and there was no corresponding critical deviation, it was scored as a false alarm. The reliability cf the automated system was manipulated so that participants experienced one of two different proportions of automation failures, either 10% or 35% for the high and low reliability conditions, respectively.

Compensatory tracking. A compensatory tracking task was presented in the upper center portion of the display (see Figure 1). Participants were asked to keep a green circle as close as possible to a set of cross hairs in the center of the display window. The movement of the circle within the display window was derived from a forcing function that consisted of a sum of non-harmonic sine waves. In the adaptive condition, the participants’ engagement index was used to switch the tracking task between

ExperimentaZ tasks. Participants operated a modified version of the Multi-Task

a d a e g &@a;.., sl;t~.cz=&&g &e +&= p r h h i

keys on the keyboard that corresponded to the numbers beside each of the pumps, (i.e., 1, 2,3,4,5,6,7, and 8). Once activated, a pump would turn from a transparent square outline to an opaque square and fuel would begin moving between the two tanks at a constant rate. Participants were told to keep the level in each of the top two tanks at 2500 gallons and that any strategy of turning pumps on and off could be used to maintain those levels. Performance was determined by the squared deviations above or below the 2500 gallon level for each of the main tanks. This calculation was made each time participants turned any pump on or off. The deviation scores from both tanks were then averaged yielding a composite value.

Situation awareness task The MAT was modified to include a number of digital and analog displays (e.g., vertical speed indicator, GPS heading, oil pressure, and auto pilot odoff) that surrounded the three standard MAT tasks (see Figure 1). The values on each of the displays fluctuated throughout the experiment according to a prespecified script, where no gauge remained in one position for a period exceeding five minutes. Participants were asked to monitor the gauges for later recall without sacrificing performance on the three primary MAT tasks.

were asked tc! report the cuxe~t values for a sample of five of the h e displays. A different sample of displays was queried after each trial. Participants’ reports for each display were then compared to the actual values to provide an objective measure of SA. The SA measure was calculated by establishing the proportion of correct responses divided by the total number of queried items for each of the three experimental trials.

Complacency potential rating scale. Prior to the start of the experiment, participants were asked to complete the Complacency-Potential Rating Scale (CPRS; Singh et al., 1993). The CPRS is a u)-item scale developed to measure attitudes regarding commonly encountered automated devices that reflect the potential for automation-induced complacency. The CPRS has been found to demonstrate both high internal consistency (r = -87) and test-retest reliability (r = 90). Participants were divided into high and low complacency potential groups according to their scores on the CPRS utilizing a median split procedure.

Experimental procedure. After completing the CPRS, participants were prepared for EEG recording. The participants’ scalps were first treated with rubbing alcohol and an electrolyte gel. Electrodes with conducting gel were then attached to each of the four recording sites as well as the ground and reference points. Next, participants were given instructions regarding the MAT and then completed a 2O-min practice session to become acquainted with the experimental task and to establish each participant’s baseliie level of engagement. The baseline value was calculated from the participant’s mean level of engagement over the 20-min practice period. Each participant then completed three, 15- min experimental trials. At the end of each trial, the participants reported the current values for a sample of five displays. Following the last experimental trial, they completed the NASA-Task Load Index (nx; Hart & Staveland, 1988) and were then debriefed.

Experimental design. A 2 Condition (adaptive and yoke) X 3 Trial X 2 System Reliability (high and low) X 2 Complacency Potential (high and low) mixed-subjects experimental design was used with feedback condition, system reliability, and complacency potential manipulations as nested variables.

At the. end of each hid, !he computer mozitor :yeat blank a d &e participants

The dependent measures included performance on the three subtasks of the MAT with RMS used for the tracking task, a squared deviation score for the resource management task, and the proportion of correctly identified system failures for the monitoring task. Further, the proportion of gauge positions correctly recalled was used as a measure of SA and the NASA-TLX was used to assess subjective workload.

Results

Grow differences. Four participants were eliminated from the analyses due to their inability to adequately perform the experimental tasks. For the remaining 36 participants, each of the dependent measures was analyzed with a mixed ANOVA procedure. The means for the adaptive and yoke control groups for all measures are shown in the top half of Table 1. The results of the analysis, however, showed no significant group differences with respect to compensatory tracking, resource management, system monitoring, or subjective workload.

Situation awareness. The analysis did yield significant findings for the SA measure. SIDecifically, a sigdicmt effect for ainplzency ptentid ws found, F(1,28) = 1157, p = -002. High conplacency ptzntiai individuals performed more poorly on the SAGAT, achieving only 545% accuracy. In contrast, individuals with low complacency potential achieved 73.3% accuracy on the SAGAT measure. A significant effect was also found for trials, F(2,56) = 16.07, p < .001. SAGAT performance improved horn 50% to 66% to 77% over the three trials. A Tukey HSD posttest demonstrated a significant difference between only the first and third experimental trials for the SAGAT data. No significant effect was found for feedback condition on SAGAT performance. However, there was a significant interaction between complacency

8 0 1

40' High LOW

Complacency Potential Figure 2. The interaction between feedback condition and complacency potential for mean percent correct on the SAGAT measure.

potential and feedback condition, F(l,28) = 6.80, p = .015 (see Figure 2). Specifically, high complacency potential, yoked participants performed more poorly on the SAGAT task than both low complacency potential, yoked and low complacency potential, adaptive participants. Consistent with our hypothesis, those individuals predisposed to complacent attitudes operating under a schedule of task mode switches uncoupled to their own EEG demonstrated the poorest performance on the SA task.

Discussion

The primary purpose of the first experiment was to determine whether operator performance with an adaptive, brain-based system would exceed that of individuals who had no control over system operation. Specifically, would operators whose EEG determined the schedule of task mode transitions demonstrate enhanced performance relative to those individuals whose EEG had no impact on the experimental tasks? In addition, the impact of a brain-based system on subjective workload and SA for those individuals who differed in complacency potential was examined.

the a&ptive an?, yoke cmtrol groups; however, there were a number of trends in the data that were in the expected directions (see Table 1). Specifically, operators in the adaptive condition had better monitoring performance, higher SA, and lower subjective workload than those in the yoke control condition. These trends were also accompanied by poorer performance by the adaptive group on the resource management task. Thus, the marginal differences between the two groups may reflect subtle differences in task priorities.

The resdts ic&cxted &zit pr€cjmmce ZTOSS ail measures was comparable for

Table 1 Means (and Standard Deviations) for Feedback Condition in Experiment 1 and Experiment 2 for Compensatory Tracking (CT), Resource Management (RM), System Monitoring (SM), Situation Awareness (SA), and Mental Workload (MW).

CT Experiment 1 Adaptive 29.765

(9.706)

Yoke 28.108

Experiment 2 (12.902)

Adaptable 33.584 (17.407)

Yoke 27.217 (5.883)

RM

159.790 (97.497)

146.264 (81.856)

182.799 (83.962)

127.676 (52.327)

SM

0.866 (0.223)

0.771 (0.282)

0.888 (0.165)

0.817 (0.197)

SA

68.148 (26.78)

60.741 (29.833)

64.444 (24.776)

71.111 (25.966)

Mw

58.583 (16.741)

63.306 (16.750)

73.472 (12.509)

60.500 (15.410)

Although there were no significant differences between groups, there was evidence that the adaptive system could improve performance on a secondary task, as reflected in the SA measure. More specifically, the results indicated that an individual’s attitudes toward automation could have an impact on their overall awareness of system states. Those individuals who placed more trust in the automated processes, (Le., high in complacency potential), focused less on the overall status of the system and demonstrated lower levels of SA. Further, the interaction between feedback condition and complacency potential showed that SA was similar for both high and low complacency individuals when their EEG was used to invoke switches among task modes. By contrast, complacency potential did have a signillcant effect on the SA of individuals under a pattern of task mode switches uncoupled to their own EEG. Under these conditions, high as compared to low complacency individuals had lower levels of SA. As such, the results from Experiment 1 provide evidence that the use of a brain-based system may have the potential to ameliorate the effects of complacency by bolstering available attentional capacity and in turn, improving SA.

The second experiment assessed the ability of operators to choose when automation was needed. Participants performed the same suite of tasks used in Experiment 1; however, they were able to invoke the automation whenever it was most appropriate for maintaining optimal performance on the compensatory tracking task. Their performance was then compared to a yoked group who experienced the same schedule of mode transitions but who were unable to effect changes themselves. Performance implications on subjective workload and SA as well as the role of complacency potential were also examined.

Method

Participants. The participants were 40 undergraduate students whose ages ranged from 18 to 35 years (M = 22). The sample included 17 women and 23 men with a comparable distribution of women and men in each condition. All participants had normal or corrected-to-normal visual acuity.

experiment excluding the EEG recording equipment.

Experiment 1. All participants operated the same modified MAT battery. However, in contrast to the adaptive condition used to automate the tracking task in Experiment 1, participants in the second experiment were permitted to choose when to invoke the automation. Thus, if a participant felt his or her performance falling below an optimal level, they could press the spacebar on the computer keyboard to automate the tracking task for 10 s, relieving them of that responsibility for a short time. After 10 s, the tracking task would revert back to manual mode and the participants once again were required to use the joystick to perform that task. The 10 s duration used for the automation was derived from the first experiment and represents the mean length of time

Apparatus. The equipment used in Experiment 2 was the same as the previous

Experimental tasks. The tasks in Experiment 2 were very similar to those used in

the tracking task was in automatic mode for all individuals across a l l trials. Participants were told that they could automate the task as much as they felt necessary in order to perform at their best.

Consistent with Experiment 1, those individuals in the control condition were yoked to one of the individuals in the adaptable condition. As such, they received the same schedule of switches between automatic and manual modes as their corresponding participant in the adaptable condition but were unable to initiate changes in the automation themselves.

Reliability (high and low) X 2 Complacency Potential (high and low) mixed experimental design was again used with feedback condition, system reliability, and complacency potential as nested variables. Similar to the first experiment, the dependent measures included performance on the three subtasks of the MAT. Situation awareness and subjective workload were also assessed. In addition, the number of mode transitions that the participants initiated was also examined.

same as Experiment 1 with the exception that ES EEG rmrdhgs t s k hstmctims were augmented to include m exphation of how and when participants should choose to automate the tracking task.

EipmimmtaZ design. A 2 Condition (adaptable and yoke) X 3 Trial X 2 System

Experimentalprocedure. The experimental procedure for Experiment 2 was the made. Further,

Results

Group difierences. Similar to Experiment 1, four participants were eliminated from the analyses due to their inability to adequately perform the experimental tasks leaving 36 total participants. The dependent measures were analyzed with a mixed ANOVA procedure. The means for the adaptable and yoke control groups for all measures are shown in the bottom half of Table 1. The results of the analysis yielded significant effects for the system monitoring, resource management, and subjective workload measures.

System mnitoring. A significant effect for system reliability was found, F(1,28) = 7.01, p = .013. Participants in the high reliability condition were able to correctly identify only 79.6% of the total system failures. In contrast, those participants in the low reliability condition were able to correctly identify 90.9% of the system failures across trials.

Resource management. A significant main effect of condition was found for the resource management task, F(1,28) = 11.82 ,~ = .002. The mean combined deviations for both tanks in the resource management task were M= 182.80, SD = 83.96, and M = 127.68, SD = 52.33, for the adaptable and yoked participants, respectively. Participants who had to press a button to switch modes had greater error on the resource management task than those participants whose pattern of mode switches was not under their control.

was also found for the resource management task, F(1,28) = 6 . 8 1 , ~ = .014 (see Figure 3). Using a Tukey HSD posttest, the participants in the low system reliability, high complacency potential group demonstrated significantly higher average deviations on the resource management task than those in the high system reliability, high complacency potential or low system reliability, low complacency potential groups. There were no

A significant interaction between system reliability and complacency potential

differences between the high system reliability, low complacency potential participants relative to any of the other groups. This finding indicates that those individuals with high complacency potential who worked with the low reliability system monitoring task performed more poorly on the adjacent resource management task.

I -t High Reliability I +Low Reliability

Figure 3. The interaction between system reliability and complacency potential for deviations on the resource management task.

Subjective workload A significant effect for the adaptable versus yoked control condition was found with respect to levels of subjective workload, F( 1 , B ) = 6.53, p = .016. Those participants who could invoke the automation themselves reported significantly higher levels of workload (M = 73.47, SD = 12.51) than those individuals who had no control over the pattern of mode switches (M = 60.5, SD = 15.41).

Regression analyses: Mode switches. For the mode switches analysis only data from those individuals who had control over the automation were used. The number of switches between automatic and adaptable modes was used to predict performance on the other dependent measures, (i.e., compensatory tracking, system monitoring, resource management, subjective workload, and SA). Two significant relationships were observed. With respect to tracking error, there was a significant positive relationship between the total number of switches and the average amount of tracking error across trials, F (132) = 117.46,~ e .001, R2 = .693. Those individuals who invoked the automation more frequently had poorer performance on the compensatory tracking task. Further, with respect to subjective workload, another positive relationship was found, F (1,52) = 21.05, p < .001,$ = 288. Those individuals who invoked the automation more frequently also reported higher levels of mental workload.

Discussion

The primary purpose of Experiment 2 was to determine whether operator performance with an adaptable system would exceed that of individuals who had no control over system operation. Specifically, could operators who had control over automation invocation use it in a way that would improve their performance? In addition, the effect of an adaptable system on subjective workload and SA for participants with high and low complacency potential was also examined. The results indicated sigdicant effects for system monitoring and resource management as well as subjective workload and the overall number of mode transitions.

Performance on the system monitoring task was found to be better for individuals operating under low reliability. This finding is consistent with the notion that low reliability systems require more monitoring because they are more likely to fail. Previous research has shown that the reliability of an automated system impacts an operator’s ability to effectively monitor that system (Lee & Moray, 1992; Muir & Moray, 1996; Parasuraman et al., 1993). Lee and Moray demonstrated that both trust and strategies for u3mg auruIIiauuu varied according to the overall reliability of automation. Specifically, highly reliable systems engender trust, which in turn impacts an operator’s reliance on automation. The present results, taken together with previous research, suggest that operators are less likely to monitor the performance of a highly reliable system.

The results also indicated that those individuals who had control over automation in the tracking task had poorer perfomance on the resource management task. This finding suggests that providing operators with the ability to invoke automation represents an added task responsibility. In addition to the other task requirements, operators were also required to monitor their own performance to determine if and when automation was necessary. Because of this additional task responsibility, a resource tradeoff occurred resulting in degraded performance on the resource management portion of the MAT. In addition, individuals in the low reliability, high complacency potential group also performed more poorly on the resource management task. A closer examination of the data, however, suggests that this interaction may be due to the greater number of mode transitions initiated by the participants under low reliability. Although the main effect for system reliability and mode switches did not reach statistical significance, the data show that those individuals operating under low system reliability produced nearly twice as many mode switches as those in the high reliability condition. It is conceivable that the attentional resources required to monitor their own performance and the more frequent switching between modes of automation may have depleted the resources necessary to perform the resource management task.

Related to the additional cognitive burden in the adaptable condition, participants in Experiment 2 also reported significantly higher levels of mental workload. Operators who had the option to invoke automation reported workload levels that were approximately 17% higher than operators in the yoke control condition. This finding provides additional support for the performance consequences noted earlier. Those participants in the adaptable condition, who had to monitor their own performance and determine when automation was necessary, were in essence performing another task. This added responsibility required the use of additional attentional resources and led to degraded performance on the MAT as well as increased mental workload.

..-:-- ..-.4- ..d--

Finally, those individuals who tended to use the automation more frequently had poorer tracking performance than those individuals who chose to remain mostly in manual mode. This finding may also be related to the additional burden felt by operators in the adaptable condition who had to choose when automation was necessary. The added responsibility of invoking automation distracted operators from the compensatory tracking task, one of their primary responsibilities. Therefore, despite the ostensible benefits an adaptable system gives operators, (Le., the ability to choose when they need additional help), the results from this experiment suggests that they are unable to effectively manage the extra task, which results in degraded performance and increased mental workload. This finding is consistent with the resource theory of attention described by Kahneman (1973). Specifically, Kahneman argues that the attentional resources of any operator are finite, (Le., that only a limited number of tasks at a certain level of difficulty can be completed by an operator before he or she starts to experience increased mental workload and/or degraded performance). In Experiment 2, those operators who invoked automation more frequently may have used up attentional resources needed to adequately complete the remaining tasks resulting in degraded perfommce md haeased mentd wrkload.

COMPARISON BETWEEN EXPERIMENTS

A comparison of the results from Experiments 1 and 2 was performed to address whether the benefits of an adaptive, brain-based system exceed those of a system that employed adaptable, user-initiated control. Participants from the adaptive condition in Experiment 1, whose pattern of automation switches was derived from their EEG, were compared to participants in the adaptable condition from Experiment 2, who choose when to manually switch modes of automation. The yoked control subjects from both experiments as well as the complacency potential manipulation were omitted fiom this analysis.

Results

Group differences. A 2 Condition (adaptive and adaptable) X 3 Trial X 2 System Reliability (high and low) mixed ANOVA procedure was performed using each of the dependent measures. The means for the adaptive and adaptable can be seen in Table 1. The results of the analysis yielded only one significant effect. Specifically, a main effect was found for subjective workload, F (1,32) = 8 . 6 8 , ~ = .006. On average, participants’ reports of subjective workload were 20.3% lower for the adaptive condition when compared with those participants in the adaptable condition. Therefore, those participants in the adaptive condition whose pattern of mode transitions was derived fiom their own EEG, reported significantly lower levels of mental workload than those participants who were required to manually invoke automation mode changes.

Discussion

The primary purpose of the comparison between the first and second experiments was to determine if performance with an adaptive system would exceed that of an

adaptable system. Specifically, would operators using a brain-based system controlled by their own EEG, demonstrate enhanced performance relative to operators using a system that required adaptable control? In addition, would the systems have different effects with respect to operator workload or SA and what would be the role of complacency potential?

With respect to performance implications, there was limited evidence for performance enhancement using the adaptive, brain-based system; however, some trends in the data can be seen in Table 1. Specifically, the adaptive system produced lower levels of error on the tracking and resource management tasks and higher SA perfOlTWllce.

As noted earlier, systems with higher reliability can engender trust, which precludes effective monitoring (Lee & Moray, 1992; Muir & Moray, 1996). Although the reliability factor did.not reach statistical significance in the overall cornparison, it did achieve significance in Experiment 2, and also approached significance in Experiment 1. Consistent with the findings of Lee and Moray and Muir and Moray, the reliability of an automated system does appear to impact an operator’s ability to monitor effectively. Additionally, opratm h the sdaptable condition had approximately i6% more error on the resource management iasiC. However, the high degree of variabiiity in the number and schedule of mode transitions between operators may have obscured the significance of this effect. Operators in the adaptive condition also demonstrated slight improvements in compensatory tracking performance and in their level of SA.

significant increase in workload for those operators using the adaptable system. This is an important finding. As noted in Experiment 2, the resources required to determine if and when automation is needed, created an additional burden for operators in the adaptable condition. In the overall comparison, those individuals who operated with the brain-based system reported, on average, 20% less mental workload than those using the adaptable system. Despite the absence of significant differences in performance, this finding is critical. Specifically, this result indicates that the adaptable system requires significantly more effort to maintain a level of performance approaching that found with the adaptive system. Moreover, these differences in workload suggest that differences in performance might eventually be observed if individuals were required to operate for longer periods than were used in the present study.

Using workload in an overall comparison between these two systems is consistent with the view that workload is a meta-measure (Vidulich, 2003). All things being equal, it is advantageous to use systems that elicit lower levels of mental workload. A system that elicits less workload than another may facilitate performance when task or situational demands are increased. Therefore, it appears that using adaptive, brain-based technology may provide a distinct advantage to system operators.

The overall comparison between Experiments 1 and 2 did reveal a statistically

OVERALL DISCUSSION

The primary purpose of the present investigation was to make a direct comparison between adaptive and adaptable technology, (i.e., brain-based and user-initiated paradigms) and to determine whether either system could elicit improved performance on a secondary task. It was anticipated that the brain-based, adaptive system would produce

better performance because it provides a tighter coupling between an operator’s available cognitive resources and the immediate demands of the task. Although the adaptive system produced better performance on the tracking and resource management tasks and higher SA than the adaptable system, these differences were marginal and did not reach statistical sigdicauce.

At first glance, the lack of a significant performance advantage for the adaptive system may seem at odds with the findings of Freeman and his colleagues (Freeman et al., 1999; Freeman et al., 2OOO) who reported performance benefits with a brain-based, adaptive system. However, it is important to understand that the benefits observed by Freeman and his colleagues were obtained by comparing a negative feedback contingency between the engagement index and the automatidmanual task mode changes, designed to stabilize engagement (as was used in the present study), with a positive feedback contingency designed to drive engagement levels to the extremes. On the other hand, Prinzel et al. (2OOO) did compare tracking performance with a brain-based, negative feedback adaptive system to manual tracking without the aid of automation. They found that the adaptive system improved tracking performance by about 15% over fully manual tr3chg. In the preseiit study, dttiiigh #e difference in iraeing scores between the adaptive and adaptable conditions was not statistically significant, the performance advantage for the adaptive group was on the order of 12% and is therefore similar to what was observed by Prinzel et al.

adaptive and adaptable systems. In Experiment 1, those individuals with high complacency potential, operating under a schedule of task mode changes uncoupled to their EEG, had poorer SA than those individuals who worked with the brain-based system or had low complacency potential. This result supports the notion that a brain- based system may help to reduce the impact of maladaptive attitudes toward automated devices by helping operators maintain an appropriate level of engagement.

The final purpose of the present study was to examine whether adaptive or adaptable automation would lower workload and/or improve SA. It was anticipated that the brain-based system would elicit lower levels of workload and improve SA. As previously mentioned, the results from Experiment 1 do provide evidence that a brain- based system can improve SA, especially for individuals who differ in complacency potential. However, the most compelling evidence in the present study is the significant increase in mental workload associated with the user-initiated, adaptable system. Although there is limited evidence of a performance benefit between the two systems, the brain-based, adaptive system did elicit a substantially reduced level of mental workload. Taken together, the results from Experiment 2 and the overall comparison indicate that operators may be overburdened with an adaptable, user-initiated system. The added responsibility of determining if and when they need extra help, and manually invoking the automation, may divert critical attentional resources away fiom the task itself and result in degraded performance and increased workload. The lower workload may represent one of the primary benefits of adaptive systems. Because adaptive systems require no overt inputs from operators to initiate system changes, attentional and cognitive resources are maximized. Adaptable systems by contrast, require operators to make manual inputs, which in turn deplete critical attentional resources necessary for maintaining performance. The increase in available cognitive resources associated with a

The present study also assessed the effects of complacency potential for both

brain-based, adaptive system, therefore, constitutes a significant benefit, especially during periods of operation where workload is already high. One of the most fundamental reasons for using automation is to reduce the likelihood of human error due to excessive workload (Parasuraman & Riley, 1997). Presently, it appears that a user- initiated, adaptable system may actually increase workload relative to a brain-based adaptive system.

automation in Experiment 2 had an overwhelming preference for retaining manual control. As noted earlier, previous research has demonstrated that individuals tend to stick with manual control even when the option to invoke automation exists (Lee & Moray, 1992; Riley, 1994a). Although there was a tremendous amount of variability in the strategies operators used (some operators remained in automated mode for up to 40% of the entire experiment) most operators remained under automated control only about 7% of the time. Thus, despite the higher workload associated with the user-initiated adaptable system, operators were still hesitant to invoke automation.

This reluctance to use automation raises a potential limitation of the present investigation. It is net clear whether the diiatioli of the automatic mode used in EFrin?eot 2 was approphte. S p ~ d i d l y , ihe 10-s duration used in Experiment 2 was derived fiom the mean overall duration of the automation mode recorded from the brain- based system in Experiment 1. It is possible that alternative methods for determining the nature and duration of mode durations in adaptable systems are more appropriate. Kaber et al. (2001) suggest that adaptive systems must support manual performance, performance under automation, and the transitions between the two states. Results fiom the present study suggest that the transitions between modes may have been difficult for operators to manage, especially in the adaptable condition. Potentially, a longer duration of automation or the ability of operators to determine the duration they personally consider most appropriate may be more appropriate. This issue is an important area for further research on user-initiated forms of adaptive systems.

variability for the duration and number of mode transitions in the brain-based system and the strategies and number of automation invocations for operators in the user-initiated system. Given the large degree of variability for the time operaiors had to manually perform the tracking task, they may have had qualitatively different experiences. For example, it may be difficult to compare the performance of individuals in Experiment 2 who used automation for nearly 1/3 of the experiment with those who rarely invoked it. Harris, Hancock, and Arthur (1993), examining task load and automation usage, found few effects and argued that any potential patterns were masked by operator variability in automation usage strategies. Consistent with their findings, any significant performance differences in the present study may be obscured by the tremendous amount of variability between individuals with respect to the schedule of mode transitions that each experienced.

It is also interesting to note that those individuals who had the ability to initiate

Another potential concern with the present investigation is the high degree of

CONCLUSIONS

The primary purpose of this study was to assess whether operators using an adaptive, brain-based system would perform differently than those using an adaptable

system requiring user-initiated control. The results showed that a brain-based, adaptive automation system may improve SA, especially for those individuals who differ in complacency potential. Although it has been suggested that adaptive automation can be used to improve SA (Kaber et al., ml), the present study provides some of the first empirical data to support that contention. In addition, brain-based, adaptive systems may elicit lower levels of workload. It is critical to minimize the baseline level of workload that operators experience when using any system. The present results suggest that under similar task conditions, a brain-based system may generate significantly less workload than one that requires adaptable control. As a result, the increased cognitive resources available to operators may potentially enhance performance, especially during conditions where workload is already high.

Given the criticisms associated with conventional automation, adaptive systems may be the next evolutionary step (Scerbo, 1996). Automation that allows for dynamic function allocation between human and machine may reduce workload, improve monitoring efficiency, and improve situation awareness. The present study represents one of the first empirical comparisons between different forms of adaptive automation and yielded some encouraging results. Adaptive automation has the potential to significantly transform how operators interact with mmplex systems. Hcwever, fwther research addressing different types of adaptive strategies is critical for determining the viability of adaptive automation in real-world settings.

ACKNOWLEDGEMENTS

This research was supported by the NASA Langley Research Center, Grant No. NAG- 1-02062.

REFERENCES

Byme, E. A., & Parasuraman, R. (1996). Psychophysiology and adaptive automation.

Comstock, J.R. Jr., & Amegard, R.J. (1992). lrhe multi-attribute task battery for human Biological Psychology, 42,249-268.

operator workload and strategic behavior research vechnical Memorandum NO. 104174). Hampton, VA. NASA Langley Research Center.

Proceedings of the Human Factors Society 32"'Annuul Meeting @p. 97-101). Santa Monica, CA: Human Factors Society.

Human Factors, 37,32-64.

Mouloua (Eds.), Automation and Human Performance: Theory and Applications (pp. 163-182). Mahwah, NJ: Lawrence Erlbaum Associates.

adaptive recovery under automation. Journal of Experimental Psychology: Learning, Memory, & Cognition, 26,395410.

Endsley, M.R. (1988). Design and evaluation for situation awareness enhancement. In

Endsley, M.R. (1995). Toward a theory of situation awareness in dynamic systems.

Endsley, M.R. (1996). Automation and situation awareness. In R. Parasuraman & M.

Farrell, S., & Lewandowsky, S. (2OOO). A connectionist model of complacency and

Freeman, F., Mikulka, P. P h l , L, & Scerbo, M. (1999). Evaluation of an adaptive automation system using three indices With a visual tracking task. Biological psycho lo^, 50,61-76.

Freeman, F., Mikulka, P., S c e h , M., Prinzel, L, & Clouatre, K. (2OOO). Evaluation of a psychophysiologically controlled adaptive automation system, using performance on a tracking task. Applied Psychophysiology and Biofeedback, 25,103-115.

Ergonomics, 9,991-1005.

In P A Hanmk (Ed.), Human Fuctors Psycbbgy (pp. 305-345). North Holland: Elsevier.

interface design. LEEE Transactions on Systems, Man, and Cybernetics, 18,647- 658.

Hanmk, P A , Chignell, M.H., & Loewenthal, A. (1985). An adaptive human-machine system. In Proceedings of the 1985 IEEE International Conference on Systems, Man and Cybernetics (pp. 627-630), Tucson: Thtitute r?f HectrkA and Electronics Engineers.

Lawrence Erlbaum Associates.

automation use, performance, and workload. In Proceedings of the International Sympsium on Aviation PsycbrogY (pp. 89OA-89OF). Columbus: Ohio State University.

Hart, S. G. & Staveland, L.E. (1988). Development of NASA-TLX ('Task Load Index): Results of empirical and theoretical research. In P.A. Hancock, & N. Meshkati (Eds.), Human mental worklard (pp. 239-250). Amsterdam: North Holland Press.

(Ed.), W P M m : An Approach to Systems Integration (pp. 257-296). New York: Van Nostrand Reinhold.

Hilburn, B., Jorna, P.G., Byme, E A , & Parasuraman, R. (1997). The effect of adaptive air traffic control (ATC) decision aiding on controller mental workload. In M. Mouloua & J. Koonce (Eds.), Human-automation Interaction (pp.84-91). Mahwah, NJ: Lawrence Erlbaum Associates.

based on secondary task workload measurement. International Journal of Cognitive Ergonomics, 3,169-187.

automation for complex systems. International Journal of Cognitive Ergonomics,

Gaillard, A.W.K. (1993). Comparing the concepts of mental load and stress.

Hanmk, P A , & Chignell, M.H. (1987). Adaptive control in human-machine systems.

Hanmk, PA., & Chignell, M.H. (1988). Mental workload dynamics in adaptive

Hancock, P.A., & Desmond, P.A. (2001). Stress, Workload, and Fatigue. Mahwah, NJ:

Harris, W., Hancock, P.A., & Arthur, E. (1993). The effect of task load projection on

Hart, S.G., & Wickens, C. (1990). Workload assessment and prediction. In H.R. Booher

Kaber, D. B., & Riley, J. M. (1999). Adaptive automation of a dynamic control task

Kaber, D.B., Riley, J.M., Tan, K., & Endsley, M.R. (2001). On the design of adaptive

5,37-57. Kahneman, D. (1973). Attention and Efiort. Englewood Cliffs, NJ: Prentice-Hall. Lee, J., & Moray, N. (1992). Trust, control strategies and allocation of function in

Lee, J. D., & Moray, N. (1994). Trust, self-confidence, and operators' adaptation to human-machine systems. Ergonomics, 35,1243-1270.

automation. Internutiom1 Journal of Human Computer Studies, 40,153-184.

Morrison, J.G., & Gluckman, J.P. (1994). Definitions and prospective guidelines for the zpplication of adaptive automation. In M. Mouloua & R. Parasuraman (Eds.), Human Performunce in Automated Systems: Current Research and Trends (pp. 256-263). Mahwah, NJ: Lawrence Erlbaum Associates.

Muir, B. M. (1987). Trust between humans and machines, and the design of decision aids. International Journal of Man Machine Studies, 27,527-539.

Muir, B. M. (1994). Trust in automation: I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37, 1905-1922.

Muir, B. M., & Moray, N. (1996). Trust in automation: II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39,429-460.

Parasuraman, R., Bahri, T., Deaton, J.E., Momson, J.G., & Barnes, M. (1992). Theory and design of adaptive automation in aviation systems (Progress Report No. NAWCADWAR-92066-60). Warminster, PA: Naval Air Warfare Center, Aircraft Division.

Parasuraman, R., & Byme, E. (2003). Automation and human performance in aviation. In P.S. Tsang & M.A. Vidulich (Eds.), Principles ofAviation Psychology (pp. 311-357). Mahwah, NJ: Lawrence Erlbaum Associates.

automation-induced "complacency." International Journal of Aviation Parasuraman, R., Molloy, R., & She, I. L. (1993). Perfommce consqnences of

Psychology, 3,l-23. Parasuraman, R., Mouloua, M., & Molloy, R. (1996). Effects of adaptive task allocation

on monitoring of automated systems. Human Factors, 38,665-679. Parasuraman, R., Mouloua, M., Molloy, R., & Hilburn, B. (1996). Monitoring of

automated systems. In R. Parasuraman & M. Mouloua (Eds.), Automation and Human Performance: Theory and Applications (pp. 91-1 16). Mahwah, NJ: Lawrence Erlbaum Associates.

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39,230-253.

Pope, A.T., Bogart, E.H., & Bartolome, D.S. (1995). Biocybernetic system validates index of operator engagement in automated task. Biological PsychoZogy, 40, 187- 195.

closed-loop system for examining psychophysiological measures for adaptive task allocation. International Journal of Aviation Psychology, 10,393-410.

Riley, V. (1994a). Human use of automation. Unpublished doctoral dissertation, University of Minnesota.

Rouse, W.B. (1976). Adaptive allocation of decision making responsibility between supervisor and computer. In T.B. Sheridan & G. Johannsen (Eds.), Monitoring Behavior and Supervisory Control @p. 295-306). New York Plenum.

interfaces: Outline of an approach to supporting operatciis of complex systems. IEEE Transactions on Systems, Man, and Cybernetics, 22,1296-1308.

Scerbo, M.W. (1996). Theoretical perspectives on adaptive automation. In R. Parasuraman & M. Mouloua (Eds.), Automation and Human Performunce: Theory and Applications (pp. 37-63). Mahwah, NJ: Lawrence Erlbaum Associates.

Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., & Pope, A.T. (2OOO). A

Rouse, W.B., Geddes, N.D., & Curry, R.E. (1986). An architecture for intelligent

Scerbo, M.W. (2001). Adaptive automation. In W. Karwowski (Ed.), Zntermztional Encyclopeduz of Ergonomics and Human Factors (pp. 1077-1079). London: Taylor and Francis, Inc.

Scerbo, M.W., Freeman, F.F., & Mikulka, PJ. (2000). A biocybernetic system for adaptive automation. In R.W. Backs & W. Boucsein (Eds.), Engineering Psychophyswlogy: Issues and Applications (pp.24-254). Mahwah, NJ: Lawrence Erlbaum Associates.

Singh, I. L, Moiloy, R., & Paraswaman, R. (1993). Automation-induced "complacency": Development of the complacency-potential rating scale. International Journal of Aviation Psychology, 3,111-122.

control monitoring task with and without computer aiding. Aviation, Space, and Environmental Medicine, 40,744-748.

Vidulich, M.A. (2003). Mental workload and situation awareness: Essential concepts for aviation psychology practice. In P.S. Tsang & M.A. Vidulich (Eds.), Principles ofAviation Psychology @p. 115-146). Mahwah, NJ: Lawrence Erlbaum Associates.

Wiener, E L (1988). Cockpit automation. In E.L. Wiener & D.C. Nagel (Eds.), Human Factors in Aviation (pp. 433-461). San Diego, CA: Academic Press.

Wiener, E. L, & Curry, R. E. (1980). Flight-deck automation: Promises and problems. Ergonomics, 23, 995-1011.

Thackray, R. I., & Touchstone, R. M. (1989). Detection efficieccy on an air traffic