Mixed-Initiative Interaction and Robotic Systems Julie A. Adams and Pramila Rani Electrical Engineering and Computer Science

Vanderbilt University Nashville, USA.

[julie.a.adams, pramila.rani@vanderbilt.edu]

Nilanjan Sarkar Mechanical Engineering

Vanderbilt University Nashville, USA.

Nilanjan.sarkar@vanderbilt.edu

Abstract A truly collaborative human-robot interaction framework should allow the participating agents to assume and relinquish initiative depending upon their own capabilities and their understanding of the environment. The goal of this work is to define and develop a mixed-initiative humanrobot collaborative architecture in which affect-based sensing plays a critical role in initiative switching. Affect-based sensing implies that the robot detects the humans emotional state in order to determine which actions to pursue. We have conducted an extensive literature review of Mixed-Initiative Interaction that has provided a basis for our architectural development. In particular, we are applying Rileys (Riley 1989) general model of mixed-initiative interaction to our architecture development. We have developed a preliminary architecture and are now collecting affect-based participant data that will be used to test the system. The purpose of this paper is to provide an overview of Rileys model, its application in our development, present our preliminary architecture, and present the affect-based interaction constraints that affect the architecture development.

Introduction

In the foreseeable future, humans will remain a critical component in robotic systems. The robotics field has not developed fully autonomous capabilities for real-world situations. The recent DARPA Grand Challenge is an excellent example. The Grand Challenge mapped a 149 mile route between California and Nevada that robots were to autonomously navigate with no human intervention. Teams were provided with complete route information just a few hours before the race. The result was that none of the robots made it through the entire route. Carnegie Mellon Universitys Red Team made the longest autonomous trek (7.4 miles) before it ran off course, became stuck, and caught on fire. The participating teams should be applauded for their efforts and the advances that they made, but this race illustrates the need for humans-in-the-loop. In the Red Teams situation, a human could have intervened to inform the robot that it was potentially going to run off the course. If the robot had run off course, then the human could have intervened in an attempt to help the robot understand what went wrong and how to remedy the situation. Such relationships are not strict teleoperation; rather collaboration is needed between the human and the robot. If the human is to be elevated to a true supervisory position, an environment facilitating such collaborate must

be created. Mixed-initiative interaction (MII) is one mechanism for creating this relationship.

Our focus is based upon providing MII for affect-based human-robotic interaction. We have developed an MII architecture that will be incorporated into an affect-based robotic system. This architecture has been developed based upon an extensive literature review of the MII research.

The following section provides a historical MII literature overview. This is followed by a brief description of the affect-based system and its applications. We then describe Rileys (1989) general MII model, our MII architecture and its development based upon Rileys model. This is followed by a discussion on Initiative and Affect-based Sensing. A general discussion is then presented, which is followed by our conclusions and future work.

A Review of Mixed-Initiative Interaction

One of the first references to the term Mixed-Initiative (MI) was by Carbonell (Carbonell 1971). Carbonell associated the term with Computer Assisted Instruction (CAI) in his SCHOLAR expert system. This system was designed to maintain a dialogue with students during instruction. Carbonells objective was to create a CAI system that would permit students to ask the system questions thus creating a two-way interaction. He also felt that in order to attain truly mixed initiative, the system was required to contain real knowledge so that it was capable of understanding and answering the students questions.

An important question is: What is Initiative? Donaldson (Donaldson & Cohen 1998) defined four theories regarding initiative and the associated applications scope within a planning domain. Their objective was: to explore productive synthesis of the complementary strengths of both humans and machines to build effective plans more quickly and with greater reliability. The primary message from this work is that initiative is associated with control, goals, and conversational turns while providing insight into interruptions, plan failures, differences in beliefs, lack of interactivity, and degree of commitment to problem solving.

Allen (Hearst 1999) felt that mixed initiative might not necessarily involve a human. He defines mixed initiative as a flexible interaction strategy where each agent can contribute to that task that it can do best. The idea is to allow the agent who knows best to proceed and coordinate the activities of the other team members.

Horwitz (Horwitz 1999) defines MII in a human-computer interaction context. He uses the term to refer too: the methods that explicitly support an efficient, natural interleaving of contribution by users and automated services aimed at converging on solutions to problems. The idea was to develop a shared understanding of the goals and provide the capability for both entities to solve the problem in the most appropriate manner.

Ramakrishnan et al. (Ramakrishnan et al. 2002) claim that true MII cannot occur unless users are able to take out-of-turn interactions. They have developed a speech-based system that provides such interactions.

We surveyed over thirty articles. As can been observed from the discussion thus far, there are many interpretations of MII. In addition to the varied interpretations, this concept has been applied to many domains including:

Planning (Burnstein et al. 2003, Hearst 1999, Mitchell 1997),

Agents (DAloisi et al. 1997, Lester et al. 1999, Rich & Sidner 1998, Skinner Unknown),

Human-machine interaction (Bell et al. 2000, Fleming & Cohen 1999, Goldman et al. 1997, Penner et al. 1997), and

Understanding dialogue and discourse (Chu-Carroll & Brown 1997, Donaldson & Cohen 1997, Guinn 1998, Ishizaki et al. 1999, Lemon et al. 2001).

Kortenkamp et al. (Kortenkamp et al. 1997) introduce MII for human-robot teams. The MII occurs at the planning level of the 3T architecture. They list four advantages to adding MII: Increased flexibility of overall solutions; more robust system behavior; more tractable planning; and improved user involvement in planning. Sherwood et al. (Sherwood et al. 2001) also incorporates MII at the mission planning stage for a Mars Rover.

Anzai (Anzai 1994) raises the issue of incorporating MII into human-robot interaction. Anzai states call the mixture of robot-to-human communication with the one (communication) from humans to robots as mixed initiative interaction, Anzais platform is that prior HRI work has focused on the human specifically communicating to the robot but not robots communicating to humans. MII was intended to provide the capability for the robot to request information from the human.

Horiguchi and Sawaragi (Horiguchi & Sawaragi 2001) provide MII for a mobile robot via force-feedback through the joystick. The joystick is the primary interaction capability to control the robot. They provide a graphical display of additional information but this display does not permit the human to command the robot. The strength of the joystick inputs is employed to determine the humans initiative level when commanding the robot. This group (Zheng et al. 2004) recently started to extend their work to the Urban Search and Rescue domain.

Bruemmer et al. (Bruemmer et al. 2002, 2003) incorporate MII into a fully teleoperated system for hazardous environments. In this case, the MII permits the

robot to use initiative to ensure it does not harm itself or the environment. Their hypothesis is that the MII should permit a reduction in instrumentation, number of human operators, human exposure to hazardous materials, and the overall mission time.

Goodrich et al. (Goodrich et al. 2001) feel that the primary element in MII is the on-going dialogue between human and robot in which both parties share responsibility for mission safety and success. Their work focuses on providing MII that includes multiple levels of autonomy.

Finally, Murphy et al. (Murphy et al.2000) developed a MII system for urban search and rescue. Their system employs an intelligent agent between the human operator and the robots. The purpose of this intelligent agent is to inform the human of the presence of potential victims while providing the operator with perceptual assistance.

Affect-Based Human-Robot Interaction

Over the years, the traditional impression of emotions as being maladaptive has changed to one of being functional. The latest scientific findings indicate that emotions play an essential role in rational decision-making, perception, learning, and a variety of other cognitive tasks (Picard 1997). Hence it appears that endowing robots with a degree of emotional intelligence would open the doors to more meaningful and natural interaction between humans and robots. The current trend in human-robot interaction requires explicit communication from the human. A human communicates with a robot by either typing-in or speaking explicit instructions. Additional modes such as eye gaze, head motion, gestures, and electromyogram signals are also being used to command and control robots. Such communication modes are generally effective in many applications; however, a human-robot interaction (HRI) that relies solely on explicit communication ignores the potential gain of implicit communication via affect-based HRI. Affect-based HRI requires the detection, interpretation, and response to the humans emotional (affective) states by the robot. This can also be referred to as affective communication since the humans affective or emotional states are identified and recognized. Given the crucial role theoretically ascribed to emotions in human social communication (Scherer 1984, Smith & Lazarus 1990), the gains of endowing HRI with affective communication could be substantial.

There are numerous potential applications of emotion-sensitive robots. A robot that is capable of sensing emotional states such as panic, fatigue, stress, and inattention could immediately take meaningful actions to assist a human during search and rescue operations, fire fighting, space/remote planet/underwater exploration, and warfare. Rehabilitation robotics is yet another application of such versatile robots. Robotic devices for rehabilitation could employ the emotion-sensing capability to provide exercise sequences that are comfortable but still

challenging for the patient. A prototype Nursebot (Roy et al. 2000) was developed to assist the elderly by reminding them when to take medicine, monitoring for falls, and providing a video stream to a remote observer. Emotional sensing could augment the utility of such healthcare robots. Other applications in this domain might include industrial robots that can sense worker fatigue on the shop floor and take necessary precautions to avoid accidents. The robotic toy industry could also benefit from such research where a robot that could understand and respond to a childs emotions thus becoming an extraordinarily engaging toy. These potential applications could eventually lead to personal robots that act as understanding human companions. Physiological Responses for Emotion Recognition

There exist good evidence that the physiological activity associated with affective state can be differentiated and systematically organized (Bradley 2000). Therefore, our work focuses on using physiological signals to detect emotions. Emotions are closely associated with the autonomic nervous system (ANS). ANS regulates the body's internal environment. It is composed of the sympathetic branch, which generally functions in emergency situations, and the parasympathetic branch, which dominates during periods of relaxation. The transition from one emotional state to another, for example, from relaxed to anxious is accompanied by dynamic shifts in ANS activity indicators. The physiological signals we examined include various parameters of cardiovascular activity (Frijda 1986, Cinotti et al. 2001), including: interbeat interval, relative pulse volume, pulse transit time, electrodermal activity (tonic and phasic response from skin conductance (hman et al. 2000)) and electromyogram (EMG) activity (facial activity including activity of the corrugator supercilii [eyebrow] and masseter [jaw], (Smith 1989)). These signals were selected because they can be non-invasively measured and are relatively resistant to movement artifact. Various signal processing techniques such as Fourier transform, wavelet transform, thresholding, and peak detection, were used to derive the relevant parameters from the physiological signals. All of these parameters are powerful indicators of a humans underlying emotional state response. We have exploited this dependence of a humans physiological response on emotions to detect and identify affective states in real-time using advanced signal processing techniques. A prototypical robotic system based on Brook's subsumption architecture that can adapt its autonomy level based on affective sensing has been designed and implemented by Rani et al (Rani et al. 2004).

General Mixed-Initiative Interaction Model

Riley (Riley 1989) defined a general mixed-initiative interaction model for human-machine systems (to the best of our knowledge, ours is the first application to HRI). His model defines a taxonomy containing two factors that define the level of intelligence and the level of autonomy. The combination of the two factors determines the actual model components that compose the final system. The taxonomy defines twelve automation levels and seven intelligence levels. As the levels of intelligence and autonomy increase a larger number of system components are included from the general model.

The model is essentially a loop through a structure that represents the machine (robot), the human operator, and the world. The machine (robot) and human operator representations surround the world module and each of the two larger representations are divided into input and output modules. A complete system model (highest levels of automation and intelligence) includes four inference modules: operator state, operator intent, world state, and operators knowledge. Such a model would also include modules for determining information to provide to the human operator, planning actions and maintaining a self-model. At this level of intelligence and automation, the human essentially becomes a partner with the system with the ability to provide information, request information, as well as command and/or control the system.

The autonomy and intelligence levels are determined by completing a Function Allocation Issues and Tradeoffs (FAIT) analysis (Riley 1992). The FAIT analysis identifies potential human factors issues and requirements. The full analysis conducts a functional decomposition to identify characteristics specific to the equipment, the human operators, and the operational environment. The analysis is composed of two components. The first contains two sets of questions regarding autonomy and intelligence. The results of these questions determine the appropriate autonomy and intelligence levels for the particular system, thus defining the particular model (modules) that the system requires. The second component requires that each system module be analyzed to identify the specific characteristics that are relevant to system operation.

The autonomy levels include: no autonomy, information fuser, simple aid, advisor, interactive advisor, adaptive advisor, servant, assistant, associate, partner, supervisor, and full autonomy. The intelligence levels include: simply providing raw data (no intelligence), procedural, context responsive, personalized, inferred intent responsive, operator state responsive, and operator predictive.

We have developed a preliminary MII architecture for affect-based HRI based upon completing the first component of the FAIT analysis. The second component of the FAIT analysis is quite involved and time consuming. Due to on-going data collection necessary for

understanding the affect based sensing, we have not yet completed the second FAIT analysis component.

Proposed Architecture

Based upon the results of completing the first component of Rileys FAIT analysis, the resulting system requires various levels of automation up to and including full autonomy. As well, the system requires levels of intelligence from the raw data level up to and including operator predictive capabilities.

Figure 1. The MII human-robot collaborative architecture.

Figure 1 provides the high level mixed-initiative humanrobot collaborative architecture. It can be seen that both the robot and the human obtain and receive information from the world. They interact with each other via the HRI. Each entity (human and robot) performs an inference based upon all received information to form representations of the world, itself, and its partner. The robot and the human dynamically modify the goals and constraints by interacting via the HRI. Once the goals and constraints are determined, they are sent to the planning block where the steps to accomplish the given mission are outlined and validated with the human and the robot. The final plan is then passed to the execution block that executes the steps one by one. The execution of each plan step changes the world state, as well as the human operator and the robot states. These changes are transmitted to the robot and the human operator who accordingly update their representations.

Figure 2 provides the details that feed into the Operator State block (from the Robots Inference block in Figure 1). The figure demonstrates how implicit and explicit communication is employed to determine the human operator's physical and mental state. The affect-sensing block determines the humans implicit state (affective state prediction using physiological signals) and the HRI along and world state help determine the explicit operator state.

Figure 2. Operator state module detail.

Figure 3. Affect sensing block.

Figure 3 provides the Affect Sensing block details (from Figure 2). The humans physiological signals indicate the humans affective state and are measured using biofeedback sensors (Cardiac activity, electrodermal response etc.). The information regarding the humans affect state from the previous time instant can be used to determine the current operator state. Therefore the robot is supplied with the previous operator state for purposes of context-based reasoning. The block hold signifies the operation of retaining or holding the current operator state so that it can be used in the future as a representation of the previous operator state.

Figure 4. Dynamic goal and constraint change block detail.

Figure 4 provides the block diagram that demonstrates the process of dynamically changing the goals and constraints in real time (Dynamically determined Goals and Constraints block in Figure 1). The human operator and the robot manipulate the goals and constraints through the HRI. The HRI is connected to the goals block to facilitate this interaction. Either the human or the robot (whoever has the initiative to change the goals and constraints) modifies the existing goals and constraints to suit the long term and short-term objectives as required.

Figure 5. Planning and control block detail.

Figure 5 provides the Planning block detail (from Figure 1). Here again the agent who has the initiative/authority to conduct planning does the planning. If the robot directs the planning, these activities are communicated to the human via the HRI. If the human is responsible for the planning, this is directed via the HRI. The planner uses the current goals and constraints as well as the last task that was executed to determine the best plan. The plan that is generated is cross-validated before execution. This validation requires sending the plan to the agents via the HRI. Once the plan is confirmed then it is sequentially executed and each task is validated before execution.

It should be noted that the MII starts at the HRI level where either the human or robot assumes initiative to dynamically determine the mission goals and constraints. For instance, if the task is to repair part of a workstation, the goals (what to repair, what the repaired performance level should be, etc.) and constraints (available resources, time constraints, etc.) need to be dynamically determined. The planning block generates the sequence of tasks necessary to achieve the goals. Planning also involves the task distribution between the agents and can be achieved by either the human or the robot. This is determined by the interaction between the two agents via the HRI. Once completed the plan is verified via the HRI. Upon verification, the plan is executed. Either of the two agents can then assume initiative, which is done through the HRI. Either the human or the robot can intervene in the execution sequence by raising an interrupt that stops the execution if a threat is perceived or an error occurs. This architecture should be flexible enough to allow the agents

to assume and relinquish initiative depending upon their capabilities, resources, inferences, and interactions with each other. The architecture should also enable the system to quickly react to the changing environment by dynamically changing the goals and constraints. The planning block with an embedded feasibility analysis component will enable detailed plan analysis before execution. The process of forming models of each other and the world creates the agents knowledge of each others capabilities and limitations. This is a critical requirement for initiative switching. Hence, this architecture should enable us to implement fast and reliable affect-based as well as standard mixed-initiative interactions between humans and robots.

Initiative and Affect-based Sensing

A crucial question in MII is: Why take initiative? This issue was touched upon in the Introduction section and we further evaluate this issue as it relates to our system. Goals and constraints dynamically change with changing world, robot, and operator states. If there is a sudden robot breakdown, the system constraints may change. Any world state change may also change the mission goals. For instance, if the goal is to track a white object and the object color changes, the goals may change accordingly.

The human maintains internal world and robot state representations. The human also has an internal representation of himself or herself. The human may detect any changes in the robot or world states that prompt the human to make changes to the mission goals and constraints. Similarly the robot (which also has representations of the world, the human and itself) can detect unpredicted changes in the world and human operator states that prompt modifications to the goals and mission constraints. If the robot detects extreme stress or panic in the human, the robot may make efforts to rescue the human from a risky situation or abort the mission. Hence, both the human operator and the robot have the authority to take initiative dependent upon their environmental perceptions.

An important question is: When should a particular agent assume initiative? Some triggers that may cause the robot or human to take initiative away from the other agent include: the human or robot is in danger, expresses an inability to perform, or requests assistance; the world state changes; or the resource requirements change.

The next question is: How to assume initiative? This is a domain dependent issue but manners in which the human or robot can take initiative include: modification of the mission goals or plan; altering the resource constraints, altering the manner of execution; or aborting the mission.

While these are basic questions, there are additional constraints and requirements when developing MII for an affect-sensitive architecture. First, the robot should be able to detect, recognize, and respond to human affective states

while executing routine tasks. Required capabilities to accommodate this requirement include:

Reactive response to certain affective states (i.e. fatigue, drowsiness, inattention, etc.).

Deliberative response to other affective states (i.e. anger, frustration, etc.).

Knowledge and ability to prioritize the tasks at hand.

Learning in order to diagnose affective states. Knowledge of when and how to take the initiative. An additional requirement is the provision for a

communication/interaction capability between the robot and human. This mechanism is necessary to permit negotiation with the human regarding impending tasks and goals. This interaction may occur via speech, standard graphical user interface capabilities or other multimodal interaction capabilities.

Initiative Control Matrix

Initiative can be seized either by the human or the robot. In cases of conflict the human operator has the ultimate control and gains the initiative. There exist four possible means of seizing the initiative.

The first method requires the robot to seize the initiative from the human operator. This may occur when the human is drowsy, sleepy, or dangerously inattentive. In this case, the robot may cut the human off from the system, go into an autopilot mode, or may simply shut the process down.

The robot may offer the initiative to the human operator. If the human is bored or under challenged, the robot may offer the human the initiative to assume some of the autonomous tasks such as planning, updating environment variables, conducting feasibility checks, etc. The purpose is to maintain the humans vigilance level by making the task more interesting or interactive.

The human may also seize the initiative from the machine. If the world state unexpectedly or suddenly changes, the human may seize initiative. This may be done because the human perceives that the robot may not be equipped to handle the current situation.

Finally, the human may offer the initiative to the robot. This may occur when the current task is an easy, routine job, or is so complicated that it is beyond the humans capabilities. If the task is routine, then this will typically occur when the human operator has more pressing tasks to complete. When the task is complicated, the human may offer initiative because he or she does not feel confident in the ability to successfully complete the task.

Based upon the four combinations of offering and seizing may occur. They are: Robot offers Human Seizes (Ro-Hs); Robot seizes Human offers (Rs-Ho); Robot seizes Human Seizes (Rs-Hs); and Robot offers Human offers (Ro-Ho).

The Ro-Hs situation occurs when the robot offers initiative to the human and the human operator seizes the

opportunity to take initiative. The Rs-Ho situation occurs when the Human offers initiative to the robot and the robot seizes the initiative. Rs-Hs occurs when both the human and the robot simultaneously seize initiative. In this case the human has higher priority and he or she seizes the initiative. Finally, the Ro-Ho situation occurs when both the human and the robot simultaneously offer initiative. In this case neither the human nor the robot believes that the task can be done. When this case arises, the following options exist: share the initiative; if possible, replan; shut down the system and/or abort the task; start the task all over again; consult the main control station.

Figure 6. Human/Robot Initiative-Switching Matrix.

An initiative-switching matrix is shown in Figure 6. Each cell represents a typical scenario during an HRI sequence. The upper, left cell represents the case when both the human and the robot are ready to give up the initiative to the other agent (Ro-Ho). In this case the human decides who will retain the initiative. The Robot seizes, Human offers cell indicates the case where the robot seizes the initiative while the Robot offers, Human seizes cell represents the case in which the human seizes the initiative. The ambivalent term refers to scenarios where the agent can either give or take initiative, i.e., it is capable of doing the task entirely on its own or with some assistance. Hence if the human is ambivalent about taking initiative and the robot seizes initiative, then the human will relinquish initiative and vice versa. The same holds for the robot ambivalence. If both the agents are ambivalent then they can either share the initiative or the human can assign initiative.

Discussion

The robots ability to interpret implicit human operator states (emotional/affective state) is critical when implementing a mixed-initiative interaction between humans and robots. There are various emotional indicators that can be exploited to identify affective states: facial expressions, gestures, vocal intonation, physiology etc. We focus on using physiological responses, as these are generally involuntary and less dependent on culture, gender, and age than the other emotional indicators. These responses offer an opportunity to recognize emotions that

may be less intuitive for humans but more appropriate for robots, which can acquire the physiological signals in real-time and employ various signal processing and pattern recognition techniques to infer the underlying emotional states. Recent innovations in affective computing (Picard 1997) and wearable computers have made it feasible to process physiological signals using small, lightweight biofeedback sensors that are non-invasive, comfortable, unobtrusive, and fast enough for real-time applications. The potential of physiological sensing for affect recognition has already been demonstrated by the pioneering work of Picard and her colleagues (Picard 1997, Healy & Picard 1998).

Currently we are collecting data from fifteen volunteer participants as they engage in a series of cognitive tasks designed to systematically and differentially evoke the emotions targeted towards recognition (boredom, task-engagement, anxiety, and frustration). The data collected will be used to develop a model of each individuals physiological expression of the targeted emotions. This will involve identifying, for each participant, the physiological correlations and response patterns that characterize the experience of each of the emotions. Special emphasis is being placed on identifying physiological markers and patterns that differentiate each of the states from the others. The resulting models will then provide the basis for the development of affect recognition algorithms for each individual, which will also be built using this initial data set.

Once the affective models for determining a humans implicit state are determined, the MII architecture will be incorporated in a human-robot collaboration task. In the actual MII system, a mobile robot (Trilobot) will be used as a test bed. The initial experiments will send the original physiological data (originally collected using Biopac System) to the robot as if it were being communicated in real-time. The robot will receive this data and interpret the humans affective state. The robot will employ other sensors (infrared sensors, light sensors, motion detection sensors, etc.) to sense its environment and formulate a world model. The robot can also send messages to a human operator via the HRI. The system will be tested using human-robot collaboration applications such as exploration, and search and rescue.

Conclusions

Mixed-initiative interaction plays a crucial role in

developing collaborative HRI. The focus is to provide an interaction that permits the human and robot the ability to interact in a manner similar to human interaction. The interaction focuses on which agent has the initiative and how initiative transitions from one agent to another. MII is not a new concept; Carbonell (Carbonell 1971) introduced the term in 1971 in relation to Computer Assisted Instruction. Throughout the years, the MII concept has

been applied to a large domain of problems including: dialogue and discourse understanding, planning, human-machine interaction, and human-robotic interaction.

Our work focuses on the development of affect-based interaction between a human operator and a robot. We have developed a preliminary mixed-initiative humanrobot collaborative architecture that relies on affect-based and standard human-robot interaction capabilities. Rileys (Riley 1989) general model of MII has been applied to the development of this architecture. In addition to the architecture presentation, we have also presented some affect-based constraints that were considered while developing this architecture. Currently data collection is being conducted to develop the affective models that will be used for testing this architecture.

The future work includes designing HRI evaluations wherein we can elicit a range of emotions from the participants and test the speed and reliability of the robot's MII architecture in real-time. Future work also consists of expanding the range of tasks and contexts to which this framework can be applied and increasing the reliability and sophistication of the emotion recognition.

References Anzai, Y. 1994. Human-Robot-Computer Interaction: a New Paradigm of Research in Robotics. Advanced Robotics, 8(4): 357-369. Bell, B., Franke, J., & Mendenhall, H. 2000. Leveraging Task Models for Team Intent Inference. Proc. 2000 International Conference on Artificial Intelligence. Bradley, M., M., 2000. Emotion and Motovation. In J. T. Cacioppo, L G. Tassinary, & G. G. Berntson (Eds.). Handbook of Psychophysiology, 2nd Ed, pp. 602-642. New York: Cambridge University Press. Bruemmer, D.J., Marble, J.L., Dudenhoeffer, D.D., Anderson, N.O. & McKay, M.D. 2002. Mixed-Initiative Control for Remote Characterization of Hazardous Environments. Proc. of the 36th Hawaii International Conference on System Sciences. Burnstein, M., Ferguson, G., & Allen, J. 2003. Integrating Agent-Based Mixed-Initiative Control with an Existing Multi-Agent Planning System. University of Rochester, Computer Science Department, Technical Report 729. Carbonell, J.R. 1971. Mixed-Initiative Man-Computer Instructional Dialogues, BBN Report No. 1971, Job No. 11399, Bolt Beranek and Newman, Inc. Chu-Carroll, J., & Brown, M.K. 1997. Initiative in Collaborative Interactions Its Cues and Effects. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction , Tech Report SS-97-04, pp. 16-22. Cinotti, G., Loi, G., & Pappalardo, M. 2001. Frequency Decomposition and compounding of Ultrasound Medical Images With Wavelet Packets. IEEE Trans. Med. Imagining, 20(8): 764-771. DAloisi, D., Cesta A., & Brancaleoni, R. 1997. Mixed-Initiative Aspects in an Agent-Based System. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction , Tech Report SS-97-04, pp. 30-36.

Donaldson, T. & Cohen, R. 1997. A Constraint Satisfaction Framework for Managing Mixed-Initiative Discourse. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction, Tech Report SS-97-04, pp. 37-43. Fleming, M. & Cohen, R. 1999. User Modeling in the Design of Interactive Interface Agents. Proc. of the 7th International Conference on User Modeling, pp. 66-76. Frijda, N., H. 1986. The Emotions, chapter Physiology of Emotion, pages 124-175. Studies in Emotion and Social Interaction. Cambridge University Press, Cambridge. Goldman, R.P., Guerlain, S., Miller, C., & Musliner, D.J. 1997. Integrated Task Representation for Indirect Interaction. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction, Technical Report SS-97-04. Goodrich, M.A., Olsen, D.R., Crandall, J.W. & Palmer, T.J. 2001. Experiments in Adjustable Autonomy. International Joint Conference on Artificial Intelligence - Workshop Autonomy, Delegation, and Control: Interacting with Autonomous Agents. Guinn, C.I. 1998. An Analysis of Initiative Selection in Collaborative Task-Oriented Discourse. User Modeling and User-Adapted Interaction, 8: 255-314. Healy, J., & Picard, R. W., 1998. Digital Processing of Affective Signals. Technical Reports #444, Vision and Modeling Group, Media Lab, MIT. Hearst, M.A. Ed., 1999. Trends & Controversies: Mixed-Initiative Interaction. IEEE Intelligent Systems, 14(5): 14-23. Horiguchi, Y., & Sawaragi, T. 2001. Naturalistic Human-Robot Collaboration Mediated by Shared Communicational Modality in Teleoperation System. Proc. of the Sixth International Computer Science Conference of Active Media, pp. 24-35. Horvitz, E., 1999. Principles of Mixed-Imitative User Interfaces. Proc. of the Conference on Human Factors in Computing Systems, pp.159-166. Ishizaki, M., Crocker, M., & Mellish, C. 1999. Exploring Mixed-Initiative Dialogue Using Computer Dialogue Simulation. User Modeling and User-Adapted Interaction, 9L 79-91. Kortenkamp, D., Bonasso, R.P., Ryan, D. & Schreckenghost, D. 1997. Traded Control with Autonomous Robots as Mixed Initiative Interaction. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction, Technical Report SS-97-04. Lester, J.C., Stone, B.A., & Stelling, G.D. 1999. Lifelike Pedagogical Agents for Mixed-Initiative Problem Solving in Constructivist Learning Environments. User Modeling and User-Adapted Interaction, 9: 1-44. Lemon, O., Bracy, A., Gruenstein, A., & Peters, S. 2001. The WITAS Multi-Modal Dialogue System I. Proc. of the 7th European Conference on Speech Communication and Technology, pp. 1559-1562. Mitchell, S. 1997. An Architecture for Real-Time Mixed Initiative Collaborative Planning. 1997 AAAI Spring Symposium: Computational Models for Mixed Initiative Interaction, Tech Report SS-97-04, pp. 111-113. Murphy, R., Casper J.L., Micire, M.J., & Hyams, J. 2000. Mixed-Initiative Control of Multiple Heterogeneous Robots for Urban Search and Rescue. University of South Florida, Center for Robot Assisted Search and Rescue Technical Report, CRASAR-TR2000-11. hman, A., Hamm, A., & Hugdahl, K. 2000. Cognition and the autonomic nervous system: Orienting, anticipation, and conditioning. In J. T. Cacioppo, L G. Tassinary, & G. G. Berntson (Eds.). Handbook of

Psychophysiology, 2nd Ed, pp. 533-575. New York: Cambridge University Press. Penner, R.R., Nelson, K.S., & Soken, N.H. 1997. Facilitating Human Interactions in Mixed Initiative Systems Through Dynamic Interaction Generation. Proc. of the IEEE International Conference on Systems, Man and Cybernetics, pp. 714-719. Ramakrishnan, N., Capra, R., & Perez-Quinones, M. 2002. Mixed-Initiative Interaction = Mixed Computation. 2002 ACM SIGPLAN Workshop on Partial Evaluation and Semantics-Based Program Manipulation, pp. 119-130. Picard, R. 1997. Affective Computing. The MIT Press, Cambridge, Massachusetts. Rani, P., Sarkar, N., Smith, C. & Kirby, L. 2004. Anxiety Detecting Robotic Systems Towards Implicit Human-Robot Collaboration. Robotica, 22(1): 85-95. Rich, C., & Sidner, C.L. 1998. COLLAGENT: A Collaboration Manager for Software Interface Agents. User Modeling and User-Adapted Interaction, 8: 315-350. Riley, V. 1989. A General Model of Mixed-Initiative Human-Machine Systems. Proc. of the Human Factors Soceity33rd Annual Meeting, pp. 124-128. Riley, V. 1992. Human Factors Issues of Data Link: Application of A Systems Analysis. Society of Aeronautic Engineers, Aerotech 92, Anaheim, CA. Roy, N., Baltus, G, Fox, D., Gemperle, F., Goetz, J., Hirsch, T., Margaritis, D., Montemerlo, M., Pineau, J., Schulte, J., & Thrun, S. 2000. Towards Personal Service Robots for the Elderly. Workshop on Interactive Robots and Entertainment. Scherer, K. R. 1984. On the nature and function of emotion: A component process approach. In K. R. Scherer & P. Ekman (Eds.), Approaches to Emotion, pp. 293-317. Hillsdale, NJ, Erlbaum. Sherwood, R., Mishkin, A. Estlin, T., Chein, S., Backes, P., Norris, J., Cooper, B., Maxwell, S., & Rabideau, G. 2001. Autonomously Generation Operations Sequences for a Mars Rover using AI-based Planning. Proc. of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 803-808. Skinner, J.M. Unknown. Multi-Agent Systems and Mixed-Initiative Intelligence. LEF Grant report. www.csc.com/aboutus/lef/mds67_off/uploads/skinner_mixed_initiative_agents.pdf. Smith, C., A., & Lazarus, R., S. 1990. Emotion and adaptation. Handbook of personality: Theory and research, L.A. Pervin (Ed.), pp. 609-637, New York, Guilford. Smith, C. A. 1989. Dimensions of appraisal and physiological response in emotion. Journal of Personality and Social Psychology, 56(3): 339-353. Wortman, B.C., Loftus, E. F., & Weaver, C. 1998. Psychology, McGraw-Hill, 5th edition. Zheng, X.-Z., Tsuchiya, K., Sawaragi T., Osuka, K., Tsujita, K., Horiguchi, Y., & Aoi, S. 2004. Development of Human-Machine Interface in Disaster-Purposed Search Robot Systems that Serve as Surrogates for Human. Proc. of the IEEE International Conference on Robotics and Automation, pp. 225-230.