Upload
louise-banks
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Mental Models for Human-Robot Interaction
Christian Lebiere ([email protected])1
Florian Jentsch and Scott Ososky2
1Psychology Department, Carnegie Mellon University2Institute for Simulation and Training, University of
Central Florida
Cognitive Models of Mental Models
• Mental models provide a representation of situation, various entities, capabilities, & past decisions/actions
• Current models are non-computational descriptions• Cognitive models can provide computational link to
overall robotic intelligence architecture for dual uses:– Provide a quantitative, predictive understanding of human
team shared mental models– Support improved design of human-robot interaction tools
and protocols– Provide a cognitively-based computational basis for
implementation of mental models in robots
Representation Components
• Mental model representation– Ontology of concepts and decisions
• Lexical (WordNet), Structural (FrameNet), Statistical (LSA)
– Symbolic frameworks• Decision trees, semantic networks
– Statistical frameworks• Bayesian networks, semantic similarities
• Knowledge of task situation– Situation awareness – mapping to levels of SA– Environment limitations – who sees/knows what (perspective)– Architectural limitations – who remembers what (WM, decay)
Reasoning and inference
• Inferring mental models– Instance-based learning (Gonzalez & Lebiere)
• E.g., Learning to control systems by observation or imitation
• Inferring current knowledge– Perspective-taking in spatial domain (Trafton)
• E.g., hide and seek, collaborative work
• Predicting decisions– Theory of mind recursion (Trafton, Bringsjord)– Imagery-based simulation (Wintermutte)– Shared plan execution in MOUT (Best & Lebiere)– Sequence learning in game environments (West & Lebiere)
Cognitive Architectures• Computational representation of
invariant cognitive mechanisms• Behavior selection
– Production systems– Utility – rewards and costs
• Memories– Working memory: buffers– Long-term: semantic/episodic– Activation mechanisms
• Learning– Symbolic and statistical
• Human factor limitations– Perceptual-motor parameters
• Individual differences– Strategies and knowledge– Capacity parameters
EnvironmentPr
oduc
tions
(Bas
al G
angl
ia)
Retrieval Buffer (VLPFC)
Matching (Striatum)
Selection (Pallidum)
Execution (Thalamus)
Goal Buffer (DLPFC)
Visual Buffer (Parietal)
Manual Buffer (Motor)
Manual Module (Motor/Cerebellum)
Visual Module (Occipital/etc)
Intentional Module (aPFC)
Declarative Module (Temporal/Hippocampus)
Pursuit Task
• Follow that Guy: human soldier and robot teammate– Shared mental model of pursuit situation scenario
• Set of data encoding various scenarios• Items organized according to SMMs held by expert teams
(Equipment, Task, Team Interaction, Team)• Decision tree built using information from police “foot
pursuit” procedures• For each decision, the most critical item is listed
– However, other factors may be considered in weighing decision
• Loop to end or continue the pursuit given fluid situation
Data
Scenario Data and Decision Tree
Part 1: Who should pursue?
Start
H-R Communication reliable (5x5)?
Is the terrain negotiable for robot?
Are suspects armed?
Robot only pursuit
Soldier only pursuit Team pursuitHold position,
report incident
Continue to Part 2: pursuit loop
Is the threat immediate (civilians, etc.)
Are sensors reliable in the search area?
Current last known location?
YES YES YES YES
YES
YES
No No No No
Is backup support available?
Immediate threat / critical situation?
No
NoNo
YES
No
YES
EQ-C3 SK-E3 EQ-S3 SK-S2 SK-S8
IA-A1 SK-S8 SK-S7
Is the suspect armed?
Was this, or is there potential for a violent crime?
Can a perimeter be set up to contain the suspect?
Do you have supervisor clearance?
Deciding whether to
pursue
Do you know the identity of the suspect?
Are backup units available to assist you?
Begin or Continue pursuit Do you have line
of sight with suspect?
Can you apprehend them at a later time?
What are the traveling surface conditions?
What is the pedestrian traffic like?
What are the weather conditions?
YesNo NoYes YesNo No Yes
Yes No Yes No Yes No
Are communications functioning properly?
Yes No
Light/ ModerateHeavy Good/
Fair PoorGood/
Fair PoorYes No
SK-S2SK-S1 SK-S3 SK-A1
IA-A1 EQ-C3 SK-A2
IA-R1
TM-W1 SK-E1 SK-E2 SK-E3
General Cognitive Model• Develop general model that takes mental models in the form
of decision trees and learns to retrieve and execute them• Each decision is represented as sequence of chained steps• Each piece of data is represented as separate chunk• Model (7 p* production rules) depends on declarative memory
to retrieve rule steps, data items and decision instances– No hardcoded decision logic
• Each decision depends on matching against past instances combining activation recency, frequency and partial matching
• Stochasticity of activation results in probabilistic decisions• Run model in Monte Carlo mode for decision distribution• Cross-validation: train on some scenarios, test on others
Individual Decision Inference
Overall Decision Agreement
Generalized Condition
• 35 scenarios• 3 experts• Intermediate
decisions• Relative
rankings• Desirability
ratings• Comments
Results• Match to first-last
ranks, poor middle• Slightly different
ratings pattern• Comparable cross-
expert correlations
Learning
• Proceduralize individual steps from declarative instructions to production rules to replicate learning curve from novice to proficiency and expertise
• Apply feature selection using utility learning to encode and use only a subset of data items for each decision
• Learn shortcuts that combine multiple individual binary decisions into single, multi-outcome decision
• Generate rankings/ratings from probability judgments generated from activation of memory retrievals
• Abstract decision instances into discrete types
Future Work
• Validate model against human participants data along entire learning curve and broad range of situations
• Explore Bayesian network formalism as alternative to enhance generalization in multi-step decisions
• Integrate cognitive model in multi-agent simulations to validate computational mental model in dynamic decision-making setting
• Integrate computational cognitive model on robotic platform to assess ability to improve human-robot interaction through shared models