Upload
elizabeth-mccoy
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Improving Improving Human-Robot InteractionHuman-Robot Interaction
Jill Drury, The MITRE CorporationJill Drury, The MITRE Corporation
Collaborators:Collaborators:
Holly Yanco, UMass LowellHolly Yanco, UMass LowellJean Scholtz, NISTJean Scholtz, NIST
Mike Baker, Bob Casey, Dan Hestand, Mike Baker, Bob Casey, Dan Hestand, Brenden Keyes, Phil Thoren (UML)Brenden Keyes, Phil Thoren (UML)
Methodology for Evaluating HRIMethodology for Evaluating HRI
Two approaches: field and laboratoryTwo approaches: field and laboratoryField work: so far, robotics competitionsField work: so far, robotics competitions See many different user interfaces but have no See many different user interfaces but have no
control over what operator doescontrol over what operator does Difficult to collect dataDifficult to collect data Can see what they did – but there isn’t time to Can see what they did – but there isn’t time to
determine whydetermine why Best used to get an idea of the difficulties in the real Best used to get an idea of the difficulties in the real
worldworld Can identify “critical events” but don’t know for certain Can identify “critical events” but don’t know for certain
whether operator was aware of themwhether operator was aware of them
Methodology for Evaluating HRIMethodology for Evaluating HRI
Laboratory studiesLaboratory studies Take what we learned in the real world and Take what we learned in the real world and
isolate factors to determine effects isolate factors to determine effects Repeatability is still difficult to achieve due to Repeatability is still difficult to achieve due to
fragile nature of robotsfragile nature of robots
Analysis FrameworksAnalysis Frameworks
Taxonomy to define human/robot systemTaxonomy to define human/robot system
Detailed definition of human-robot Detailed definition of human-robot interaction awarenessinteraction awareness
Coding scheme/metrics for analyzing dataCoding scheme/metrics for analyzing data
Scholtz’ evaluation guidelinesScholtz’ evaluation guidelines
TaxonomyTaxonomyAutonomyAutonomy
Amount of interventionAmount of intervention
Human-robot ratioHuman-robot ratio
Level of shared interactionLevel of shared interaction
Composition of robot teamsComposition of robot teams
Available sensorsAvailable sensors
Sensor FusionSensor Fusion
CriticalityCriticality
TimeTime
SpaceSpace
Awareness as a concept from Awareness as a concept from CSCWCSCW
CSCW software:CSCW software: ““...makes the user aware that ...makes the user aware that
he is part of a group, while he is part of a group, while most other software seeks to most other software seeks to hide and protect users from hide and protect users from each other” [Lynch et al. 1990]each other” [Lynch et al. 1990]
HRI software:HRI software: Makes humans aware of Makes humans aware of
robots’ status and activities via robots’ status and activities via the interfacethe interface
What is “awareness”?What is “awareness”?
No standard definition in the CSCW fieldNo standard definition in the CSCW field We’ve seen at least 16 different definitions!We’ve seen at least 16 different definitions!
Are many different types of awareness, e.g.Are many different types of awareness, e.g. Concept awarenessConcept awareness Conversational awarenessConversational awareness Group-structural awarenessGroup-structural awareness Informal awarenessInformal awareness Peripheral awarenessPeripheral awareness
Common thread: understanding that participants have of Common thread: understanding that participants have of each other in a shared environmenteach other in a shared environment
Situation awarenessSituation awareness Social awarenessSocial awareness Task awarenessTask awareness Workspace Workspace
awarenessawareness
But CSCW systems are But CSCW systems are different from robotic systems…different from robotic systems…
CSCW:CSCW: Multiple humans interacting via a CSCW systemMultiple humans interacting via a CSCW system
Robotics:Robotics: Single or multiple humans interacting with a single or Single or multiple humans interacting with a single or
multiple robotsmultiple robots Non-symmetrical relationships between humans and Non-symmetrical relationships between humans and
robots; e.g., differences in robots; e.g., differences in Free willFree will
CognitionCognition
Tailoring an awareness Tailoring an awareness definition for HRI: a base casedefinition for HRI: a base caseGiven one human and one robot...Given one human and one robot...
... HRI awareness is the understanding that the human ... HRI awareness is the understanding that the human has of the has of the location, location, activities, activities, status, and status, and surroundings of the robot; and surroundings of the robot; and
the knowledge that the robot has ofthe knowledge that the robot has of the human’s commands necessary to direct its activities and the human’s commands necessary to direct its activities and the constraints under which it must operatethe constraints under which it must operate
An awareness framework: An awareness framework: General caseGeneral case
Given n humans and m robots working together Given n humans and m robots working together on a synchronous task, HRI awareness consists on a synchronous task, HRI awareness consists of five components:of five components: Human-robot awarenessHuman-robot awareness Human-human awarenessHuman-human awareness Robot-human awarenessRobot-human awareness Robot-robot awarenessRobot-robot awareness Humans’ overall mission awarenessHumans’ overall mission awareness
General case: a detailed lookGeneral case: a detailed look
Given n humans and m robots working together Given n humans and m robots working together on a synchronous task, HRI awareness consists on a synchronous task, HRI awareness consists of five components:of five components: Human-robotHuman-robot: the understanding that the humans : the understanding that the humans
have of the locations, identities, activities, status and have of the locations, identities, activities, status and surroundings of the robots. Further, the surroundings of the robots. Further, the understanding of the certainty with which humans understanding of the certainty with which humans know this information.know this information.
Human-humanHuman-human: the understanding that the humans : the understanding that the humans have of the locations, identities and activities of their have of the locations, identities and activities of their fellow human collaboratorsfellow human collaborators
General case, concludedGeneral case, concluded Robot-humanRobot-human: the robots’ knowledge of the humans’ : the robots’ knowledge of the humans’
commands needed to direct activities and any human-commands needed to direct activities and any human-delineated constraints that may require command delineated constraints that may require command noncompliance or a modified course of actionnoncompliance or a modified course of action
Robot-robotRobot-robot: the knowledge that the robots have of the : the knowledge that the robots have of the commands given to them, if any, by other robots, the commands given to them, if any, by other robots, the tactical plans of the other robots, and the robot-to-robot tactical plans of the other robots, and the robot-to-robot coordination necessary to dynamically reallocate tasks coordination necessary to dynamically reallocate tasks among robots if necessary.among robots if necessary.
Humans’ overall mission awarenessHumans’ overall mission awareness: the humans’ : the humans’ understanding of the overall goals of the joint human-robot understanding of the overall goals of the joint human-robot
activities and the measurement of the moment-by-momentactivities and the measurement of the moment-by-moment progress obtained against the goalsprogress obtained against the goals..
Coding Scheme: Problems Coding Scheme: Problems Relating to Critical IncidentsRelating to Critical Incidents
Critical incident: Robot has, or could, cause Critical incident: Robot has, or could, cause harm or damage harm or damage
Types of problems:Types of problems: Local navigationLocal navigation Global navigationGlobal navigation Obstacle encounterObstacle encounter Vehicle stateVehicle state Victim identification (specific to search and Victim identification (specific to search and
rescue)rescue)
Some Metrics for HRISome Metrics for HRI
Time spent navigating, on UI overhead and Time spent navigating, on UI overhead and avoiding obstaclesavoiding obstaclesAmount of space coveredAmount of space coveredNumber of victims foundNumber of victims foundCritical incidentsCritical incidents Positive outcomesPositive outcomes Negative outcomesNegative outcomes
Operator interventionsOperator interventions Amount of time robot needs helpAmount of time robot needs help Time to acquire situation awarenessTime to acquire situation awareness Reason for interventionReason for intervention
Scholtz’ Guidelines (tailored)Scholtz’ Guidelines (tailored)Is sufficient status and robot location information available so that the operator knows the robot is operating correctly and avoiding obstacles?
Is the information coming from the robots presented in a manner that minimizes operator memory load, including the amount of information fusion that needs to be performed in the operators’ heads?
Are the means of interaction provided by the interface efficient and effective for the human and the robot (e.g., are shortcuts provided for the human)?
Does the interface support the operator directing the actions of more than one robot simultaneously?
Will the interface design allow for adding more sensors and more autonomy?
Design GuidelinesDesign Guidelines
Enhance awarenessEnhance awareness Provide a map of where the robot has beenProvide a map of where the robot has been Provide more spatial information about the robot in Provide more spatial information about the robot in
the environment to make the operators more award of the environment to make the operators more award of their robot’s immediate surroundingstheir robot’s immediate surroundings
Lower cognitive loadLower cognitive load Provide fused sensor information to avoid making the Provide fused sensor information to avoid making the
user fuse data mentallyuser fuse data mentally Display important information near or fused with the Display important information near or fused with the
video imagevideo image
Design Guidelines, concludedDesign Guidelines, concluded
Increase efficiencyIncrease efficiency Provide user interfaces that support multiple Provide user interfaces that support multiple
robots in a single window, if possiblerobots in a single window, if possible In general, minimize the use of multiple In general, minimize the use of multiple
windows and maximize use of the primary windows and maximize use of the primary viewing areaviewing area
Provide help in choosing robot modalityProvide help in choosing robot modality Give the operator assistance in determining Give the operator assistance in determining
the most appropriate level of robot autonomy the most appropriate level of robot autonomy at any given timeat any given time
Fusing InformationFusing Information
Victims can be missed in video imagesVictims can be missed in video images
Fusing Infrared and Color VideoFusing Infrared and Color Video
Fusing Infrared and Color VideoFusing Infrared and Color Video
Fusing Infrared and Color VideoFusing Infrared and Color Video
Other Sensor Modalities for USAROther Sensor Modalities for USAR
COCO22 detection detection
AudioAudio
Overlay of four sensor modalitiesOverlay of four sensor modalities
Overlay of four sensor modalitiesOverlay of four sensor modalities