Matthew Johnson has worked at the Florida Institute for Human and Machine Cognition since 2002. He received his B.S. in Aerospace Engineering from the University of Notre Dame, a M.S. in Computer Science from Texas A&M – Corpus Christi, and his PhD in Computer Science through Delft University of Technology in the Netherlands.Prior to working for IHMC, he flew both fixed and rotary wing aircraft in the Navy, retiring after 20 years of service. He has worked on numerous projects including the Oz flight display for reducing the cognitive workload in the cockpit, Augmented Cognition for improving human performance, and several human-‐robot coordination projects for both NASA and the Department of Defense.He has worked on advanced robotic control projects such as the DARPA Little Dog project developing walking algorithms for a quadruped robot on rough terrain and the IHMC lower body humanoid developing low-‐gravity walking gaits for NASA. Most recently, he has been working on developing interfaces to enable micro-‐air vehicle control in complex urban environments and competing in the DARPA Robotics Challenge. Matthew’s research interest focuses on improving performance in human-‐machine systems through design of more effective human-‐machine teamwork.
PowerPoint Presentation
Presented by Matthew [email protected] Telecon
03-18-15
Florida Institute for Human and Machine CognitionCoactive
DesignDesigning for Interdependence in Robotics1
1
Goals For This TalkShake up your assumptions about robotics
Provide a new perspective on designing robots2
Past FISO Robotics TopicsSurface Telerobotics (Tunstel
JHU/APL)Humans in the loop for flexibility and handling
contingenciesCo-Robots (Thangavelu USC)Innovative Robotic Systems
(Akin UMD)use robots to make humans as capable as possibleOn-orbit
Servicing: Telepresence(Artigas DLR)Humans in Orbit and
Tele-robotics (Pica SI/UH)Human and Robotic Integration (Marques
NASA ARC)Future human spaceflight will be more than developing
automation/robotic technology it will have to be about integrating
these technologies with people.Important gap : design
guidelinesEVA/Robotic Servicing (Akin UMD)
3
Why do we need a new approach? Function Allocation
(Fitts)characterize the general strengths and weaknesses of humans
and machinesSupervisory Control (Sheridan)a human oversees
autonomous systems, statically allocating tasks to them.Adjustable
Autonomy (Dorais)autonomous systems operate with dynamically
varying levels of independenceSliding Autonomy (Dias)Same as
adjustable autonomyAdaptive Automation (Sheridan)the system must
decide at runtime which functions to automateFlexible autonomy
(Technology horizons)the system can vary the degree of autonomy
from essentially none to fullMixed-initiative interaction (Allen)An
interaction strategy, where each agent can contribute what it does
bestCollaborative Control (Fong)Allows the human to close the
perceptual or cognitive loopsCognitive Task Analysis, Human Factors
and othersProvides an understanding of human needs, usability,
etc.Task AllocationDynamicTask AllocationBoth partiesNot just
taskallocationHuman side4
4
Why do we need a new approach?Autonomy is unidimensionalThe
conceptualization of levels of autonomy is a useful scientific
grounding for the development of autonomous system roadmapsAutonomy
is a widgetAutonomous systems are autonomousOnce achieved, full
autonomy obviates the need for human-machine collaborationAs
machines acquire more autonomy, they work as simple multipliers of
human capabilityFull autonomy is not only possible, but always
desirable
Bradshaw, J.M, Robert R. Hoffman, Matthew Johnson, and David D.
Woods. The Seven Deadly Myths of "Autonomous Systems. IEEE
Intelligent Systems, May/June 2013 (vol. 28 iss. 3), pp.
54-61.Johnson, M., Bradshaw, J.M., Hoffman, R. R., Feltovich, P.
J., and Woods, D. D. Seven Cardinal Virtues for Human-Machine
Teamwork: Examples from the DARPA Robotic Challenge. IEEE
Intelligent Systems, November/December 2014 (vol. 29 iss. 6), pp.
74-80Seven deadly myths of autonomy that engender a host of other
serious misconceptions and consequences.5
Why do we need a new approach?Task allocation leads to the
misconception that we are developing black boxes of control
6
6
Why do we need a new approach?Task allocation leads to the
misconception that we are developing black boxes of control7
7
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
8
8
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences Matter
9
9
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences Matter
10
10
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences MatterLevels Are Neither Ordinal nor
Representative of Value
11
11
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences MatterLevels Are Neither Ordinal nor
Representative of ValueAutonomy is Relative to the Context of the
Activity
12
12
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences MatterLevels Are Neither Ordinal nor
Representative of ValueAutonomy is Relative to the Context of the
ActivityLevels of Autonomy Encourage Reductive Thinking
13
13
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences MatterLevels Are Neither Ordinal nor
Representative of ValueAutonomy is Relative to the Context of the
ActivityLevels of Autonomy Encourage Reductive Thinking
14
14
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.
Functional Differences MatterLevels Are Neither Ordinal nor
Representative of ValueAutonomy is Relative to the Context of the
ActivityLevels of Autonomy Encourage Reductive ThinkingThe Levels
of Autonomy Concept Is Insufficient to Meet Future Challenges
15
15
Why do we need a new approach?Few of these approaches provide a
method or a comprehensive approach to determining requirements and
most are based on LOA.Functional Differences MatterLevels Are
Neither Ordinal nor Representative of ValueAutonomy is Relative to
the Context of the ActivityLevels of Autonomy Encourage Reductive
ThinkingThe Levels of Autonomy Concept Is Insufficient to Meet
Future ChallengesLevels Provide Insufficient Guidance to the
Designer
16
16
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
17
17
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
18
18
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
19Johnson, Matthew, J.M. Bradshaw, Paul J. Feltovich, Catholijn
Jonker, Birna van Riemsdijk, and Maarten Sierhuis. Autonomy and
Interdependence in Human-Agent-Robot Teams. IEEE Intelligent
Systems, March/April 2012 (vol. 27 iss. 2), pp. 43-51.
19
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
20
20
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
21
21
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
22
22
Why do we need a new approach?Focusing solely on autonomy
ignores issues that have plagued systems from delivering the
promised improvements in performance
23
23
Dependent Independent-Autonomy-Why do we need a new
approach?
24
24
Dependent Independent Interdependent-Autonomy--Teamwork-Why do
we need a new approach?
25
25
Dependent Independent Interdependent-Autonomy--Teamwork-Why do
we need a new approach?
26
26
Dependent Independent Interdependent-Autonomy--Teamwork-Why do
we need a new approach?27
27
Dependent Independent Interdependent-Autonomy--Teamwork-Why do
we need a new approach?28
28
Dependent Independent Interdependent-Autonomy--Teamwork-Why do
we need a new approach?by designing for interdependence.Coactive
Design is about enabling autonomy to reach its potential29
29
What is Coactive Design?Coactive Design is about designing
human-robot systems that support interdependence.
30
30
The Theory of Interdependence
31
31
Interdependence describes the set of complementary relationships
that two or more parties rely on to manage required or
opportunistic dependencies in joint activity
The Theory of Interdependence32
32
Interdependence describes the set of complementary relationships
that two or more parties rely on to manage required or
opportunistic dependencies in joint activity
involves both the control algorithms and the interfaceThe Theory
of Interdependence33
33
Interdependence describes the set of complementary relationships
that two or more parties rely on to manage required or
opportunistic dependencies in joint activity
Not limited to compensating for some missing ability or hard
constraints, but also involves normal supportive behavior
associated with teamwork or soft constraints.The Theory of
Interdependence34
34
The Theory of InterdependenceInterdependence describes the set
of complementary relationships that two or more parties rely on to
manage required or opportunistic dependencies in joint activityWhat
kind of relationships are used to manage dependencies in joint
activity?
35
35
36
36
(Sheridan & Verplank, 1978)37What is Coactive Design?
37
(Sheridan & Verplank, 1978)Autonomy and Control
TheoryInterface Design and Human Factors38What is Coactive
Design?
38
(Sheridan & Verplank, 1978)These are where the requirements
for supporting interdependence come from.Autonomy and Control
TheoryInterface Design and Human Factors39What is Coactive
Design?
39
What is Coactive Design?
(Sheridan & Verplank, 1978)These are where the requirements
for supporting interdependence come
from.ObservabilityPredictabilityDirectabilityAutonomy and Control
TheoryInterface Design and Human FactorsInterdependence40
40
What is Coactive Design?Coactive Design is about designing
human-robot systems that support interdependence.Coactive Design
highlights three key capabilities that are needed for effective
human-robot teamwork: observability, predictability and
directability (OPD).Coactive Design is a design method that helps
design support for interdependence and understand the impact of
change.
41
Identification process
Selection and implementation processEvaluation of change
process
42
42
What is Coactive Design?Coactive Design is about designing
human-robot systems that support interdependence.Coactive Design
highlights three key capabilities that are needed for effective
human-robot teamwork: observability, predictability and
directability (OPD).Coactive Design is a design method that helps
design support for interdependence and understand the impact of
change.
43Johnson, M., J.M. Bradshaw, P. J. Feltovich, C. M. Jonker, M.
B. van Riemsdijk, and M. Sierhuis. Coactive design: Designing
support for interdependence in joint activity. Journal of
Human-Robot Interaction, Vol. 3, No. 1, 2014, pp. 43-69.
Why is interdependence important?Because it provides specific
guidance to an engineer44
44
So, what does this mean to an engineer?improve the teaming of
unmanned systems with the manned force - Unmanned Systems
Integrated Roadmap FY2011-2036
The goal of the National Robotics Initiative is to accelerate
the development and use of robots in the United States that work
beside, or cooperatively with, people. and symbiotic
relationship... NRI 2013
human-system collaboration Defense Science Board 2012
combine the inherent strengths of manned platforms with the
strengths of UAS UAS Roadmap 2010-2035
shared-control concept the operator works in-the-loop with the
robot Roadmap for US Robotics 2009
45
45
Why is interdependence important?Because it provides specific
guidance to an engineerBecause it addresses the core issues of
integrating a robot into human activity
46
Core Issues in Robotics Providing SupportHuman NeedsWhat is the
robot doing?What is the robot going to do next?How can we get the
robot to do what we need?
47
47
Human NeedsIssuesWhat is the robot doing?ObservabilityWhat is
the robot going to do next?PredictabilityHow can we get the robot
to do what we need?Directability
Core Issues in Robotics Providing Support48
48
Human NeedsIssuesRobot NeedsWhat is the robot doing?Mutual
ObservabilityWhat is the intent of the human?What is the robot
going to do next?Mutual PredictabilityWhat does the human need from
me?
How can we get the robot to do what we need?Mutual
DirectabilityCan the human provide help?
Core Issues in Robotics Providing Support49
49
Why is interdependence important?Because it provides specific
guidance to an engineerBecause it addresses the core issues of
integrating a robot into human activityBecause it helps analyze a
design
50
51
51
Interdependence Analysis (IA) TableTeam Member Role
AlternativesPerformerSupporting Team MembersI can do it allMy
assistance could improve efficiencyI can do it all but my
reliability is < 100%My assistance could improve reliabilityI
can contribute but need assistanceMy assistance is requiredI cannot
do itI cannot provide assistance
52
52
Feasible Interdependence combinations
53
53
IA Table Example for Collaborative
ControlTasksHierarchicalSub-tasksRequiredCapacitiesTeam Member Role
AlternativesAlternative 1PerformerSupporting Team
MembersRobotHumanNavigationAvoid obstaclesSense obstaclesInterpret
if obstacle is passableDecide to avoid to proceed
observabilitydirectabilitypredictabilitynotification
54
54
Why is interdependence important?Because it provides specific
guidance to an engineerBecause it addresses the core issues of
integrating a robot into human activityBecause it helps analyze a
designBecause it helps determining how to allocate resources
55
56
56
There are many aspects of this task (e.g. perception, walking,
manipulation), so if we do not have infinite time, where should we
focus our efforts?57
57
Hose Task IA Table
58
58
Why is interdependence important?Because it provides specific
guidance to an engineerBecause it addresses the core issues of
integrating a robot into human activityBecause it helps analyze a
designBecause it helps determining how to allocate resourcesBecause
it helps understand the impact of change
59
59
It is common to want to automate parts of a given task, Lets say
we wanted to automate the task of picking the hose up off the
table. What would be the impact of doing this?60
60
Hose Task IA Table
61
61
Hose Task IA Table
62
62
How was interdependence beneficial?It helped design an effective
interface for the operator. 63
63
64
64
How was interdependence beneficial?It helped design an effective
interface for the operator. It helped ensure control algorithms had
appropriate hooks for OPD.65
66
66
Coactive Design
67
67
How was interdependence beneficial?It helped design an effective
interface for the operator.It helped ensure control algorithms had
appropriate hooks for OPD.It provided flexibility by identifying
alternative ways to accomplish the same task. 68
How was interdependence beneficial?69If you fail to plan, you
are planning to fail.-Benjamin Franklin In robotics, If you dont
plan to fail, you are failing to plan.-Matthew Johnson
70
How was interdependence beneficial?It helped design an effective
interface for the operator.It helped ensure control algorithms had
appropriate hooks for OPD.It provided flexibility by identifying
alternative ways to accomplish the same task. It helps us build a
resilient system, not based on flawless performance, but on the
ability to recognize and adapt to uncertainty, misfortune and
surprise.71
DRC Trials Results72
DRC Trials Results
73
Thank you! Any
Questions?http://www.ihmc.us/users/mjohnson/publications.html
74
