JohnsonSpaceCenter

Space Robotic CapabilitiesDavid Kortenkamp (NASA Johnson Space Center)

Liam Pedersen (NASA Ames)Trey Smith (Carnegie Mellon University)

Illah Nourbakhsh (Carnegie Mellon University)

David Wettergreen (Carnegie Mellon University)

Dan Clancy (NASA Ames)

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 2

!

Motivation

ScienceObjectives

MissionConcepts

Robots

Human/Robots

Robots

Human/Robots Human/Robots

Robots

Robot capability METRICS

Return all data Select targets Return selected dataCharacterize site

Recognize unforeseen scientific opportunities

Breakthrough10 year forecast

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 3

Methodology

• How do we measure the state-of-the-art in space robotic capabilities?– What is important?

• Functionalities, e.g., mobility, assembly– How do you measure it?

• State-of-the-art metrics (qualitative)• Performance metrics, e.g., distance traveled• Space readiness metrics

• What is the state of the art?– Flown robotic systems, e.g., Sojourner– Fielded robotic systems, e.g., Nomad– Laboratory demonstrations

• What is the future?– Projections, breakthroughs and roadmaps

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 4

Mission Scenarios

In-Space Missions

Planetary Surface Missions

Exploration

Work Operations

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 5

Surface reconnaissance

In-Space Assembly, Inspection, and Maintenance

Planetary Surface Exploration

In-depth site surveySample acquisition and analysis

Human exploration assistance

AssemblyInspection

Maintenance and repairHuman EVA assistance

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 6

Space Robotic Functionalities

• Derived from mission scenario requirements

• Provide means for organizing and evaluating various robotic technologies

• Deliberately limited:– Space robotics, not robotics

– Two mission scenarios

• Motivated by existing space robotics research

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 7

Mars Surface Exploration Functionalities

Mobility

Human Interaction

Science OperationsMobility Autonomy

Mobility Mechanism Sample Manipulation

Perception, Planning, Execution

Human-Robot Interaction

Terrain assessment, path planning, visual servoing

Extreme terrain access, energy efficiency

Tele-operation to human supervision; robot/EVA astronaut teams

On-board and ground tools; data analysis, target selection, operations planning and execution

Position sensors, collect and process samples

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 8

In-Space Assembly, Inspection, and Maintenance Functionalities

Assembling structures

Maintenance and Repair

Inspecting structuresTransporting and mating

Making connections

Locomotion

Component change-out

Accessing components

Move self and other massive elements; path planning, coverage patterns

Manipulate small objects and tools; hand-eye coordination; fine motion planning

Opening covers; removing blankets

Manipulation and sensing; grasping; turning bolts

Path planning to cover an area; visual servoing on an anomaly

Data analysisRecognizing and characterizing anomalies; taking appropriate action

Human EVA AssistanceMonitoring

TeamingPhysical interaction; sensing of human intention

Tracking crew members; video archiving

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 9

Metrics

• State-of-the-art metrics (qualitative)– Precise definitions– Generalize to many systems

• Performance measures (quantitative)– Resist temptation to use many easy to measure but uninformative numbers– Cannot be reported for some fielded systems, but will hopefully “set the

bar” for future reporting of results

• Space readiness metrics– Mass, power, size, computation, etc.

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 10

What is the current state-of-art?

• Provide a list of functionalities and metrics to rate our progress in that particular functionality

• Ask experts to “check off” the metric that corresponds to the state-of-the-art and the metric that will be the state-of-the-art in 10 years

• Experts currently being polled

• The following slides are preliminary assessments of the state-of-the-art that could change as we get more input

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 11

Surface Exploration Metrics

Surface reconnaissance

Planetary Surface Exploration

In-depth site surveySample acquisition and analysis

Human exploration assistance

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 12

Flight SOA Fielded SOA

10 year Forecast

10 m1 m 100 m 1000+ m

Surface Mobility Metric

Traverse distance per command cycle

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 13

Surface Mobility Metric

Autonomous mobility in terrain types

Flight SOA Fielded SOA

10 year Forecast

DunesLevel

ConsolidatedBoulder

Field Escarpment

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 14

Sample Approach and Instrument Placement Metric

Remote measurements

Simple surface contact measurements

Multiple targets in single cycle, highly robust

Precision surface contact measurements

Flight SOA Fielded SOA 10 year forecast

Command cycles / operation : MultipleMultiple Single Highly autonomous

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 15

Whole Sample Manipulation

Imprecise and unpredictable manipulation

Precise and predictable manipulation

Operate in complex environment w/ clutter, constraints and occlusions

BreakthroughFlight SOA

10 year forecast

Command cycles / operation : MultipleMultiple Single

Manipulate complex shapes

Highly autonomous

Example manipulators:

GripperScoops, clamshell

Dexterous gripper

Human hand

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 16

Onboard Science Perception and Science Plan Execution

None (tele-operation)

Time stamped sequence

Flexible time,contingencies

Prioritized task list with constraints

High level science goals

Return all data Select targets Return selected data Characterize site

10 years

Recognize unforeseen scientific opportunities

Breakthrough10 years

Execution:

Perception:

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 17

Human Exploration Assistance

Generic obstacle avoidance

Fielded SOA 10 year forecast

Sensing of humans

Tracking of humans Tracking of human body parts (i.e., gestures)

Recognition of humans and their activities

Simple, static gestures

Fielded SOA 10 year forecast

Dynamic gestures Hand signals Gestures linked to natural language

Gesturerecognition

Recognition of human physical and mental state

Breakthrough

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 18

State-of-the-art example

• EVA Robotic Assistant at NASA Johnson Space Center

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 19

In-Space Robotic Operations

• Assembly

• Inspection

• Maintenance

• Human EVA Assistance

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 20

In-Space Robotic Assembly

Teleoperated capture of fixed component

Flight SOA Fielded SOA10 year forecast

Payloadcapture

Autonomous capture of fixed component

Teleoperated capture of free-flying component

Autonomous capture of free-flying component

One or more basic elements

Flight SOA Fielded SOA10 year forecast

Multiple components and orientations

Large mass or flexible components

Complex assembly; gossamer components

Teleoperated mating of robot friendly connectors

Flight SOA Fielded SOA10 year forecast

Autonomous mating of robot friendly connectors

Teleoperated mating of EVA connectors

Autonomous mating of EVA connectors

Gross assembly

Mating connnectors

Autonomous mating of arbitrary connectors

Breakthrough

Breakthrough

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 21

State-of-the-art Example

• Skyworker from Carnegie Mellon University

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 22

State-of-the-art Example

• Dira from Carnegie Mellon University and NASA JSC

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 23

In-Space Robotic Maintenance

Open loop control

Flight SOA Fielded SOA 10 year forecast

Locating acomponent

Closed loop control using special markers

A priori model of undamaged component

A priori model of damaged component

Teleoperated grasping of robot friendly component

Flight SOA Fielded SOA10 year forecast

Autonomous grasping of robot friendly component

Teleoperated grasp of component w/ handle

Autonomous grasp of component w/handle

Grasping acomponent

Autonomous grasp of arbitrary component

Breakthrough

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 24

State-of-the-art Example

• Robonaut NASA Johnson Space Center Humanoid Robot

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 25

In-Space Inspection

Visual inspection of specific site -- teleop

Flight SOA Fielded SOA10 year forecast

Inspectingstructure

Visual inspection of a large area -- teleop

Visual inspection of a large area -- autonomous

Visual inspection of complex structure -- autonomous

No data analysis

Flight SOA Fielded SOA 10 year forecast

Moaicing of images Filtering of data Detecting modeled anomalies automatically

Analyzingdata

Detecting unmodeledanomalies automatically

Breakthrough

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 26

Human EVA Assistance

No commands --teleoperation

Flight SOA Fielded SOA 10 year forecast

Human-Robot Communication

Text-based commands Speech-based commands Multi-model interaction

Holding object forhuman

Flight SOA Fielded SOA 10 year forecast

Handing objects tohuman

Taking objects fromhuman

Carrying/rescuinghuman

PhysicalInteraction

Interactive dialogue

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 27

In-Space Assembly Overall Evaluation

Teleoperated robots that move large components and mate parts

Closely supervised, semi-autonomous robots that move large components and mate parts

Teleoperated robots that can mate parts and make fine connections between parts

Closely supervised, semi-autonomous robots that mate parts and make fine connections between parts

Autonomous robots that move large components and mate parts with minimal human intervention

Autonomous robots that mate parts and make fine connections between parts with minimal human intervention

Autonomous robots that perform complete assembly of complicated structure (e.g., large telescope) from start to finish with substantial support from ground-based or in-space humans

Autonomous robots that perform complete assembly of complicatedstructures (e.g., large telescope) from start to finish with minimal human intervention

JohnsonSpaceCenter

12/18/2001 Space Robotics State-of-Art 28

Conclusions

• Space-fielded robotic systems lag far behind the current state-of-the-art

• In-space assembly lags behind surface exploration– Not as much of an agency initiative

• Requirements for space robotics are growing– Planetary exploration

– In-space assembly of next generation space telescopes at the LaGrange points – little human capability