Science Activity Planner User Interface Redesign Ben Glenn Alesha Jordan Jenica Rangos Angela Wagner Darin Wonn August 4th, 2004 A Joint Project of NASA

Science Activity PlannerUser Interface Redesign

Ben GlennAlesha JordanJenica RangosAngela WagnerDarin Wonn

August 4th, 2004

A Joint Project ofNASA Ames Human Computer Interaction GroupCarnegie Mellon Human Computer Interaction InstituteJet Propulsion Laboratory

Agenda

• MER Mission Overview• NASA’s Science Planning Tools• Project Scope & Design Challenges• Preliminary Research Findings• Science Planning Process Redesign• Design Methodology & Rationale• Prototype Demonstration &

Interaction Walkthroughs• Conclusions & Recommendations

MER Mission Overview

• Mars Exploration Rovers: Spirit and Opportunity

• Mission goal: Search for signs of water on Mars

• Rovers were designed for a 90 sol primary mission, but are still going strong after over 200 sols on the surface of Mars

• MER mission costs around $4M per day to operate

NASA’s Science Planning ToolsPrimary applications used in planning• Science Activity Planner (SAP)

– Downlink: data analysis– Uplink: activity planning

• Constraint Editor (CE)– Browser app for setting constraints

• MER Activity Plan Generator (MAPGEN)– Automated optimized scheduler

ScienceActivityPlanner

ScienceActivityPlanner

ConstraintEditor

ConstraintEditor MAPGENMAPGEN

Science Activity Planner (SAP)

Constraint Editor (CE)

MER Activity Plan Generator (MAPGEN)

Project Scope

Specify a tool for NASA scientists to track and review science data, and create and constrain rover activities in an immersive environment. The tool must capture and convey the scientists’ intent throughout the planning cycle.

Project Scope

What does this really mean?Design an application that supports data analysis, activity planning, and constraint editing across all stages of the mission.

SAPDownlink

(data)

SAP Uplink

(planning)

ConstraintEditorSAP 2.0

Design Challenges

• Support distributed collaboration• Support multiple mission stages

– Primary vs. Extended– Templatized vs. Non-templatized

• “Immersive” planning environment– Navigation through time and space– Visual representation of activities– Control visual clutter

• Provide round-trip data tracking• Integrate Constraint Editor features

Process Overview

& Prototype

Preliminary Research

Initial focus: Mixed-initiative planning• In-situ observations of NASA

scientists during MER primary mission– Two trips to JPL (Pasadena)– Observed six planning cycles

• Contextual inquiries and modeling of “analogous environments”– PAT Transit– Taco Bell– LITA (CMU autonomous rover

project)

Preliminary Research

Initial focus: Mixed-initiative planning

• Think-alouds using planning software– Microsoft Project (8 users)– Miss Bride Wedding Planner (6

users)

• Literature review (30+ papers)• Expert interviews (3 experts)• Competitive analysis (15+

systems)

FocusFocus

Mixed initiative planning overview

SystemModel

UserData

SystemConstraints

ActivityConstraints

Set Goals & Priorities

GeneratePlan

Entry into Planner

ModifyPlan

FinalizePlan

Iterate

SAPSAP

ConstraintEditor

ConstraintEditor

MAPGENMAPGEN

Root Cause #3Human can’t convey

intent to system

Science Planning Process Redesign

Preliminary design focus: Specify a science planning process to address breakdowns in current planning process and to support our assumptions about MSL ‘09

– Distributed collaboration– Multiple mission phases

User Interface DesignMethodology & Rationale

Design Methodology

Protocols Tools Users

Design Cycle 0Process validation, multi-phase mission definition/analysis

Contextual inquiry, In-situ observation

Flow diagrams 12 users• 2 Tactical Act Planners• 4 Payload Uplink Leads• 2 Geologists• 2 Mineralogists• 2 Rover Planners

Design Cycle 1Data-tracking, constraint/plan visualizations

Collaborative design

Paper prototypes

5 users• 2 Payload Uplink Leads• 2 Geologists• 1 Long Term Planner

Design Cycle 2Interaction modes, screen elements, planning processes

Think-aloud testing, Collaborative design

VB prototype 7 users• 4 Payload Uplink Leads• 1 Geologist• 2 Systems Planners

Design Cycle 3Immersive planning, spatial navigation, activity creation

Think-aloud testing, Collaborative design

VB prototype,paper prototypes

6 users• 4 Geologists• 1 Mineralogist• 1 Atmospheric

Protocols, Tools, Users

Interface Components

Interface Design: Mission Library

Key Features• Complete record of mission

plans– Executed & unexecuted activities– Supports reuse of prior activities

• Quick access to data products– Round-trip data tracking (status)– Thumbnails for visual scanning– Easily export to external apps

• Filters allow user to display only the activities and data products relevant to their immediate task


Open Data Product

Export Data Product

View Details

Copy to Editor

Palette Menu

Thumbnail

Open Info Panel

Status Indicator

Filter Turnbuckle


Reset filters to start state

Filter criteria


Design Cycle 1

Test Results• All users identified a need to easily access

mission data• 4 out of 5 users found filter feature useful• 3 users suggested replacing Science Group

filter with another filter• 2 users didn’t understand “Show All” label

Test Results• 5 of 7 users found the filter behavior intuitive• 3 users suggested replacing Target Type

filter with another filter• 4 users wanted ability to filter by Site• 3 users requested ability to export data to an

external application


Design Cycle 2

Test Results• Users continued to express concerns

about screen real estate, prompting creation of turnbuckle for hiding filters

• Specific behavior of View/Export buttons requires further testing


Design Cycle 3

Interface Design: Hopper

Key Features• Central queue for distributed,

collaborative activity planning• Supports work-flow

management, conflict prevention/resolution– Scientists “check-out” assigned

activities to their Editor to do detailed parameter specification

– “Quick Edit” button immediately checks out and opens an activity for editing


Key Features (continued)• Persistent planning

– Unexecuted activities persist in the Hopper until they “expire” due to certain constraints (i.e. rover moves away from target)

– Activities that are designated as repeating or systematic persist in the Hopper and are automatically re-submitted to the planner


Key Features (continued)• Long-range planning

– Hopper provides look-ahead– Scientists can begin planning

activities days in advance

• Intelligent ordering– Items in Hopper are sequenced

according to ordering constraints

• Visual constraint representation– Visualizations may be toggled

on/off to control clutter


Quick Edit

Check-out to Editor

Palette Menu

Ordering Constraint Visualization

Open Info Panel

Constraint Indicator

View Details

Toggle Constraint Visualizations

Lock indicates item is checked out by another user

Interface Design: HopperDesign Cycle 1

Test Results• 5 of 7 users recognized need for a Hopper;

2 were “enthusiastic”• 3 users didn’t understand constraint arrows;

4 expressed concerns about the scalability of this representation


Test Results• Lingering confusion about the Hopper-

Editor distinction motivated removal of all editing capabilities from Hopper

• Approximately half of the users expressed concern that the check-out process was an extra step that would slow them down


Test Results• 5 of 6 users instinctively used drag-and-

drop to check activities out of the Hopper• 3 users suggested there be a visual

indicator that activities are checked-out

Interface Design: Editor

Key Features• Private workspace for editing

and creating activities– Activities checked-out of

Hopper are edited in the Editor– Activities are built from scratch

with the toolbar or reused by copying from Mission Library or Template Library

• Visual constraint representation, intelligent ordering (as in Hopper)

Interface Design: Editor

New Activity

Delete Item

Palette Menu

Open Info Panel

Constraint Indicator

View Details

Toggle Constraint Visualizations

Indicates item is checked out of the Hopper

Check-in to Hopper

New Observation

Interface Design: EditorDesign Cycle 1

Design Cycle 2 Design Cycle 3

Interface Design: Palettes

Key Features• User-customizable

– Palettes may be resized, docked

– Palettes can be combined with other palettes to preserve screen real estate

• Palettes support drag-and-drop– Copy activities from Mission

Library and Template Library– Check activities into or out of

the Hopper

Interface Design: Palettes

Test Results• Users consistently requested ability to

create custom views of their workspace• Screen real estate a concern but felt it

would be mitigated through resizable palettes (and multiple monitors)

• Strong support for drop-and-drop functionality across all design cycles

Design Cycle 2 Design Cycle 3

Details Panel Widget

“Immersive” Planning Environment

Key Features• Interactive space that allows

scientists to plan activities and retrieve data in-context

• Four primary components– Viewer window– Navigator palette– View Control palette– Activity toolbar

• User data for these components will be presented after UI spec

Interface Design: Viewer Window

Key Features• Image data is overlaid on a 3d

terrain model and displayed as a navigable space

• User can pan, rotate, and zoom using the mouse– Cursor changes to indicate

action– Interaction mechanism similar

to QuickTimeVR

• Multiple views can be opened in the main window

Interface Design: Navigator

Key Features• Overhead view of Mars

provides spatial context to scientists

• Targets represent locations with navigable image data– Selecting a target changes the

Viewer window to that location

• Slider for adjusting map scale• Filter for specifying

classification of data products to be displayed

Interface Design: Navigator

Filter Menu

Palette Menu

Selected Target

Unselected Target

Scale Slider

Interface Design: View Control

Key Features• Controls azimuth (direction),

elevation and zoom of current view in the Viewer window

• Visually indicates the rover’s orientation when the navigable data product was captured

Interface Design: View Control

Palette Menu

Zoom Slider

Elevation Control

Azimuth Control

Rover Orientation

Interface Design: Activity Toolbar

Key Features• Tools for selecting, panning,

zooming, and targeting• Instrument tools for building

activities in the interactive planning environment

Interface Design: Activity Toolbar

Target Tool

Zoom Tool

Camera Instruments

Select Tool

Imagers and Spectrometers

Pan Tool

Drive, arm, and magnet activities

“Immersive” vs. “Non-Immersive”

• The two interfaces complement each other– Palettes are used to toggle items

on/off in the “immersive” space, reducing visual clutter

– Select-state is consistent across both interfaces

• Some tasks are better suited to a particular interface– Target creation in immersive UI– Constraint editing using palettes


Controlling the immersive view using the palettes


Spatially locating an activity via the Editor palette

“Immersive” Planning Environment

Test Results• 6 of 6 users liked the Photoshop “eyeball”

method for toggling visibility of activities• 5 of 6 users understood color-coded palettes• Users performed unexpected drag-n-drop

actions between immersive/non-immersive

This green star was intended to indicate the location of a data productAll 6 users misinterpreted the green star to be a clickable target

This red arrow indicates a data product is available off-screenHalf the users thought the arrow showed the rover’s orientation

Prototype DemonstrationInteraction Walkthroughs

Prototype Overview

• Used for late-stage user testing• Developed in Visual Basic .NET• Functioning features

– Dynamic display of data architecture

– Activity reuse from Mission Library– Checking activities in/out of Hopper– Creating new observations/activities

in Editor– Navigation between Martian

locations via the Navigator

Interaction Diagrams

Storyboards illustrating interactions not fully implemented in prototype

– Activity Representation – Target Creation– Targeted Activity Creation– Non-Targeted Activity Creation– Navigation

Conclusions and RecommendationsAddressing our design challenges…• Supported distributed collaboration

– via Hopper workflow features

• Supported multiple mission stages– via flexible planning process

• Provided round-trip data tracking– via Mission Library data filters

• Integrated Constraint Editor features– via Activity Details constraints tab

• New immersive planning environment– designed and tested navigation

techniques– developed visual language for activities– managed clutter via palette view toggles

Conclusions and RecommendationsAreas for future exploration…• Multiple Hopper views to visualize ordering

and time-of-Sol constraints• Constraint specification in collaborative

spaces• Resource budgets and visualizations• Further testing and refinement of

“immersive” planning and navigation techniques

A sneak preview of the ’09 mission…

Documents

Science Activity Planner User Interface Redesign Ben Glenn Alesha Jordan Jenica Rangos Angela Wagner Darin Wonn August 4th, 2004 A Joint Project of NASA