Information Enhancement of Synthetic Environments

Nicholas F. PolysVirginia Bioinformatics Institute

Virginia Tech Computer Science &

Center for Human Computer Interaction

Presentation

• Introduction

• HCI & Usability Engineering

• Integrated Information Spaces

• Research:– CML -> X3D– CML -> Snap + Diverse– Biological Simulation in PathSim

What is HCI?

• A multidisciplinary science of the interface: psychology, design & media, human factors, sociology, computer science

• Experimental methods to rationalize UI features, design, and software architecture

Usability Engineering

Where the rubber meets the road…• Scenario-Based Design:

– Activities– Information– Interaction– Claims analysis

See:ROSSON, M.B. AND CARROLL, J. 2002. Usability Engineering:

Scenario based development of Human-Computer Interaction. NY,

Morgan Kaufmann.

Problem: Integrated Information SpacesResearchers, designers, students, and analysts

need to explore, search, recognize, and compare patterns across a wide variety of data types and applications.

Some representations are better suited to different data types and questions…

Fundamental Information Types

– Abstract information • -> Information Visualizations

– Perceptual or spatial information • -> Virtual Environments

– Temporal Information

User Problem:

• Perceptual, environmental phenomena alone do not guarantee accurate mental models

• Interfaces for information and controls are consistently under-represented and poorly integrated in VEs

Developer Problem:

• Increasingly, objects rendered in VEs have metadata and annotations associated with them. These may change over time.

• Lack of design guidelines and information architectures for efficient management of Integrated Information Spaces

Approach: Information-Rich Virtual Environments (IRVEs)

IRVEs are at the union of VE and Info Vis:perceptual + abstract information is simultaneously accessible and linked

• Define the issues and the task space for this class of applications

• Define the design space to solve and support them• Apply usability engineering methods to identify

tradeoffs and enumerate design guidelines

Some Research Questions:

Where, how, and when should the abstract information be displayed in the VE?

In what form should the information be displayed?

How can interrelationships between the spatial and abstract information be represented?

IRVE Activity Design

See also : Bowman, Doug, North, Chris, Chen, J., Polys, N., Pyla, P., Yilmaz, U., (2003) “Information-Rich Virtual Environments: Theory, Tools, and Research Agenda”. Proc. Virtual Reality Software and Technology (VRST)

Activity Task Example

Exploring Overview data loading, choosing the data and display method

Finding Details-on-demand

pop-ups, queries, lenses, zooming via 3D navigation etc.

Pattern Recognition

Relating, Comparing

rendering associations; brushing and linking

Task-Knowledge Structure

e.g. for Situational Awareness and decision support systems…

[Entity Relationship diagrams per task and media]

• SUTCLIFFE, A. AND FARADAY, P. 1994. “Designing Presentation in Multimedia Interfaces”, Proc. CHI '94, 92-98.

Research Thrust:

Data Models behind IRVEs

CML via XSLT

CML via XSLT

CML is a good example of an IRVE database

POLYS, NICHOLAS F. 2003. “Stylesheet Transformations for Interactive Visualization: Towards a Web3D Chemistry Curricula”, Proceedings of the Web3D 2003 Symposium, ACM SIGGRAPH.

http://www.3DeZ.net/X3D/CML

Research Thrust:

• Interactions and Architectures for IRVEs

Snap2Diverse: Coordinating Information Visualizations and Virtual Environments

• DIVERSE is an multi-platform open-source VR toolkit [diverse.sourceforge.net]

• Snap-Together Visualization is a free toolkit for information visualization [infovis.cs.vt.edu]

Multiple, Coordinated Views for IRVEs

• Users can index to and from perceptual and abstract information through ‘brushing and linking’– Diverse as a Snap-able component– Unique Ids– Event communication (e.g. select, load)

Snap Coordination

Manager

É

Component 1

CAVE Adapter (RMI Messaging

Events)

DIVERSEAtomView application

Java Component

Adapter Component 2

System overview

Application: Cheminformatics

Snap2Diverse Video

POLYS, N. F., NORTH, C., BOWMAN, D. A., RAY, A., MOLDENHAUER, M., AND DANDEKAR, C. 2004. “Snap2Diverse: Coordinating Information Visualizations and Virtual Environments”.

Proc. SPIE Conference on Visualization and Data Analysis.

http://csgrad.cs.vt.edu/~npolys

Usability Evaluation Summary

• Snap2Diverse as tested is:– high-threshold for learnability– high-ceiling for functionality

• Users were able to brush&link to solve tasks• In most cases, users chose suitable visualizations to

recover the information required for the finding and comparing tasks

• Given a choice, users tended to recover information from the VE instead of InfoVis

• Xwand is a moded interface and switching between 2D and 3D navigation and selection was not intuitive.

Lessons Learned

Simple events provide ‘glue’ to support complex functionality:

• Benefits of exposing VE scenegraphs to external events; enabling technology must be extended

• Benefits of composable and integrated information spaces

• Designers must consider tasks and the knowledge required for completion (ie Sutcliffe and Faraday, Shneiderman)

Research Thrust:

Objects and Patterns for IRVEs • X3D Event Utilities

• MFSequencers

• Information Panels

• Semantic Objects

• HUDs

PathSim Demo

http://www.vbi.vt.edu/~pathsim

Paper to be presented at Web3D 2004:

Polys, N., Bowman, D., North, C., Laubenbacher, R., Duca, K., (2004). “PathSim Visualizer: An Information-Rich Virtual Environment for Systems Biology” Proceedings of Web3D 2004, ACM SIGGRAPH.

Summary

• There is an increasing demand for information-rich functionality across a wide variety of domains

• A systematic HCI approach to design problems of Integrated Information Spaces is required

• Goal is to understand tradeoffs of design and software architectures to build better interfaces

Future Work

• IRVE Testbed construction• Controlled experiments and usability

evaluations to identify the significant design dimensions and techniques for integrated information spaces.

• Identify tradeoffs, guidelines, and design patterns for this class of problems.

• Propose future X3D standard components:– Annotation (labels, metadata)– Compositing (HUD, ApplicationTexture)

Thanks and happy hacking!

npolys@vt.edu