Collaborative Expedition Workshop, NSF, Aug. 19, 2008

Deploying a Lightweight, Context-LinkedEvent Notification System

for a Distributed Scientific Team

Cecilia R. Aragon, Ph.D.Computational Research Division

Lawrence Berkeley National Laboratory

2

Collaborators

• Sarah Poon, Rollin Thomas, Brian Lee;• Nearby Supernova Factory collaboration:

– US:• G. Aldering, C. Aragon, S. Bailey, S. Bongard, M. Childress,

S. Loken, P. Nugent, S. Perlmutter, K. Runge, R. Thomas, B. Weaver (LBNL)

• R. Kessler (Chicago)• C. Baltay, N. Ellman, D. Rabinowitz, R. Scalzo (Yale)

– France:• C. Buton, Y. Copin, E. Gangler, G. Smadja (IPNL, Lyon)• P. Antilogus, J. Li, R. Pain, R. Pereira, C. Wu (LPNHE, Paris)• E. Pecontal, G. Rigaudier (CRAL, Lyon)

3

Talk Outline

• Motivation and challenges• Proposed solution: context-linked tools• Astrophysics case study• Science background• Telescope operational challenges• Design and implementation of “Bert”• User evaluation• Discussion and conclusions

4

Motivation

• Distributed teams need to analyze increasingly vast and complex data streams– Often working under time pressure– Examples: first responders, pilots, air traffic

controllers, hospital staff, astrophysicists operating large telescopes

• Software tools can facilitate collaborative decision-making

• Lightweight tools needed– low cost– low barriers to entry

5

Challenges in DevelopingCollaborative Tools

• Difficulty of gaining user acceptance of custom tools

• Cost• Event notification must be appropriately

but not overly intrusive– Especially in time-critical, distributed “control

room” situations• Need to integrate with existing

environment

6

Proposed Solution: Context-Linked Tools

• Aviation analogy: “scene-linked” symbology on head-up displays improves pilot performance and situation awareness [McCann 95, Aragon 05]

• Context-linked tools: both task and context information included in shared communication space– Task: immediately pertinent to the job– Context: background processes, events in

environment

7

Lightweight, Context-LinkedEvent Notification System

• “Bert” - virtual assistant/event notification system

• Developed for Nearby Supernova Factory– international astrophysics collaboration– largest data volume supernova search currently

in operation– processes 50GB of image data per night– time-critical telescope operation

• Integrated with instant messaging (“chat”) client already used by collaboration for communication

8

Science Background: Supernovae

• Stellar explosions• Can be used to measure

the expansion rate of theuniverse

• Answer fundamentalquestions about theuniverse

9

• Stellar explosions appearing as bright spots near galaxies• Rare: 1-2 per millennium• Random and fleeting: wax and wane within several weeks

Type Ia Supernovae

[slide from R. Romano]

10

Fundamental Questions about the Universe

• Einstein 1915: Why doesn’t gravity cause the universe to collapse?– The cosmological constant Λ

• Hubble 1929: The universe is expanding, no need for Λ– Next question: will the universe ever contract?– Need to measure the deceleration rate

• Riess, Perlmutter 1998: The rate of expansion is accelerating

• Question: Why?

11

Universe Composition

12

The Nearby Supernova Factory

An experiment to collect data on more Type Ia supernovae than have ever been studied in a single project before and answer some fundamental cosmological questions about the nature of the universe and “dark energy.”

13

SNfactory supernova searchdata flow

[slide by S. Bailey]

14

Telescope Operational Challenges

• Astronomers must monitor complex, dynamic operational data streams

• Multi-million dollar telescope with custom hardware– Equipment failures occur (e.g. telescope gets stuck, filters or

cooling systems fail)• Telescope must be operated within constraints or damage

may occur– sunrise, maneuvering limitations

• Located on remote mountaintops– adverse weather may require immediate closing – hypoxia– night operations

• Critical decisions may need to be made rapidly– in as little as 45 seconds

15

Telescope Remote Operation (VNC)

16

Further Challenges

• Telescope time is expensive ($4K/night)• Supernovae are rare and fleeting

– occur about once per millennium in a typical galaxy

– visible for a few weeks• Optimal schedule necessary for maximum

scientific benefit• Unpredictable variations in schedule

caused by:– upper air turbulence, fog, changing cloud cover

17

Cross-Cultural Collaboration

• Nearby Supernova Factory– about 30 collaborators, half in U.S., half in France

• Telescope in Hawaii typically remotely operated by two to six people on a given night, across geographically separated time zones (US west/east coast, France)

• Correct decisions must be made collaboratively– some team members have never met– cultural differences, language differences– unfamiliarity with telescope interface– diurnal rhythms, environmental cues are opposite to

telescope environment (up to 11 hour time difference)

18

Bert

• Lightweight virtual assistant in the chat developed to assist with time-critical science tasks and general awareness

• Two main functions– announces relevant events, e.g. completed

exposures, number of minutes till sunrise– responds to user queries for information

• In operational use since 2005

19

Previous Work

• Scientific collaboratories– Wulf’s “center without walls” [Wulf 93]– Upper Atmospheric Research Collaboratory (UARC) [Finholt

95, Olson 01]• The use of chat

– Grounding and orienting tool– Keep distributed members aware of status at physical site

[Bimholtz 05]– Trouble shooting, answering technical questions– Tickertape CVS notifications for distributed software

development team [Fitzpatrick 06]– Persistent chat benefits [Erickson 99, Geyer 04, Halverson 04,

Fono 06]

20

Bert Implementation

• Written in Perl, uses standard CPAN modules (Comprehensive Perl Archive Network)

• Communicates with chat network via OSCAR protocol employed by AOL Instant Messenger system

• At start-up, Bert connects to chat network, finds preset chat room

• Forks a listener process that accepts connections from processes on the summit machine and forwards them to Bert

21

Bert Implementation (cont’d)

• Event notification– Programs send messages to Bert via simple

Unix file socket– Unix “at” command used to trigger a Bert

message at pre-specified time

22

Bert Implementation (cont’d)

• User commands to Bert– Users type Bert’s name in a chat message– Primitive syntax

• “bert, where is the moon?”• “bert, what is humidity?”• “bert, where is the telescope?”• “bert, when is sunrise”

– Get help from telescope observing experts (sends SMS (text) message)

• “bert, SOS [name]”

23

Bert Implementation (cont’d)

• User interface– Cross-cultural user-centered design– Greetings– Eliza [Weizenbaum 66]

24

Bert Implementation (cont’d)

• Speech synthesis (iChatter)• Integration with telescope automated

instrument control system (AIC)• Logging of chats for training purposes

25

User Evaluation: Methodology

• Analysis of chat logs– Three years of logs– 449,684 comments (users, Bert, system)– Bert comments: 145,667 or about 32%of total– Focused on April - December 2007 logs

(current state of operations)• 21 unique users• 150,644 comments

• Interviews with four observers (“shifters”)– Three have been observing since before Bert’s

integration

26

User Evaluation

• Synchronous and asynchronous uses of chat• Comparison with pre-Bert chat logs

– typical problem: confusion about sunriseArt: So sunrise is at 6:00 am. And now it’s 5.Art: We are on spectrum 2 of 4, about 1/2 way through, which is 2500 s

left.John: No, it’s 4Art: Eh? Hawaii is 2 h behind...Art: It’s 7 hereJohn: read director ...Art: Wow I am really just crazy.John: yes ...Art: I have no sense of time or distance.

27

Categories of Synchronous Bert Use

1. Collective Decision MakingStef: Rene, did the sky spectra of the other day been useful?Rene: I haven’t looked at it yet StefStef: okRene: are you wondering if it would be useful to take an other one?Stef: yesRene: I would say yes if we have time for it and if it is before 18 deg twilightStef: bert when is sunriseBERT: 18deg twilight is at 2007-04-13 14:51:00 UTC (in 30 min).BERT: 12deg twilight is at 2007-04-13 15:17:00 UTC (in 56 min).BERT: Sunrise is at 2007-04-13 15:57:00 UTC (in 96 min).Stef: yepRene: goodRene: lets’ go for it then

28

Categories of Synchronous Bert Use (cont’d)

2. Collective Trouble ShootingBERT: aic: ERROR: command [ point_object ] returned an error.BERT: aic: ERROR: event 18 terminating on fatal error at 2007-05-03 10:40:22 UTCChi: yesChi: Stef, can you log in?Stef: yes coming inChi: seen Fatal errorStef: ok I am inChi: don't match?Stef: weird: there are starsStef: just try it againChi: OK BERT: aic: New aic session started 2007-05-03 10:43:13 UTC with schedule file

aicSched/sch.070503.tcl.BERT: aic: Starting event 18 (SNF20070326-012, type Supernova)Stef: should be far from moonStef: Dec is off by 4 degreesStef: 4 minutesChi: yes Stef: that's a lotStef: I think telescope is stuck

29

Categories of Synchronous Bert Use (cont’d)

3. Keeping Track of Events– short responses to Bert’s announcements

4. Analysis Discussions– Bert announcements lead to focused discussion– 45-second time limit to make determination whether

or not to take supernova spectrumBERT: Prepare for check_match target confirmation, be prepared with stop_script!Chi: okDom: don't worry, you get another window just like it in Analysis.Chi: nothing in ...Dom: bummerChi: i stop !

5. Humor

30

Asynchronous Uses of Bert

1. Orientation– Bert announcements provide context

2. Training/Increasing Group Knowledge– Senior scientists review chat logs and email

feedback– Experts at first did not realize the steep learning

curve required of non-experts and beginner shifters– Scientists in U.S. could not monitor chat all night -

used logs to unmask training problems– Informed design and development of groupware

31

Context-Linked Information

• Group awareness of:– task information, e.g. AIC notifications– context information, e.g. time until sunrise

32

Discussion and Benefits

• Context-linked information aids in communication and collaborative work– Provides continuous environmental data– Informs awareness of time, schedule– Crucial in coordinating time-critical tasks– Reduction of coordination efforts to establish

common ground (less time spent figuring out state of tasks, beneficial when working under time pressure)

• Lightweight, easy-to-use– Did not require elaborate setup

33

Context-Linked Information

• Aids time-critical work– facilitates task prioritization (environmental state notices

inform on task urgency)– easy to ask for help or information w/o context switch– voice synthesis avoids overloading user’s visual bandwidth

• Presented within persistent, shared communication space– enables rich, asynchronous analysis of shared work– chat communication reveals shifters’ frame of mind– Bert’s notifications allow link with actual events and work

products

34

Further Work

• Detect degree of shifter’s cognitive overload and interruptibility– adaptive verbosity for Bert

• More in-depth analysis of half a million lines of chat logs

• Controlled studies on the effect of context-linked information in distributed teams

35

Acknowledgments

We would like to thank the scientists of the SNfactory collaboration for their time and detailed feedback. The authors wish to recognize and acknowledge the very significant cultural role and reverence that the summit of Mauna Kea has always had within the indigenous Hawaiian community. We are most fortunate to have the opportunity to conduct observations from this mountain. This work was supported in part by the Director, Office of Science, Office of Advanced Scientific Computing Research, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, and by the Director, Office of Science, Office of High Energy Physics, of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, and by a grant from the Gordon & Betty Moore Foundation. This research used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the U.S. Department of Energy under Contract

No. DE-AC02-05CH11231.

Collaborative Expedition Workshop, NSF, Aug. 19, 2008

Questions?

Cecilia R. AragonCRAragon@lbl.gov