Cooltown User Studies
Margaret Fleck, Marcos Frid, Tim Kindberg, Eamonn O’Brien-Strain, Rakhi Rajani and Mirjana Spasojevic
Mobile Systems and Services Lab
Hewlett-Packard Labs
From Informing to Remembering: Deploying a Ubiquitous System in an Interactive Science Museum
Cooltown User Studies
Reality Check: User Studies What works with ordinary users? What is their experience like?
Cooltown User Studies
Study actual use: how to interleave electronic functionality with physical, nomadic activities
Inform design, stimulate design
Physical
Virtual
Web pages
Exhibits
Research Focus: Cooltown User Studies
Cooltown User Studies
Exploratorium – science museum
Exploratorium mission Public exhibits space Educational activities, teachers programs Center for media and communication-“dissolve
the walls of the building”
NSF funded Electronic Guidebook project Investigate use of handhelds, wireless networks Increase level of engagement and help learning Pre-visit, during and post-visit interaction Expand Exploratorium’s educational mission
Cooltown User Studies
Motivation
ElectronicGuidebook
Rich web based content
… stimulating… … exploration …
Cooltowntechnologies
Cooltown User Studies
Challenges
… hard to navigate …
… hands-on …
… vigorous, noisy …
Cooltown User Studies
Proposed functionalities
Informer Content delivered to the user next to the exhibit
Suggester/Guider Things to do while at the exhibit, things to see elsewhere
Communicator Communication among family members, field trip groups Experience sharing, virtual graffiti
Rememberer Help remember exhibits/phenomena, increase post-visit
engagement
Cooltown User Studies
Phase I
Prototype hybrid Electronic Guidebook: Informer, Suggester, Rememberer
Variety of technologies (handhelds, sensing,…) Diverse user population (staff, teachers,
students, …) User tasks (browse, bookmark, treasure hunt, …) Measure interactions with physical & virtual
resources (web logs, observers, videos, ..) Qualitative evaluation (interviews, surveys)
Cooltown User Studies
Infrastructure overview
802.11
Pi-station: beacon (infrared), barcode, RFIDplus processing, I/O and networking
contentserver
Internet
RFID IR
real exhibit
Cooltown User Studies
Electronic Guidebook Technologies
Jornada 690 Jornada 540 Hitachi E-Plate
barcodesbeacons
Cooltown User Studies
Electronic Guidebook: User Experience
Cooltown User Studies
User Experience: first step
beacon
URL
Web page
• location sensing: get exhibit URL• first web page automatically downloaded
Cooltown User Studies
User Experience: top page, follow to information nuggets
top page(image map)
info nuggets
Cooltown User Studies
Remembering content: personal scrapbook
press bookmark button to save web page capture experience and make it memorable
personal experience
personal online scrapbook
personal artifacts
postcard
collage poster
booklet
Cooltown User Studies
Phase I experiments (Summer 2001)
34 users: teachers, kids, staff, developers (ages 10 to 50+)
Data collected: detailed web logs, observations, interviews
Cooltown User Studies
Phase I key findings
Low-level & generic Verified feasibility of basic components
Beacons/barcodes acceptable for location sensing Wireless connectivity (802.11) Browser UI Web pages easily accessible Issues with stylus
Battery life will continue to be an issue Potential for getting lost in cyberspace!
Cooltown User Studies
Phase I key findings
Domain-specific (Exploratorium-related) Most appropriate user groups: teachers, explainers, repeat
visitors Easy adoption with younger visitors Personalization of content is important Audio needs earplugs Users reacted positively to the 'my scrapbook' idea Potential negative effects
distraction from exhibits by on-line content carrying devices affects interaction at the exhibits
Ongoing: evaluation of content for self-directed informal learning
Cooltown User Studies
Choosing a tool
Informer, Suggester Too obtrusive for many visitors
Guider, Communicator Less frequent distractions
Rememberer Minimizes attention shifts Extends engagement to post-
visit User interest
Physical
VirtualWeb pages
Exhibits
Cooltown User Studies
What is Rememberer?
• Record of the exhibits to remember. User controls what is captured.
• Components
• "remember-this" technology for selecting objects during the visit
• the visit record (set of web pages)
• a physical token, which reminds the user of the visit
Cooltown User Studies
Phase II infrastructure: Rememberer
802.11
contentserver
Internet
RFID IR
real exhibit
“remember-this”
personal artifact
(web page, postcard, etc.)
Cooltown User Studies
Rememberer: research questions
Interference with exhibit manipulation? Does the tool help with recall? Influence on discussion and social
interaction among people who visited together with people who are remote
Does the tool help as a resource? in itself as a launching point for further exploration
Cooltown User Studies
Rememberer: experiments
Round 1: informally verified basic concept RFID tags, cameras, web pages for photos Users like having photos taken this way Web pages accessed after the visit and annotated
Round 2: detailed evaluation (23 individuals/groups)
Cooltown User Studies
Round 2: experiment description
Experiment setup Mixture of exhibits; non-instrumented, with
beacon, with beacon & camera Individuals and small groups Control: no device, all exhibits non-instrumented Observations & interviews
Diverse Exploratorium-associated volunteers (!) Kids and adults, male & female
Cooltown User Studies
Use of ‘remember-this’
PropensityAll instrumented exhibits - 74%Exhibits w/ beacons - 67%Exhibits w/ beacons and camera - 86%
‘Interference’Beacon pickup: 10 out of 17 said “easy or very easy”Observed beacon problems: 10% of 140 usesCamera adjustment: 25% of 63 uses50% said wanted more camera control (what/when)
Cooltown User Studies
Time spent at exhibits
No instrumentation
1 min
5 sec
Beacons 1 min
49 sec
Beacons w/ camera
2 min
3 sec
All exhibits 1 min 29
sec
Control group 1 min 40 sec
Time spent at exhibits
0
5
10
15
20
25
30
35
40
45
50
< 1 1-2 2-3 3-4 4-5 5-6
minutes spent at an exhibits
% o
f vi
sits
All exhibits
No instrumentation
Beacons
Beacons w/ camera
Control group
Camera: visits 20% longer than control group
Cooltown User Studies
Home use
10 out of 17 revisited web pages 4 twice 3 of them about 10 days after the experiment
Cooltown User Studies
Moving forward
Continue analysis of Rememberer at Exploratorium
More widely applicable ‘remember-this’ e.g. Jornada with camera
Exploratorium Feb ‘02
Electronic Guidebook Rememberer
Multiple domains
Cooltown User Studies
Conclusions
Nomadic computing tools – simple applications that enhance interaction with the physical world
Cooltown physical hyperlinks are an effective mechanism for invoking web-based services
Exploratorium findings are likely to apply to other domains that are dynamic with high demand on user attention
More information
Electronic Guidebook project information
www.exploratorium.edu/guidebook/
Most recent TR: HPL-2002-54
Cooltown User Studies
Backup slides
Cooltown User Studies
Sample content: orientation page
Cooltown User Studies
Sample content: exhibit instructions, information nuggets
Cooltown User Studies
PhysicalVirtual
Web pages
Exhibits
Research focus: Physical vs. Virtual navigation
Cooltown User Studies
Phenomena under investigation
Basic affordances(handhelds, beacons, web pages, audio, …)
Attention to artifacts(exhibits, online content, …)
Paths through physical and virtual space(attention switching: exhibits, devices, content, companions, …)
Higher order effects (informal learning, engagement, social interaction, …)
com
ple
xit
y