<Insert Picture Here>
Comparing Information Graphics: A Critical Look at Eye Tracking
Joe Goldberg and Jon Helfman
BELIV ‘10 – 4/10/10
Introduction
• How to select the most appropriate graph type?
• Consider three graphs, showing identical data. Can eye tracking methods help determine which graph type(s) • Are easiest to comprehend?• Are most appropriate for tasks?• Can be compared efficiently?
• Eye tracking may potentially reveal trends that are not easily observed by traditional usability metrics...but, eye tracking brings its own set of challenges
Eye Tracking Challenges: Practice-Related
• When is Eye Tracking Appropriate, and What Can it Tell Us?
• Designing and Conducting Studies• Length/number of tasks?• Allow mouse? Voice?• Recruitment profile? Number?• Allow dynamic stimuli?
• Analyzing and Reporting Data• How much time required?• Use quantitative metrics?• Spatial (heatmaps) vs. temporal (scanpaths)?• How to aggregate eye tracking scanpaths?• How to extract scanning strategy from data?• When are results significant enough to be meaningful?
Eye Tracking Challenges: Technical
• How to Define Fixations?• Several methods
• How to Define AOIs?• Padding• Overlap, hierarchy
• Assign Fixations to AOIs?• Spatial location + path?
• Gaze Location Error• Calibration• Foveal Area/Resolution• Attentional Dissociation
• Manage Scanpath Interruptions?• How to Define Metrics?
• No best practices or standards
Comparison of Information Graphics:An Evaluation Study
• Objective• Which graph type(s) best support relative comparison along
multiple dimensions, within and between graphs?• Compare bar, line, and spider graphs
• Participants: 5 graph users (Oracle colleagues)• Apparatus: Tobii T60 Eye Tracking system• 6 Tasks (same order)
• 3 Easy (2 dimensions, 1 graph)• “Should Candidate 2 be hired, considering only job dimensions a and b?”
• 3 Difficult (8 dimensions, 2 graphs)• “Which job candidate exceeded the greatest number of job requirements: 2 or 4?”
Graph Stimuli
• Each 4-graph set:
• 15x15 cm, Center of screen
1 job candidate per graph, indicated by number
Job dimensions indicated by a-h
Job requirements
Candidate abilities
Decision making, validation, uncertainty
Completion vs. First Fixation Times• Completion Times Include Scanning, Decision Making, and
Validation
Scanning Strategies• Two Different Comparison Strategies
• Bar Graphs: Most of first bar graph dimensions viewed before second bar graph viewed
• Spider Graphs: Much more reciprocal scanning between dimensions, compared to bar graphs
Discussion Questions
• How can eye tracking methods help evaluate information visualizations?
• What are the most appropriate metrics?• Can eye tracking methods evaluate dynamic,
interactive visualization usability?• How to aggregate and represent group scanning
strategies on visualizations?• Is there a ‘standard practice’ for eye tracking
methods?