Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
UX Researchresults evaluation Eye Tracking
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
https://www.youtube.com/watch?v=YsCokWHGLXcErgoneers Eyetracker 2013 - Ronaldo
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
What you’ve got so far
results evaluation
Recordings of • Fixations and Saccades • its order and duration • mapped onto the test object
Next steps
by all of your tested users (separate)
ReviewCumulate and VisualizeUnderstand and communicate
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
AnalysisTools
ReviewCumulate and Visualize
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Heatmap
results evaluation
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 118
Based on either
• number of fixations or
• fixation length
recommendedone long fixation should count for more than a short one
Shows how looking is distributed over the stimulus.
Can be created with one or many users
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Heatmap
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 120
- Display Colors can be chosen
Software Defaultred = area most looked at („hot“)green = area least looked at („cold“) blanc = no fixations
n/n group
red = area most looked at („hot“)blue = area least looked at („cold“)
gray = no fixation at all
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Heatmap
Problems
• no indication of the order people looked at items
• difficult with dynamic elements
that appear or move:drop down, pop up, animation
• probably misleading
Qualities• easy to read
• very popular
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 117
analysis tools
!
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 135/137
Gaze differs from page to pageeven referring to the same test object
Influenced by
• InterestSubpages may attract different interests
• Experiencegained on previous pages of the same site
Screenshots from kiehls.com (sells cosmetics)
analysis tools
Heatmap
User Recall
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
User RecallScreenshots from kiehls.com (sells cosmetics)
Subpage/product pageUser interest: gathering information about the specific product
No fixations to Menuat the top of page!!
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 135/137
Heatmap may indicate, users did not look at the navigation. But they did!Just not on this page.
analysis tools
Experience with site: Familiar with navigation and categories of the site from previous visits to Home and other product pages
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
User Recall
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 137
HomeUser interest: • general orientation • search for categories Experience with site: • none
Many fixations to Menuat the top of page!
User want get ageneral idea of the site.
Once they’ve decided forone of the categories, interest (and gaze) changes.
analysis tools
Screenshots from kiehls.com
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
User Recall
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 135/137
Screenshots from kiehls.com (sells cosmetics)
HomeUser interest: • general orientation • search for categories
Experience with site: • none
Subpage / product pageUser interest: • gathering information about the specific product
Experience with site: • Familiar with navigation and categories …
No fixations to Menu doesn’t necessarily mean its bad designed or placed! Just no interest at the given context and moment.
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Replays
analysis tools
Watch the test session video again
You see a dot, or user’s eye, moving around
if recorded, you hear ambient noise or users comments
Reliable Gaze Mapping during Screen Changes Scroll Down, Close Ad window
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Replay
source: IMS.UX Team „WeScan“ 2017
analysis tools
ExampleEvaluation of Print Layout
TaskFind out, how to get in contactwith the company
Comparisonof two test persons
!
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Replay
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 116f
Pros
Cons
• you can slow down the replaytherefor seeing everything the user looked at
• Order of Fixations is visible
• Relation of gaze and dynamic elements of the item(drop down menu etc.) is visible
• Rather time consuming to watch even at real speed(best results by watching at 1/3 speed)The eye moves so quickly you often do have to slow and replay segments to get an accurate idea of what happened.
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Plots
source: UX-Lab Test 2016 „Visibility of Posters in natural context“
results evaluation
show the location, order, and time spent looking at locations on the stimulus
primary function: to reveal the time sequence of looking or where we look and when we look there.
https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/interpret/working-with-heat-maps-and-gaze-plots/
Time spent looking … is shown by the diameter of the fixation circles. The longer the look, the larger the circle.
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Plots
source: UX-Lab Test 2016 „Visibility of Posters in natural context“
results evaluation
Eye gaze of one user on one page
Blue dots show:
• where the user’s fixation were
• numbers in the dots depict order of fixations
• size of the dot relates to duration of fixation
• time slider may be used todepict parts of the session(first 10 sec. for instance)
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
source: IMS.UX 2019 Team „Foreign and Confused“
Gaze Plots
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 117
Problems• dots cover each other
• dots cover the area the user is interested inpossible to adjust transparency of the dots
• time consuming to analyze
Qualities• to distinguish between users
with different behaviorheatmap can be misleading in this case
analysis tools
!• difficult with dynamic elements drop down, pop up; elements that appear and disappear
source: IMS.UX 2019 Team „Foreign and Confused“
Gaze Plots
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Gaze Plots
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 126
• difficult with dynamic elements drop down, pop up; elements that appear and disappear
analysis tools
!
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
EyetrackingAreas of Interest
analysis toolshttps://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies pdf p. 138
• draw boxes around elements on pages • name them • have the system compare
how long and how often these items were looked at
analysis tools
What is it?Areas of Interest
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking analysis toolsAreas of Interest https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
What you getspreadsheet data displaying the total number of users who looked at each AOI and for how long.
difficult with dynamic elements (drop down, pop up; moving images)
analysis toolsAreas of Interest https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Analyzing Animation
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies pdf p. 139
Possible:
Technical possible to create Heatmaps,but they show the static picture of one part of the video (instead of real video situation at each moment of tracking)
Scene changes make (automatic) mapping impossible
Areas of Interest in dynamic content
analysis tools
Static data visualization(Heatmap, gaze plot) with changing (dynamic) test items very difficult
on top of that:Websites (interactive applications) asynchronous user behavior whereas in Video scenes have predetermined duration
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: https://www.youtube.com/watch?v=DExiqRSWSeM
Areas of Interest in dynamic content
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Areas of Interest in dynamic content / moving environmentInfrared ID Markers to define AOI in natural user environment
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
Infrared ID Markers to define AOI in natural user environmentAreas of Interest in dynamic content / moving environment
source: UX-Lab Test 2016 „Evaluation of machine interface in production process“
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
Areas of Interest in dynamic contentInfrared ID Markers to define AOI in natural user environment
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
Infrared ID Markers … would indicate the Posters of Interest … and therefor influence test situation
Areas of Interest in dynamic content / moving environment
source: UX-Lab Test 2015 „Visibility of Posters in natural context“
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Solution: Gaze Replay combined with human coding and note taking and manual measurement (if needed) of observation time for each item of interest
source: UX-Lab Test 2015 „Visibility of Posters in natural context“
Areas of Interest in dynamic content / moving environment
analysis tools
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
some moremetrics
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Time to first Fixation (TTFF)
source: iMotion Eye Tracking The complete Pocket guide pdf p. 15
more metrics
amount of time it takes a respondent to look at a specific AOI from stimulus onset.
Easy to compare and cumulate
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Time spent
source: iMotion Eye Tracking The complete Pocket guide pdf p. 15
Amount of time that respondents spent looking at an AOI
Prolonged visual attentionpoints at a high level of interestShorter times indicate that other areas on screen or environment are more attracting
more metrics
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Respondent Count
source: iMotion Eye Tracking The complete Pocket guide pdf p. 15
How many of respondentsactually guided their gazetowards a specific AOI
Higher account showsAOI is widely attended toLow counts shows that little attention is paid to it
more metrics
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Respondent Count
source: iMotion Eye Tracking The complete Pocket guide pdf p. 15
How many of respondentsactually guided their gazetowards a specific AOI
Higher account showsAOI is widely attended to
more metrics
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
Interpretation
Understand
ReviewCumulate and Visualize
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Eye motion is tightly linked to visual attention.
You just can‘t move your eyes without moving attention.
source: iMotion Eye Tracking The complete Pocket guide pdf p. 17
InterpretationSeeing / Perceiving
While eye tracking can tell us what people look at and what they see, it can’t tell us what people perceive
You can however certainly shift attention without moving your eyes.
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking InterpretationSeeing / Perceiving
You open the fridge in search for some milk.While it is right in front of you, you can‘t seem to find it. You keep looking until you close the door empty-handed.
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
EyetrackingSeeing / Perceiving
Even though you saw the container (and this is what eye tracking data would tell us), you were probably not paying sufficient attention to actually perceive that it was right in front of your eyes.
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
EyetrackingSeeing / Perceiving
During a search task one can easily fixate briefly on the search object and miss its presence, especially if the object has an unexpected shape or size …
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/interpret/what-does-eye-tracking-data-tell-us/
Interpretation
Sometimes Fixations do not necessarily translate into a conscious cognitive process.
Follow up the eye tracking test with an interview to assess the participant’s motivations or expectations.
Seeing / Perceiving
This may happen because our expectation of what the object (or scene) should look like modulates our visual attention and interferes with the object detection.
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
• which visual elements attract immediate attention • which visual elements attract above-average attention • if some visual elements are being ignored • in which order the visual elements are noticed • how the visual material compares to other material
Reveals
source: iMotion Eye Tracking The complete Pocket guide pdf p. 18
Interpretation
However, tracking gaze positions alone doesn‘t tell us anything particular about the cognitive processes and the emotional states that drive eye movements.
Combine Eyetracking with other data / insightsTherefor
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
iMotions Biometric Research Platform
Eyetracking
source: iMotion Eye Tracking The complete Pocket guide pdf p. 20ff
Combine Eyetracking with other data / insights
Interpretation
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
iMotions
Eyetracking
source: iMotion Eye Tracking The complete Pocket guide pdf p. 20ff
Physiological data generated with biometric sensors:
• facial expression analysis (webcam-based)a: human observationA confused expression while scanning the items in the fridge followed by a sad expression would have told us how you were feeling when you realized that the milk was missing.
• b: softwareWhile facial expressions can measure the presence of an emotion (valence), they can’t measure the intensity of that emotion (arousal) )
• Galvanic Skin Response (GSR, electrocardiography) (quantifies level of emotional arousal and stress levels based on the changes in skin conductance)
• EEG (electroencephalography) (Messung Hirnaktivität)measures the electrical activity of the brain at the scalpprovides information about the levels of engagement (arousal), motivation, frustration, cognitive workload.
Combine Eyetracking with other data / insights
Interpretation
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
source: iMotion Eye Tracking The complete Pocket guide pdf p. 22/32
• EMG (Electromyographic sensors )to monitor muscular responses to any type of stimulus; associated with emotional expressions (facial EMG) or consciously controlled hand/finger movements.
• Electroencephalography (ECG, records heart rate (EKG)) & Photoplethysmography (PGP) optical technique to detect changes in blood circulationobtain insights into physical state, anxiety and stress levels (arousal)
Combine Eyetracking with other data / insights
Interpretation
iMotions Biometric Research Platform
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking Interpretation
Easier / Low cost but very helpful:
Post Test InterviewsIA6-2018 „LandsApp“
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking InterpretationPost Test Interviewscollecting personal feelings, thoughts and attitudes
Do it immediately after the Eyetracking test!people forget, make up stories
IA6-2018 „LandsApp“
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
Synthesize Findings
Understand
ReviewCumulate and Visualize
and communicate
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking synthesize findings
Results … Eyetracking Data One list per test
source: UX-Lab Test 2013 „Usability of MMIs in Cars“
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking results evaluation
source: UX-Lab Test 2013 „Usability of MMIs in Cars“
Results … same datatransformed into visual representation
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
• Test goals • Criteria of success • Measurements
Synthesize your findings
• Test persons (number, representing target group, interested in topic)
• Equipment / Situation(accuracy of measurement, natural user situation)
• Criteria of success / Measurements(assumptions correct?)
• Test scenario / tasks / prototype(Did scenario and prototype meet the test goals? Have the tasks influenced the measured results in one way or the other?)
Based on your
Include a critical evaluation of your test conditions
synthesize findings
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Always take notes while testingpossibly think of backup recording (video) solution(system may crash, files not saved correctly, operating errors by test team)
Notes will also help to focus your analysis on specific situations and issues
Recommendations
synthesize findings
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Eyetracking
Heatmaps is a good starting pointbut do not tell the whole story!
Use Gaze plots and replays to get a detailed understanding
Search for patterns in user behavior instead of pure aggregation (heatmap)
Recommendations
source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 143
synthesize findings
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
PracticePrepare your report
• Test scenario • Critical evaluation of test conditions • Measurements and results • Interpretation / Findings • Integrate visuals and video
to communicate your results
Summarize your test results into one document:
Questions?
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
PracticeFor procedures to summarize your findings by Thinking aloud / Note taking:
Refer to the lecture Questionnaire Analysis section qualitative analysis
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
UX-ResearchResults Evaluation
Eyetracking
KP Ludwig John
Practice
hs-augsburg.de/homes/john
Schedule and Scripts: