Upload
jaceksg
View
220
Download
0
Embed Size (px)
Citation preview
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 1/40
Jacek GwizdkaDepartment of Library and Information Science
School of Communication and Information
Rutgers University
Monday, April 4, 2011
LearningaboutInforma0onSearchers
fromEye-Tracking
CONTACT:
www.jsg.tel
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 2/40
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 3/40
OverallResearchGoals
• Characteriza*onandenhancementofhumaninforma<on
interac<onmediatedbycompu<ngtechnology
• Characteriza*on:cogni<veandaffec<veuserstates–
tradi<onally:li>leaccesstothemental/emo<onalstatesofuserswhiletheyareengagedinsearchprocess
• Implicitdatacollec<onaboutsearchers’cogni<veand
affec<vestatesinrela<ontoinforma<onsearchphases
• Enhancement :Personaliza<onandAdapta<on
3
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 4/40
Example:ImplicitCharacteriza0onof
Cogni0veLoadonWebSearch
4
Qformulate
query
L view
search
results list
Bbookmark
page
ST A R T
END
C view
contentpage
97% 58% 30%
42%
95%
7%
27%35%
higher peak
cognitive load: Chigher average
cognitive load: Q & B
(Gwizdka, JASIST, 2010)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 5/40
Eye-Tracking?
• Earlya>emptslateXIXc.;early1950’s-amoviecamera
andhand-coding(Fi>s,Jones&Milton1950)
• Nowcomputerizedand“easytouse”
– infraredlightsourcesandcameras
– sta<onaryandmobile
5
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 6/40
Eye-tracking–fundamentalassump0ons
• Top-downvs.bo>om-upcontrol
– inbetween:languageprocessing(higher-level)controlswheneyes
move,visualprocessing(lower-level)controlswhereeyesmove
(Reichleetal.,1998)
• Eye-mindlinkhypothesis:a>en<oniswhereeyesare
focused(Just&Carpenter,1980;1987)
• Overtandcoverta>en<on
• A>en<oncanmovewithnoeyemovementBUTeyes
cannotmovewithouta>en<on
6
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 7/40
DatafromEye-trackingDevices
• eyegazepoints
– eyegazepointsinscreencoordinates+distance
– eyefixa<onsinscreencoordinates+validity
– pupildiameter• [headposi<on3D,distancefrommonitor]
• 50/60Hz;300Hz;1000-2000Hzeye-trackers
• common:60Hz:onedatarecordevery16.67ms
7
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 8/40
Eye-TrackingCan…
• Eyetrackingcanallowiden<fica<onofthespecificcontent
acquiredbythepersonfromWebpages
• Eyetrackingenableshighresolu<onanalysisofsearcher’s
ac<vityduringinterac<onswithinforma<onsystems• Andmore…
8
Example: composing answer and
from information on a Web page
(video)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 9/40
RelatedWorkinInforma0onScience
• Interac<onwithsearchresults
– Interac<onwithSERPs(Grankaetal.,2004;Lorigoetal.,2007;2008)
– Effectsresultspresenta<on(Cutrelletal.,2007;Kammereral.,2010)
– Relevancedetec<on(Buscher,etal.2009)
– ImplicitFeedback(Fu,X.,2009);Queryexpansion(Buscher,etal.2009)
• Relevancedetec<on
– Pupillometry(Oliveira,Aula,Russell,2009)
• Detec<onoftaskdifferencesfromeye-gazepa>erns – Reading/reasoning/search/objectmanipula<on(Iqbal&Bailey,2004)
– Informa<onalvs.transac<onaltasks(Terai,etal.,2008)
– Taskdetec*onisalsooneofourresearchinterests9
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 10/40
Experiment1:Journalismtasks–OpenWebSearch
• 32journalismstudents
• 4journalis<ctasks(realis<c,createdbyjournalismfacultyandjournalists)
• Tasks:
– advancedobituary(OBI) – interviewprepara<on(INT) – copyedi<ng(CPE) – backgroundinforma<on(BIC)
10
Note: OBI vs. CPE are most dissimilar
Task facets:
• product: factual vs. intellectual• level: whole document vs. segment• nature of task goal• complexity – number of steps needed
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 11/40
Experiment1–ResearchQues0ons
• Canwedetecttasktype(differencesintaskfacets)from
implicitinterac<ondata(e.g.,eye-tracking)?
• Howdoweaggregateinforma<onfromeye-trackingdata?
11
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 12/40
Eye-gazepaMerns
• Eye-trackingresearchhave
frequentlyanalyzedeye-gaze
posi<onaggregates('hotspots’)
– spa<otemporal-intensity–heatmaps – alsosequen<al–scanpaths
12
• Higher-orderpa>erns:
– readingmodels
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 13/40
ReadingEyePaMerns
• Readingandscanninghaveeasilydis<nguishedpa>ernsof
fixa<onsandsaccades.(Rayner&Fischer,1996)
• LexicalProcessingofWords
– Readingresearchhasestablishedwordavailabilityisafunc<onoffixa<ondura<on:
– Orthographicrecogni<on:40-50ms
• <metomovedatafromeyestomind
– Phonologicalrecogni<on:55-70ms
– Lexicalavailability(typical):113ms–150ms(Rayner,1998)• Unfamiliarorcomplexmeaningsrequirelongerprocessing
– Eyesdonotsaccadeun<lthewordhasbeenprocessed
13
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 14/40
ScanFixa0onsvs.ReadingFixa0ons
• Scanningfixa<onsprovidesomeseman<cinforma<on,
limitedtofoveal(1°visualacuity)visualfield(Rayner&Fischer,1996)
• Fixa<onsinareadingsequenceprovidemoreinforma<on
thanisolated“scanning”fixa<ons: – informa<onisgainedfromthelargerparafoveal(5°beyondfoveal
focus)region(Rayneretal.,2003)(asymmetrical,indirofreading)
– richerseman<cstructureavailablefromtextcomposi<ons
(sentences,paragraphs,etc.)
• Someofthetypesofseman<cinforma<onavailableonly
throughreadingsequencesmaybecrucialtosa<sfytask
requirements.
14
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 15/40
Readingodels
• WeimplementedtheE-ZReaderreadingmodel(Reichleetal.,
2006)
– Inputs:(eyefixa<onloca<on,dura<on)
– Fixa<ondura<on>113ms –thresholdforlexicalprocessing(Reingold&Rayner,2006)
– Thealgorithmdis<nguishesreadingfixa<onsequencesfromisolated
fixa<ons,called'scanning'fixa<ons
– Eachlexicalfixa<onisclassifiedto(S,R)(Scan,Reading)
– Thesesequencesusedtocreateastatemodel
15
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 16/40
Readingodel–StatesandCharacteris0cs
• Twostates:transi<onprobabili<es
• Numberoflexicalfixa<onsanddura<on
16
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 17/40
ExampleReadingSequence
17
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 18/40
Results:SearchTaskEffectonReading/Scanning
Taskeffectsontransi<onprobabili<esSR&RS(allsubjects&pages)
18
(Cole, Gwizdka, Liu, Bierig, Belkin & Zhang, 2010)
• For OBI, INT searchers
biased to continue reading • For CPE to continuescanning
Searchers are adoptingdifferent reading strategies for
different task types
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 19/40
Results:SearchTaskFacetsandTextAcquisi0on
Forhighlya>endedpages
19
Total Text Acquisition on
SERPs and Content
per page
Total Text Acquired on
SERPs and Content
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 20/40
Results:SearchTaskFacetsandStateTransi0ons
Forhighlya>endedpages
20
State Transitions on
SERPs per page
State Transitions on
Content pages per page
Read ScanRead Scan Scan Read
Scan Read
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 21/40
TaskFacetsEffects-Summary
Forhighlya>endedpages
21
(Cole, Gwizdka, Liu, Bierig, Belkin & Zhang, submitted, 2011)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 22/40
Scan<->ReadTransi0onProbabili0esin2Experiments
• Person’stendencytoreadscanrelatedto
scanread?(i.e.,isprelatedtoq?)
• p~1-q
correlation (Spearman ρ): 0.914 and 0.830
Journalistic tasks (N=32) Genomics tasks (N=40)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 23/40
Experiment1:Conclusions
• Searchers’reading/scanningbehavioraffectedbytask
• Tasksfacetscanbe“detected”fromeye-trackingdata(from
readingmodelproper<es)
• Readingmodelscanbebuiltonthefly(duringsearch)real-<meobserva<onsofeyemovementscanbeusedbyadap<ve
searchsystems
• Challenge:Lackofbaselinedataaboutreadingmodelsofindividuals
23
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 24/40
Experiment2:ResultListvs.OverviewTag-Cloud
37par<cipants
Everydayinforma<onseekingtasks(travel,shopping…)
-twolevelsoftaskcomplexity
Twouserinterfaces
24
1. List UI2. Overview UI
(Tag Cloud)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 25/40
Experiment2:UserAc0onsinTwoInterfaces
25
1. List
2. Overview
Tag Cloud
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 26/40
Experiment2:ResearchQues0ons
• Doesthesearchresultsoverviewbenefitusers?
• Taskeffects?
• Individualdifferences-cogni<veabilityeffects?
26
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 27/40
GeneralResults
• Searchresultsoverview(“tagcloud”)benefitedusers
– madethemfaster
– facilitatedformula<onofmoreeffec<vequeries
• Morecomplextaskswereindeedmoredemanding–required
moresearcheffort
27
(Gwizdka, Information Research, 2009)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 28/40
TaskandUIandReadingodeldifferences
• Complextasksrequiredmorereadingeffort
– Longermaxreadingfixa<onlengthandmorereadingfixa<onregressions
• OverviewUIrequiredlesseffort
– Scanningmorelikely(S-Shigher;S-Rlower;R-Shigher)
– Readingscanpathlengthtotallowerbuttotalscanpathswerelonger
(includingscanning)
– Lessandshortermeanfixa<onsperpagevisited
28
List Overview
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 29/40
TaskandUIInterac0onandReadingmodeldata
• ForcomplextasksUIeffect
– Higherprobabilityofshortreading
sequencesinOverviewUI
• ForsimpletasksUIeffect
– Shorterlengthofreadingscanpathsper
pageandlessfixa<onsperpage
• Task&UIinterac<on
– Speedofreading:
• forcomplextasksfasterreadinginOverviewthaninListUI
• forsimpletasksfasterinListthaninOverviewUI
29
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 30/40
UserInterfaceFeatures–IndividualDifferences
• Twousers,sameUIandtask
30
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 31/40
IndividualDifferences–LeastEffort?
• Highercogni<veabilitysearcherswerefasterinOverviewUI
andonsimpletasks(samenumberofqueries)
• Higherabilitysearchersdidmoreinmoredemandingsitua<ons
– highersearcheffortdidnotseemtoimprovetaskoutcomes
31
F(144,1)=4.2; p=.042 F(144,1)=3.1; p=.08
For tasks and working memory (WM)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 32/40
TaskandWorkingemory–Eye-trackingData
• HighWMlesslikelytostayinScanningstate
• HighWMhigherreadingspeed(scanpath/totalfixa<ondura<on)
• Numberanddura<onofreadingsequencesdiffers – (borderline:0.05p0.1)
• ForhighWMsearchers:
– forsimpletaskslessreading
– forcomplexmorereading
• ForlowWMnosuchdifference!
32
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 33/40
Experiment2:Conclusions
• OverviewUIwasfaster–reflectedinsomeeye-tracking
measures
• Taskcomplexitydifferencesreflectedinsomeeye-tracking
measures
• Someeffectsofcogni<veabili<esoninterac<on
– viola<onoftheleasteffortprinciplenotfullyexplainedyet
33
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 34/40
34
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 35/40
CurrentProject:
CanWeImplicitlyDetectRelevanceDecisions?
• Startwithpupillometry
– inforelevance(Oliveria,Russell,Aula,2009)
– low-leveldecision<ming(Einhäuser,etal.2010)
• AlsolookatEEG,GSR
FundedbyGoogleResearchAward
EEG
GSR
Eye trackingpupilanimation
• Implicitcharacteriza<onofInforma<onSearchProcess
usingphysiologicaldevices
• Canwedetectwhensearchersmakeinforma<onrelevancedecisions?
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 36/40
Summary&Conclusions
• Eyetrackingenableshighresolu<onanalysisofsearcher’s
ac<vityduringinterac<onswithinforma<onsystems
• Eyetrackingcansupportiden<fica<onofsearchtasktypes• Eyetrackingreflectsdifferencesinsearcherperformanceon
userinterfaces
• Eyetrackingreflectsindividualdifferencesbetween
searchers
• Highpoten<alforimplicitdetec<onofsearcherstates
36
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 37/40
SomeChallenges
• High-resolu<ondata(low-level)
• Howdowecreatehigher-levelpa>erns?
• Howdowedetectthemcomputa<onally?
• Howdowedealwithind.diffs(baselinedata)?
37(Lorigo et al., 2008)(Terai et al., 2008)
(Iqbal & Bailey, 2004)
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 38/40
High-resolu0onEye-trackingisComingSoontoYou
• Eyetrackingtechnologyisdeclininginpriceandin2-3years
couldbepartofstandarddisplays.
– Alreadyinluxurycarsandsemi-trucks(sleepdetec<on)
– Computerswithbuiltineye-tracking
38
Tobii / Lenovo
proof of concept laptop
March 2011
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 39/40
Thankyou! Ques<ons?
JacekGwizdkacontact:h>p://jsg.tel
PoODLEProject:Personaliza<onoftheDigitalLibraryExperience
IMLSgrantLG-06-07-0105-07
h>p://comminfo.rutgers.edu/research/poodle
orforshort:h>p://bit.ly/poodle_project
PoODLEPIs:NicholasJ.Belkin,JacekGwizdka,XiangminZhang
Post-Doc:RalfBierig,
PhDStudents:JingjingLiu,(nowAsstProf.),ChangLiu,MichaelCole
8/3/2019 Learning about Information Searchers from Eye- ‐Tracking
http://slidepdf.com/reader/full/learning-about-information-searchers-from-eye-tracking 40/40
OtherEye-trackingDerivedeasures
• Speedoftextprocessing
– readingspeed;scanpathlength/totalfixa<ondura<on
• Perceptualspan – numberofcharactersprocessedata<me
– Englishetc.~19characters(Rayner,1998)
– decreasesinnormalreadingwhenuserencounterscogni<vely-
demandingcontent
40