Computers in Human Behavior 26 (2010) 1555–1563

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

Influences of data filtering on human–computer interaction by gaze-contingentdisplay and eye-tracking applications

Giacomo Veneri a,b,*, Pamela Federighi a,b, Francesca Rosini b, Antonio Federico b, Alessandra Rufa a,b

a Eye Tracking and Vision, Applications Lab, University of Siena, Italyb Department of Neurological, Neurosurgical and Behavioral Science, University of Siena, Italy

a r t i c l e i n f o

Article history:Available online 19 June 2010

Keywords:Eye movementsGaze-contingent displaysFilterAttentive displaysEye-tracking

0747-5632/$ - see front matter � 2010 Elsevier Ltd. Adoi:10.1016/j.chb.2010.05.030

* Corresponding author at: Department of NeurBehavioral Science, University of Siena, Viale Bracci 20577 233136; fax +39 0577 40327.

E-mail addresses: g.veneri@unisi.it (G. Veneri), ruf

a b s t r a c t

We describe an interactive gaze-contingent display (GCD) applied to clinical applications; the systemuses a simple texture hole to inhibit peripheral vision, to test and stress overt mechanisms of visualsearching in normal subjects. The correct use of GCD in vision research is affected by tremor of the hole,due to system noise, nystagmus, eye blinking, calibration and subject reactivity. These issues compromisethe execution of task. In order to obtain a stable GCD hole, we implemented a predictive gaze-contingentdisplay (PGCD), fitting through dispersion of fixations and modulating a filter. The paper describes thePGCD and compare it with the common technique, providing evidence that humans fit exploration basedon the characteristics of the computer system; in particular we found significant difference applyingPGCD or a simple finite impulse response filter. We suggest that a correct human–computer interactionapplied to neuropsychological context must be developed taking in consideration both technical point ofview and human behavior.

� 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Human–computer interaction is one of the most importantaspect of computer system applied to neuro-cognitive and psycho-logical studies and, in particular, on vision. On vision research acommon method to test neuro-cognitive functionalities is to usemultimedia applications and to register some special human fea-tures, such as latency, correct responses, hand or eye movements(Toet, 2006); in some particular cases a large amount attentionmust be spent to apply a correct human computer interaction tech-nique. In fact, to investigate the influence of some cognitive func-tions, such as peripheral vision, scene understanding, parallelprocessing of human neural system, bias due to computer systemissues (latency, reactivity) or human physiological features (eyeblink, eye tremor) must be avoided.

A common method to study these functions of human vision isthrough eye-tracking application, which consists a remote-mounted (infrared) video camera able to record gaze data: hori-zontal and vertical movements, and pupil diameters (for a surveyof eye-tracking applications Duchowski, 2002; Morimoto &Morimoto, 2005; Rayner, 1978; Rayner, Li, Williams, Cave, & Well,2007).

ll rights reserved.

ological, Neurosurgical and, 53100 Siena, Italy. Tel.: +39

a@unisi.it (A. Rufa).

From a system analysis point of view, eye-tracking applicationsshould be distinguished from diagnostic or interactive system. Indiagnostic mode, the eye-tracker provides data about the obser-ver’s visual search and attention processes. In interactive mode,the eye-tracker is used as an input device. From a general pointof view, an interactive system responds to the observer’s actionsand interacts with him. Duchowski (2007) also distinguishes twotypes of interactive systems: (a) selective, where the point of gazeis used as a pointing device; and (b) gaze-contingent display (GCD),in which the observer’s gaze changes the rendering of complexinformation displays.

1.1. Gaze-contingent displays

Gaze-contingent displays (GCDs) and applications, have beendescribed in several articles (for example McConkie & Rayner,1975; Pomplun, Reingold, & Shen, 2001) and have been used invarious applications, such as reading, images and scenes percep-tion, virtual reality, computer graphics, and visual search studies.

Recent researches (Vinnikov, Allison, & Swierad, 2008, 2006,2002) have concentrated on correctly placing and displaying thehole on the image by various techniques and avoiding geometricaldistortion using real images as source. GCD is thought to have aneffect on visual perception and attention. Studies on the effectsof GCD in visual perception have shown that it may influence theresponse of subjects during visual search. Murphy, Duchowski,and Tyrrell (2009) developed a gaze-contingent application and

1556 G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563

found that their system improved visual search ability, suggestingthat GCD may influence cognitive response. These characteristicencourage the use of GCD to study specific cognitive responses inclinical and neuropsychological contexts.

1.2. Interactive GC displays on vision and cognitive experiments

Interactive GC applications may be suitable for perceptionexperiments, particularly those studying interactions betweencentral and peripheral vision during complex scene exploration(Simion & Shimojo, 2006): the human visual system can only re-solve detailed information within a very small area at the centerof vision (fovea). Outside this area, spatial resolution drops rapidlyfrom the center to the periphery. Conversely, the peripheral retinais more sensitive to moving targets than stationary ones. Therefore,while central vision allows the exact processing of informationcentered around the current fixation point, peripheral vision pre-pares the next exploration and supports the eye movement (sac-cade) to the next point of interest (Findlay, Brown, & Gilchrist,2001). Both central and peripheral vision modulate attention pro-cesses, either overtly by shifting the eyes to the current point ofinterest or covertly by shifting the focus of attention without anyeye movement (Posner & Petersen, 1990, 2007).

Using a central hole to see the scene only through the fovea andsmoothing its border, Simion and Shimojo (2006) were able to givesubjects the sensation of seeing through a telescope (see Fig. 5).The method forces the subject to explore the scene by foveal vision(central hole) since peripheral perception is obscured; it makespossible to split foveal and peripheral vision and their specificeffect on overt and covert attention processes: covert attention isrestricted and the subject is forced to search overtly, alternatingsequences of saccades and fixations (Fig. 1).

A correct application of the mechanism proposed on clinical andneuropsychological experiment avoiding any bias due to machineor human should be investigated.

In fact, due to the natural eyes tremor, human made the holeunstable and this affected correct execution of the task. The insta-bility of the hole during visual fixation may disturb the attention

pixels

pixe

ls

100 200 300 400 500 600 700 800 900 1000

100

200

300

400

500

600

700

Fixation

Saccade

Fig. 1. Trail making: in the simplified form of the test, subjects are asked to trace asimple sequence such as 1-A-2-B-3-C-4-D-5-E with the gaze. The numbers andletters appear in a pseudo-random distribution (letters on top and numbers on thebottom) on a 1024 � 768 px and 58 � 31 cm screen for 20,000 ms. Subject is forcedto search overtly, alternating sequences of saccades and fixations (red curve).Saccades are rapid ballistic eye movements (up to �600 deg/s) shifting the gaze(Becker & Fuchs, 1969) to a new target. Fixation maintains the gaze in one place toacquire information (Yarbus, 1967). (For interpretation of the references to colourin this figure legend, the reader is referred to the web version of this article.)

process, altering the experimental results. To solve this problem,a common technique is to apply a (low-pass) real-time digital filterto compensate for eye tremor.

The GCD system, however, should be less reactive to rapidmovements (saccades) and subjects should find moving the holetiring (Widdel, 1984). From a system point of view a simple correc-tive mechanism should be applied, disengaging the filter when thesaccade starts (high velocity). The saccade, however, is planned byhuman system 50–100 ms before the starting and should be pro-grammed according to hole reactivity. To solve this problem, weapplied applied a novel technique which tried to maintain the holestable during the fixation and to disengage the filter before thestarting.

The principle of the proposed technique was to develop a pre-dictive filter (PF) which engaged a low-pass filter when the subjectwas fixating the screen and disengaged it when the subject wasmaking a saccade. The system operated in real-time (4.17 ms/sample; 240 Hz).

The aim of this study was, therefore, to investigate how theapplication of a digital filter applied to gaze data could changethe natural behavior of visual search and reduce eye tremor, andhow the proposed novel technique should offer a compromise be-tween requirements of cognitive experiments and artifacts due toeye tremor.

2. Cognitive experiment design

We used, to stress the nervous system, a highly cognitivedemanding task (Corbetta & Shulman, 2002), namely the trailmaking test (Bowie & Harvey, 2006), in which subjects were askedto follow an alphanumeric sequence with the gaze. The trailmaking stimulus was a pop-up high contrast image consisting ofa sequence of numbers and letters arranged in an unpredictablemanner. In our version of the trail making task, numbers were atthe bottom and letters at the top of the image (Fig. 1). Subjectswere instructed to trace the alphanumeric sequence by GCapproach. We replicated the GC mode proposed by Simion andShimojo avoiding smoothing of border: in fact, during the experi-ment we realized that while the smoothing mechanism of the hole(Simion & Shimojo, 2006, 2002) is a good technique for naturalvisual exploration (natural images for instance), it was not theideal method for our trail making, since subjects have to completethe sequence with a number of goal directed saccades and highlyinformative fixations (Bowie & Harvey, 2006).

The trail making applied to GCD is particularly suitable forstressing the nervous system because it forces the subject to searchsigns (letters and numbers) and, parallel, to make a sequence.

3. Developed gaze-contingent architecture

In defining the predictive gaze-contingent display (PGCD) wesought a balance between system responsiveness and fairness ofthe experiment, between response in real-time and a stable scenedevoid of artifacts.

Our aim was to create a filter that considered overt visual explo-ration through the central fixation hole, and perceptive processesthat occur during saccade (transaccadic perception). Thus, the holedid not jump from one fixation to another but pursued eye move-ments, even during saccades. In this way, the system became par-ticularly suitable for neuro-cognitive applications.

The principle is to develop a predictive filter (PF) that can esti-mate the duration of fixation and reduce the effect of filtering data.From the perspective of the system architecture, this task isdelegated to a component that engages or disengages a simplelow-pass filter. Fig. 2 shows the architecture of the system: in

Fig. 2. Gaze-contingent system: the engagement mechanism modulates (W parameter) the FIR filter by feedback from the fixation component which calculates fixations inreal-time. The fixation component and digital IR filter work on separate threads.

G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563 1557

the center, the engagement component (EC) engages data filteringduring a fixations and disengages it before/during saccades. The ECmakes a decision on the basis of the last fixation. The characteris-tics of fixation during visual exploration have a high degree of var-iability depending on: the subject’s endogenous status (intentionalbehavior, attention), task design, and individual approach duringthe exploration of visual scenes.

Thus, fixation parameters should be calculated at runtime by asecond component of the system: the fixation component (FC),which extracts duration, dispersion and fixation centroid. The ECreceives the parameters of fixation from the FC and modulatesthe filter which is a simple is a simple, computationally efficient,finite impulse response (FIR) filter.

3.1. Fixation component (FC)

The fixation component is based on a fixation dispersion algo-rithm (I-DT) developed by Salvucci and Goldberg (2000). The algo-rithm pseudocode is Algorithm 1.

From a technical point of view, the fixation component workson a separate thread and uses a double queue method: while aqueue (current queue) is filled with gaze data (x,y), the second(secondary queue) is used to find the fixations. When the secondqueue is empty, the queues are switched. This mechanism avoidsany thread synchronization mechanism and reduces time con-sumed by lock. Each time the FC finds a fixation, it sends the cur-rent fixation dispersion and duration to the EC.

A second mechanism that ensures a prompt FC response to cur-rent data acquisition is arranged as follows: when the first queue isfull, the algorithm is switched on to process the current queue. Thistechnique also allows for a pre-allocated queue dimension set at300 samples or 1250 ms. This corresponds to the average timespent by the subject in processing each letter of TMT: average timeis 12 s (tmin ¼ 8:1 s; tmax ¼ 32:3 s) for all 10 letters; capacity of the

queue should be custom defined, but in any case, it is automati-cally extended by system.

3.2. Finite impulse response (FIR) filter

The FIR is a simple digital filter described by the followingformula:

xf ¼ bxðtÞ þ � � � þ bxðt �WÞ ð1Þ

where the b is defined by default:

b ¼ 1=W ð2Þ

or changed by the operator through the user interface.The window W ¼ ½0;Wmax� modulates the effect of filter. When

W = 0 the filter is disengaged (saccade); when W ¼Wmax the filteris at the maximum effect; Wmax was set at 20 sample.

3.3. Engagement component (EC)

The engagement component mechanism of the filter is based onthree major parameters: duration of the previous fixation, disper-sion of the previous fixation and speed of saccade. The EC usesduration of the previous fixation to predict the current durationand to set the filter window. In fact, in our model, we found thatfixation duration changed during the task (Fig. 3) and could notbe calculated a priori in a learning phase. Moreover, the expectedduration of the current fixation was closer to the previous fixationthan to the mean duration of fixations. Fig. 4 shows the differencesbetween current fixation duration (ms) and previous fixation dura-tion or mean of fixation duration.

We used a t-test to test the hypothesis that the difference inmilliseconds between the previous fixation duration and the ex-pected fixation duration was less than 50 ms. The hypothesis wasverified (p < 0.05) on 23 normal subjects.

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2x 104

0

100

200

300

400

500

Time (ms)

Fixa

tion

dura

tion

(ms)

Subject 1

Fig. 3. Fixations duration trend during the execution of task of subject 1.

Fig. 4. Pale gray bar: difference in fixation duration of (i) with respect to previousand fixation (i � 1). It means the fixation distance to previous fixation. Dark graybar: difference in fixation duration (of i) with respect to the mean of fixationsduration (distance from mean). The y-axis indicates difference in fixation durationin milliseconds. PF tries to predict fixations duration according to previous fixationwith an error of 50–100 ms.

Fig. 5. Small portion of image; on the background the trail making test which thesubject have to explore (TMT); subjects could find the letters only by moving hiseyes. Gray circle on foreground only for readable purpose.

Fig. 6. Small portion of image; on the background the trail making test which thesubject have to explore; on foreground the same trail making test smoothed(TMTb). Subjects could see position of letters and numbers, but could find it only bymoving the gaze.

1558 G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563

Based on these assumptions, the EC uses fixation dispersion andduration to modulate the window W of the FIR filter (see PF Algo-rithm 2).

3.4. Scene display

We used a simple small texture hole, which only shows the im-age on fovea (<2�) and masks the rest of image. The purpose was toinhibit completely (Fig. 5) or partially (Fig. 6) peripheral vision.Implementation was undertaken with DirectX 11 on a Windowsplatform.

3.5. Eye-tracking system and real-time adaptation

We used the ASL 6000 system. The system uses corneal reflec-tion combinated with pupil tracking to stable tracking of eyeposition. The ASL 6000 system has high spatial resolution(noise-limited to 0.1�) and a data rate of 240 samples per second.It consists of two miniature remote-mounted cameras, one forscene and one for eye-tracking. In the context of this research,however, the most important feature of the eye-tracking systemwas its ability to provide on-line gaze position data with a delayless than 12 ms, which makes the system ideal for gaze-contingentdisplay applications.

Adaptation to the real-time context was possible using standardwindows operating system on dual Core Athlon 64bit machine (seeTable 1).

The user interface was developed in C# .NET 3.5 and DirectX 11.The acquisitions system was developed in C language and the filterin C language, with some code segments implemented in assemblyfor optimization; see Table 1 for performance considerations.

4. Methods

The experiment was repeated in two different condition of GCD:(1) normal status previously described [Eq. (5)] and (2) FIR always

Table 1Processing time of each component on a AMD Athlon 64 bit dual core 3.21 GHzcomputer.

Fixation componentTime per sample 0.055 msTime per fixation considering all sample 8.47 ms

Engagement componentTime per sample Less than 0.001 ms

FIRTime per sample 0.0021 ms

Fig. 7. Fixations of normal subjects with peripheral vision inhibited during the trailmaking test (TMT).

Fig. 8. Fixations of normal subjects with peripheral vision partially inhibited duringthe trail making test (TMTb).

G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563 1559

activated (W ¼Wmax). We analyzed sequencing ability, saccadepeak velocity, amplitude, and distributions and number of fixationsin order to compare the proposed mechanism with the FIR alwaysactivated.

4.1. Efficiency of predictive filter

A method to compare expected and estimated duration wasused; a true estimation was accepted when the error was less than50 ms; in prosaccadic experiment the time spent to start a saccadeis 200–250 ms (Leigh & Zee, 2006), in visual search, time to plan anew saccade was considered greater than 50 ms (Findlay et al.,2001). Therefore, 50–100 ms should be considered an acceptablevariability between two fixations on the same context (Yarbus,1967).

4.2. Cognitive experimental design

Ten healthy subjects (6 female and 4 male, aged 20–45 years)with normal vision, not taking drugs and without refractive defectswere enrolled. All participants in the study were trained on thetrail making test by a psychologist. The test consisted in tracing asequence of numbers and letters 1-A-2-B-3-C-4-D-5-E with thegaze. Each letter/number was displayed in red (60 cd/m2) on ablack background, sub tending of � 3� 4 degrees a visual angle,and arranged in random positions to avoid geometric trajectories.Subjects were seated at a viewing distance of 78 cm from a 24” col-or monitor (51 cm � 33 cm). Eye position was recorded using anASL 6000 system, sampling pupil location at 240 Hz. A nine-pointcalibration and three-point validation procedure were repeatedseveral times to ensure all recordings had a mean spatial error ofless than 0.3�. Data was controlled by a Pentium4 dual core3 GHz computer, acquiring signals via a fast UART serial port.

Head movement was restricted using a chin rest and bite bar.Subjects performed the experiment by GCD in two different ver-

sions of TMT. In the first version, subjects discovered letters andnumbers of TMT moving only the hole of GCD (Fig. 5) (TMT). Inthe second version subjects were able to see the position of lettersand numbers masked by a blurring filter (Fig. 6) (TMTb); lettersand numbers appeared only if subject directed the gaze on the sign.The effect is evident comparing fixations made by normal subjects(Fig. 8) and the same subjects with peripheral vision inhibited by agaze-contingent mechanism (Fig. 7). Subjects were unable to useperipheral vision as they fixate empty space in search of lettersand numbers. This results in greater dispersion of fixations by sub-jects with reduced peripheral vision.

5. Results and discussion

Our preliminary results (Table 3) indicate that subjects wereable to perform the sequence 1-A-2-B-3-C-4-D-5-E correctly bothon TMT and TMTb: difference in sequencing ability was not signif-icant (p = 9.17, F(3,36) = 0,119). Subject, however, spent a lot ofsparse fixations in performing the TMT than TMTb. We calculated

euclidean distance of fixation to the nearest ROI and we foundsignificant difference (p < 0.01, a ¼ 0:0085) between TMT(mean ¼ 84:64 pxð55:81Þ) and TMTb (mean ¼ 50:06 pxð38:75Þ);see Figs. 7 and 8.

5.1. Prediction performance

Fig. 9 shows representative data raw and filtered, window pre-dicted and observed of subject 1.

We subsequently showed that the EC worked well for long fix-ations and short fixations on empty areas (see left side of Fig. 9),but had to be integrated with a dispersion mechanism (Algorithm1: Eqs. (8) and (9)) when the subject reached an area containinginformation. During small fixations the subject presumably onlyperceived the image’s exogenous characteristics while at the sametime he explored next location (Reichle, Rayner, & Pollatsek, 2003,1967). Thus, during small fixations the subject did not require pre-cision, did not acquire much information and visual search was notaffected by any noise or artifacts. During longer fixations, cognitiveprocesses such as attention, decision making and executive pro-gramming were involved, thus the EC worked well and allowedsubjects to complete the task correctly.

3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 48000

200

400

600

sample

x co

ordi

nate

in p

ixel

s

3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 48000

20

40

sample

W1 o

bser

ved

3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 48000

20

40

sample

W1 p

redi

cted

Subject 1

Fig. 9. Normal Subject: red line shows filtered data respect to raw data (blue). Observed duration. Predicted duration. (For interpretation of the references to colour in thisfigure legend, the reader is referred to the web version of this article.)

1560 G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563

Fig. 10 shows the ROC curve of filter precision calculated bycomparing predicted fixation duration and observed fixation dura-tion (less than 50 ms of error accepted).

Table 2 shows that for healthy subjects sixty per cent of correctfixation durations were predicted on an interval of ±50 ms of ac-cepted error.

5.2. Predictive filter influence versus Finite impulse response filter

During the second experiment 10 healthy subjects performedthe same test with the engagement component working normally

Fig. 10. ROC Curve of 10 subjects: state variable was set positive if differencebetween predicted fixation duration and the observed fixation duration was lessthan 50 ms. Area P50%.

Table 2Results.

Fixation duration (ms) 531.6Correct prediction 60.3%

(PF group) and with the EC always activated (IR filter alwaysengaged) (IR group). Increased peak velocity (p < 0.01,F(3,1.8E3) = 21.27) was found between PF and IR in TMT and TMTbexperiments. Post hoc Holm sidak test confirmed the hypothesisthat subjects increased saccade velocity (VPeak) in IR methodsboth on TMT (p < 0.05, a ¼ 0:0170) and TMTb (p < 0.01, a ¼0:0127). Paired t-test confirmed only significant difference in TMTb(p < 0.01), but not on TMT (p = 0.062); Fig. 12 shows how peakvelocity varied on TMT and TMTb experiment per each subject:analysis reported that 80% of the subjects increased their speedof saccades in IR experiment compared to the PF experiment. Theresult can be explained by assuming that the subjects tended toovercome the resistance of IR increasing the peak velocity of sac-cades, making exploration less natural.

Similar results (Table 3) were found in saccades amplitude; thisresult, however, is strictly correlated to peak velocity, according tothe well-known relation, called ‘‘main sequence”, which is themost important tool to evaluate rapid eye movements (Baloh, Sills,Kumley, & Honrubia, 1975; Gangemi et al., 1991; Leigh & Kennard,2004). Fig. 11 shows the non-linear correlation between saccadevelocity (peak velocity) and saccade amplitude in main sequence;fitting curves confirms differences between IR and PF.

No significant differences (p = 0.17, F(3,3E2) = 1.7) were foundin fixations duration; Holm sidak test accepted the null hypothesisof equal mean (pTMT ¼ 0:13; pTMTb ¼ 0:10).

Table 3Results: mean and variance.

Predictive filter IR filter

TMT TMTb TMT TMTb

TaskSequencing (#) 9.7

(0.678)9.6(0.966)

9.8 (0.781) 9.6(0.881)

FixationsDuration (ms) 206.24

(82.80)210.11(85.08)

201.11 (81.55) 204.99(71.80)

Distance to nearestROI (px)

80.30(51.12)

71.12(54.31)

77.80 (55.08) 68.34(30.85)

SaccadesVelocity peak

(deg/s)373.27(152.33)

395.44(143.33)

395.80(173.25) 459.93(193.83)

Amplitude (deg) 12.52(6.28)

13.33(6.28)

12.64 (5.77) 15.28(7.33)

5 10 15 20 25 30200

300

400

500

600

700

800

900TMT

amplitude (deg)

vpea

k (d

eg/s

ec)

data PFfitted curve PFdata IRfitted curve IR

5 10 15 20 25 30200

300

400

500

600

700

800

900TMTb

amplitude (deg)

vpea

k (d

eg/s

ec)

data PFfitted curve PFdata IRfitted curve IR

Fig. 11. Main sequence amplitude versus peak velocity. Fitted curve were calculated by non-linear least squares algorithm with fit functionvpeak ¼maxðvpeakÞ � ð1� eð�amplitude=cÞÞ, where c is a constant in (1,2) (Baloh et al., 1975; Gangemi et al., 1991). Root mean square error (rmse) was:rmseTMT ¼ 72:45PF; 78:01IR and rmseTMTb ¼ 75:80PF; 81:10IR. PF and IR fitted curves are different both in TMT and TMTb.

filter

IRPF

med

ian

peak

sacc

ade

(deg

/sec

)

550,0000

500,0000

450,0000

400,0000

350,0000

300,0000

250,0000

filter

IRPF

TMTbTMT109

8765

4321subject

Fig. 12. Median of Vpeak per each subjects in TMT and TMTb. On TMT experiment height of 10 subjects increased the saccade velocity. On TMTb nine of 10 subjects increasedthe saccade velocity.

G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563 1561

6. General discussion

In our experiment, people made saccades toward the targetinfluenced by the texture hole feedback and ‘‘adapted” the explora-tion according to display. Recent results suggest that saccade kine-matics are not stereotypical; for example, monkeys that make asaccade to a remembered target location have higher saccadevelocities and shorter durations when that target is also associatedwith a food reward (Takikawa, Kawagoe, Itoh, Nakahara, & Hiko-saka, 2002). Xu-Wilson, Zee, and Shadmehr (2009) provided evi-dence that the value of visual information appears to have aninfluence on the motor commands that guide saccades.

Optimal control framework (Harris & Wolpert, 2006; Todorov,2005) states that the trajectory of saccades depends on the visualstimulus. In this theory, the trajectory of a saccade is modulated

by two costs: cost associated with the motor commands, and costassociated with the time that passes before the target is foveated;brain adapts saccade in order to reduce motor command costs. Inour experiment subjects might have changed peak velocity in orderto reduce cost to move the texture hole; this action, however, didnot cause any problem in performing the task and subjects wereable to complete the sequence.

Our results also reported that the alteration in the peak velocitywas the most significant aspects in the experiment with peripheralvision partially inhibited (TMTb), and this is a further verificationthat peripheral vision plays a determinant role in the saccadesplanning (Findlay et al., 2001).

Finally the comparison between the two filters, PDF and FIR, hasshown the need to pay particular attention to the development ofGCD.

1562 G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563

7. Conclusions

Wide-spread application of gaze-contingent displays has beenhindered by a number of difficulties: (a) although eye-trackingtechnology has been available for many years, these techniqueshave so far been too imprecise or too expensive; (b) real-time up-dates of the visual display synchronized with movements of theobserver’s gaze were needed; and (c) little was known about thebehavioral consequences of using GCD and how effective it wasin different applications including neuro-cognitive contexts(Duchowski, 2007).

Authors attempted to investigate the GCD of two differentpoints of view, human (perception) and technology and providedevidence of how a simple filtering technique, that is necessary tomaintain the stable scene, could influence the exploration of space.For this reason a new system based on a filter that works not onlyon user feedback, but attempts to predict behavior was proposed.

In particular, the study here makes two main contributions. Itprovides a fast and inexpensive GCD that can be used in real casestudies and produces GC tasks to test covert and overt attentionin clinical contexts, with a good compromise between reactivityand filtering tremor. This makes the method suitable for clinicalpurposes, as it may be used to study peripheral and central vision,and overt and covert attention in certain neurological disordersand eye movements abnormalities.

The second contribution is to provide an instrument that can beconsidered not only a technological application but also an interac-tive system through human–computer interface. In the latterapplication, the system exploits its ability to take into account hu-man perception capacities and their reaction time during cognitivetasks. This make the system particularly suitable for neuro-cogni-tive applications including cases with neurological abnormalitiesaffecting visual search.

Further extensions could be the application of more complexprediction components which consider fixations made on latest1000 ms, in order to improve the engagement component andmodulate the FIR window.

Finally, the FIR filter can be replaced by a more complex filter, andthe time window, where it operates, can be adapted to take into con-sideration the correlation between dispersions and previous errors.

Appendix A. I-DT

For a fixation to be valid under the distance dispersion algo-rithm, each point in that fixation must be no further than some dis-persion threshold from every other point.

Algorithm 1. Dispersion algorithm I-DT

1:

while there are points (x,y) in the queue do 2: initialize window over first points to cover duration

threshold

3: if dispersion of window points P than threshold then 4: add points to the window as long as

dispersion 6 threshold

5: Note fixation at centroid of window points 6: Remove window points from points 7: else 8: Remove first point from points 9: end if 10: end while

A maximum dispersion threshold is set at 40, according to

common parameters. Previous studies confirmed that the disper-sion algorithm is robust enough to identify fixation sequences(Falkmer, Dahlman, Dukic, BjSllmark, & Larsson, 2008).

Appendix B. PF

See algorithm for a short pseudocode of the proposedtechnique.

Algorithm 2. Predictive filter algorithm PF

1:

for all gaze point (x,y, t) do 2: put (x,y) on I-DT algorithm queue 3: retrieve previous fixation duration ðDtÞ 4: retrieve current fixation ðxf ; yf Þ 5: Dp maxðdistanceððx; yÞ; ðxf ; yf ÞÞ;DminÞ 6: if Dp 6 Dmax then 7: {Check if fixation has been started} 8: if Fixation has not been started then 9: tstart t 10: end if 11: {Evaluate W} 12: evaluate W1

13:

if t is in predicted duration fixation Dt then 14: W2 Dmax

15:

else 16: W2 0 17: end if 18: W maxðW2 �W1;0Þ 19: W minðW;WmaxÞ 20: else 21: W 0 22: end if 23: {Decision} 24: if W P 0 then 25: [Apply filter] ðxd; ydÞ FIRðx; y;WÞ 26: else 27: {Send data to filter to prevent filter bandwidth issue} 28: Put (x,y) on FIR queue 29: ðxd; ydÞ x; y 30: end if 31: show texture hole at ðxd; ydÞ 32: end for

The following formulas describe the algorithm:

W1 ¼Wmax � log10ðDp=DminÞ ðB:1Þ

when p is position and

Dp ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðxðtÞ � xf ðt � 1ÞÞ2 þ ðyðtÞ � yf ðt � 1ÞÞ2

q� �Dmin

ðB:2Þ

and

W2 ¼ Dmax : if tstart 6 t 6 tstart þ Dtlastfixationj0 ðB:3ÞW ¼ dW2 �W1e0 ðB:4Þ

where we defined the operator

daeb ¼maxða;bÞ ðB:5Þ

Eqs. (3) and (4) express window adaptation to dispersion; Dmin isthe minimum dispersion and Dmax is the maximum dispersion alsoused by the dispersion algorithm. Eq. (5) modulates the predictionof fixation duration. In a similar manner to the dispersion algo-rithm, a fixation is started when current dispersion is less thanthe maximum dispersion

tstart ¼ t : if DpðtÞ 6 Dmax and Dpðt � 1ÞP Dmax ðB:6Þ

In order to prevent any over estimation by Eq. (5) (duration predic-tion), W is abruptly dropped to 0 when:

DpðtÞP Dmax ðB:7Þ

G. Veneri et al. / Computers in Human Behavior 26 (2010) 1555–1563 1563

References

Baloh, R. W., Sills, A. W., Kumley, W. E., & Honrubia, V. (1975). Quantitativemeasurement of saccade amplitude, duration, and velocity. Neurology, 25,1065–1070.

Becker, W., & Fuchs, A. F. (1969). Further properties of the human saccadic system:Eye movements and correction saccades with and without visual fixationpoints. Vision Research, 9, 1247–1258.

Bhme, M., Dorr, M., Martinetz, T., & Barth, E. (2006). Gaze-contingent temporalfiltering of video. In Proceedings of the 2006 symposium on eye tracking researchand applications (San Diego, California, March 27–29, 2006). ETRA ’06. New York,NY: ACM.

Bowie, C. R., & Harvey, P. D. (2006). Administration and interpretation of the trailmaking test. Nature Protocols, 1, 2277–2281.

Carlos, H., & Morimoto, M. R. M. (2005). Eye gaze tracking techniques for interactiveapplications. Computer Vision and Image Understanding, 4–24. Special Issue onEye Detection and Tracking.

Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-drivenattention in the brain. Nature Reviews Neuroscience, 3, 201–215.

Duchowski, A. T. (2002). A breadth-first survey of eye-tracking applications.Behavior Research Methods Instruments and Computers, 34, 455–470.

Duchowski, A. T. (2007). Eye tracking methodology: Theory and practice (2nd ed.).Springer.

Falkmer, T., Dahlman, J., Dukic, T., BjSllmark, A., & Larsson, M. (2008). Fixationidentification in centroid versus start-point modes using eye-tracking data.Perceptual and Motor Skills, 106, 710–724.

Findlay, J. M., Brown, V., & Gilchrist, I. D. (2001). Saccade target selection in visualsearch: The effect of information from the previous fixation. Vision Research, 41,87–95.

Gangemi, P. F., Messori, A., Baldini, S., Parigi, A., Massi, S., & Zaccara, G. (1991).Comparison of two nonlinear models for fitting saccadic eye movement data.Computer Methods and Programs in Biomedicine, 34, 291–297.

Harris, C. M., & Wolpert, D. M. (2006). The main sequence of saccades optimizesspeed-accuracy trade-off. Biological Cybernetics, 95, 21–29.

Leigh, R. J., & Kennard, C. (2004). Using saccades as a research tool in the clinicalneurosciences. Brain, 127, 460–477.

Leigh R. J., & Zee, D. S. (2006). The neurology of eye movements (Book/DVD) (4th ed.).New York: Oxford University Press. <http://www.oup.com/us/catalog/general/subject/Medicine/Neurology//dmlldz11c2EmY2k9OTc4MDE5NTMwMDkwMQ==>.

McConkie, G., & Rayner, K. (1975). The span of the effective stimulus during afixation in reading. Perception and Psychophysics, 17, 578–586.

Murphy, H. A., Duchowski, A. T., & Tyrrell, R. A. (2009). Hybrid image/model-basedgaze-contingent rendering. ACM Transactions on Applied Perception, 5, 1–21.

Pomplun, M., Reingold, E. M., & Shen, J. (2001). Peripheral and parafoveal cueing andmasking effects on saccadic selectivity in a gaze-contingent window paradigm.Vision Research, 1, 12.

Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain.Annual Review of Neuroscience, 13, 25–42.

Posner, M. I., & Rothbart, M. K. (2007). Research on attention networks as a modelfor the integration of psychological science. Annual Review of Psychology, 58,1–23.

Rayner, K. (1978). Eye movements in reading and information processing.Psychological Bulletin, 85, 618–660.

Rayner, K., Li, X., Williams, C. C., Cave, K. R., & Well, A. D. (2007). Eye movementsduring information processing tasks: Individual differences and cultural effects.Vision Research, 47, 2714–2726.

Reichle, E. D., Rayner, K., & Pollatsek, A. (2003). The e-z reader model of eye-movement control in reading: Comparisons to other models. Behavioral andBrain Science, 26, 445–576 [discussion 477–526].

Reingold, E. M., & Loschky, L. C. (2002). Reduced saliency of peripheral targets ingaze-contingent multi-resolutional displays: Blended versus sharp boundarywindows. In Symposium on eye tracking research and applications (New Orleans,Louisiana, March 25–27, 2002). ETRA ’02 (pp. 89–93). New York, NY: ACM.<http://doi.acm.org/10.1145/507072.507091>.

Salvucci, D. D., & Goldberg, J. H. (2000). Identifying fixations and saccades in eye-tracking protocols. In ETRA ’00: Proceedings of the 2000 symposium on eyetracking research and applications (pp. 71–78). New York, NY, USA: ACM.

Simion, C., & Shimojo, S. (2006). Early interactions between orienting, visualsampling and decision making in facial preference. Vision Research, 46,3331–3335.

Takikawa, Y., Kawagoe, R., Itoh, H., Nakahara, H., & Hikosaka, O. (2002). Modulationof saccadic eye movements by predicted reward outcome. Experimental BrainResearch, 142, 284–291.

Todorov, E. (2005). Stochastic optimal control and estimation methods adapted tothe noise characteristics of the sensorimotor system. Neural Computation, 17,1084–1108.

Toet, A. (2006). Gaze directed displays as an enabling technology for attentionaware systems. Computers in Human Behavior, 22, 615–647 [Attention awaresystems – Special issue: Attention aware systems].

Vinnikov, M., Allison, R. S., & Swierad, D. (2008). Real-time simulation ofvisual defects with gaze-contingent display. In Symposium on eye trackingresearch and applications (Savannah, Georgia, March 26–28, 2008). ETRA ’08(pp. 127–130). ACM, New York, NY. <http://doi.acm.org/10.1145/1344471.1344504>.

Widdel, H. (1984). Operational problems in analysing eye movements. In A.G. Gale & F. Johnson (Eds.). Theoretical and applied aspects of eyemovement research (Vol. 1, pp. 21–29). North-Holland, BV: ElsevierScience Publishers.

Xu-Wilson, M., Zee, D. S., & Shadmehr, R. (2009). The intrinsic value of visualinformation affects saccade velocities. Experimental Brain Research, 196,475–481.

Yarbus, A. L. (1967). Eye movements and vision (Vol. VII). New York: Plenum Press.