Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL … · Title: The Effects of Engagement and Gamification on Attentional Bias Modification Author: Veronique Servant Date: June 2015

Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION

1

Title: The Effects of Engagement and Gamification on Attentional Bias Modification

Author: Veronique Servant

Date: June 2015

Institution name/journal where submitted: McGill University

The use of this database indicates agreement to the terms and conditions

Academia is a database that promotes the free exchange of ideas and scholarly work, setting a

platform on which to foment and improve student discourse

EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION

2

Abstract

Attentional bias modification training has been shown to be a reliable method of reducing

rejection bias. Recently, the effects of gamification and engagement have been explored in an

attempt to create engaging versions of mental health treatments that can be provided over the

internet. In this study, a dynamic version of the matrix attentional training task was created to

make it more engaging and test for more powerful effects of attentional training. Results showed

that the dynamic condition was more engaging than the static condition, but only relative to the

perceived ease of the task. Attentional training produced signficant results, however, only in the

static faces condition through improving acceptance bias. Future research is needed to explore

additional ways to make the task more engaging and to improve the effects of attentional

training.

Keywords: acceptance bias, attentional bias, attentional bias modification training,

engagement, gamification, rejection bias, visual search


3

A great deal of research has been conducted into finding effective treatment for individuals

with anxiety disorders and related vulnerabilities. One of the psychological constructs

underlying anxiety disorders is attentional bias. This is a bias individuals show in how they

attend to certain stimuli; for example, individuals who have high anxiety often demonstrate an

attentional bias toward threatening stimuli (Zvielli, Bernstein, & Koster, 2014), and individuals

with relatively low self-esteem demonstrate an attentional bias toward rejecting stimuli

(Dandeneau & Baldwin, 2004). Attentional bias has been measured across numerous studies

using modified versions of the visual attention dot probe paradigm developed by MacLeod,

Mathews, & Tata (1986). This visual probe task (VPT) task involves participants making a

neutral response (pressing a button) to a neutral stimulus (visual probe). MacLeod, Rutherford,

Campbell, Ebsworthy, & Holker (2002) conducted a study measuring attentional bias using a

VPT which involved displaying pairs of words (one emotionally negative, one neutral), after

which a probe would appear behind one of the words. Participants then had to identify the probe

(either a single pixel or two adjacent pixels). Participants’ reaction times were measured and

used to calculate their attentional bias based on the reasoning that reaction times would be

shorter when participants’ attention was drawn to the stimulus behind which the probe appeared.

An attentional bias to automatically attend to negative stimuli is thought to exacerbate the

negative feelings contributing to low self-esteem and/or high anxiety. This reasoning led to the

development of attentional bias modification, a process of training attention in such a way as to

remove the attentional bias and thereby eliminate its negative effects. MacLeod et al. (2002)

developed a modified attentional training paradigm to modify the attentional bias toward

negative stimuli exhibited by anxious individuals. The training task involved a VPT which

displayed pairs of words (one emotionally negative, one emotionally neutral) after which a probe


4

would appear behind one of the words. By consistently presenting the probe after the

emotionally neutral word, a pattern becomes evident whereby attention is automatically directed

toward future neutral stimuli where the probe is expected to appear. This task was successful in

training participants to display an attentional bias away from, instead of towards, negative

information. MacLeod & Clarke (2015) also conducted a study involving attentional bias

modification training with individuals suffering from high anxiety, and the attentional training

was successful in reducing attention to threat and anxiety symptomatology.

Research on attentional bias modification training has predominantly involved modified

dot probe paradigms, but alternative methods have also proven successful in modifying

attentional bias. For example, Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner

(2007) conducted an attention training task where participants were instructed to perform a

visual search for a specific image (a smiling face amongst frowning faces in the experimental

condition, or a 5-petalled flower amongst 7-petalled flowers in the control condition). The

experimental condition served to modify attentional bias, as participants had to inhibit attention

to the rejecting faces in order to find the accepting face. Individuals with low self-esteem

showed significant improvements in inhibiting attention to rejecting faces after completing the

attentional training, whereas individuals in the control condition did not show such

improvements (Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner, 2007).

Despite the demonstrated effects of attentional training, the procedure – particularly the

modified dot probe task – has often been described by participants as being repetitive, boring, or

frustrating (Beard, Weisberg, & Primack, 2012). These negative feelings could lead participants

to become less involved in the task and persist less in the training, which would make it less


5

useful as an intervention. It is therefore important to ensure that participants remain engaged and

participate in the full extent of the attentional training.

Engagement

A great deal of research examining the educational effects and influences of engagement

has focused on the process of learning and the role of student engagement. An early model of

student engagement by Finn (1989, 1993) included affective (e.g. sense of belonging) and

behavioral (e.g. involvement in school) components. Research has since examined multiple

subtypes of engagement, including cognitive engagement, behavioral engagement, and emotional

or psychological engagement. Cognitive engagement has been found to be associated with

processes such as intrinsic motivation, adaptive learning strategies involving task-mastery goals,

and self-regulation (Meece, Blumenfield, & Hoyle, 1988; Appleton, Christenson, Kim, &

Reschly, 2006). Fredricks, Blumenfield, & Paris (2004) define behavioral engagement as

relating to participation and involvement in school activities, differing from emotional

engagement, which draws on the valence of social connections in the environment (such as

relations with teachers, peers, and the school). In a study examining 1st-grade children, Hughes

& Kwok (2007) found that increased classroom activity resulted in increased behavioral

engagement and was associated with higher reading achievement. Goodenow (1993) conducted

a study examining psychological engagement amongst adolescent students and found that

psychological engagement was associated with positive characteristics such as greater class

participation, increased class attendance, and greater persistence in challenging activities.

Student engagement has been recognized as an important contributor to the process of

learning. Prensky (2005) compares children of even a few decades ago to children today,

illustrating a profound difference in attitudes towards learning at school. Children today are


6

exposed to a myriad of devices that provide entertainment on a daily basis, causing them to

develop the expectation and the need to be highly engaged by their surroundings. In the typical

school context involving a teacher giving a lecture, children have difficulty paying attention and

getting into the process of learning if the material is not inherently interesting and presented in an

engaging fashion (Prensky, 2005). Educators are now faced with the challenge of modifying

their teaching styles to incorporate “the same combination of desirable goals, interesting choices,

immediate and useful feedback, and opportunities […] that engage kids in their favorite complex

computer games” (Prensky, 2005). To address this change in learning style, research has

examined engagement through active learning. Students engaging in active learning processes,

such as the use of clickers, display higher levels of class involvement, which in turn results in

better preparation, increased attention, and better recall of class material (Caldwell, 2007). A

study by Freeman et al. (2007) also examined the effects of engaging students through active

learning (responding to daily multiple-choice questions with clickers or cards) in a course on

introduction to biology. They found that students in the engaging course design had lower

failure rates and scored higher on both an identical midterm and the final exam compared to

students who had taken the course with the same professor in the past (Freeman et al., 2007).

Gamification

The importance of engagement and the consequential need for processes such as

gamification stem from the increased presence of media and technologies such as video games

and smart phones in everyday life. In recent years, a great deal of research has been conducted

regarding the process of gamification, a method of transforming services by adding game

components and elements in non-game contexts (Deterding, Dixon, Khaled, & Nacke, 2011).

Goasduff & Pettey (2011) indicated that gamification is used as a means to achieve increased


7

levels of engagement, influence positive behaviors, and stimulate innovation. They identified

four key means by which gamification can increase engagement: (a) accelerated feedback cycles;

(b) clear goals and rules of play; (c) a compelling narrative; and (d) challenging but achievable

tasks. Gartner predicted that by 2015, over 50 percent of organizations managing innovation

processes will undergo the gamification transformation (Goasduff & Pettey, 2011). In an

analysis of gamification processes, Muntean (2011) illustrates how engagement can be increased

through the application of game mechanics and dynamics to learning tasks and processes.

Therapeutic Applications

Gamification and engagement offer promising and necessary methods to provide mental

health support via contemporary methods such as the internet. A study conducted by the Kraiser

Family Foundation (Rideout & Roberts, 2010) found that children and teenagers, ages 8-18,

spend an average of 7 hours and 38 minutes using entertainment media devices (such as

computers and smart phones) per day, which equates to over 53 hours per week. On a similar

note, another study found that 37% of teens had smartphones in 2013, a sharp increase from the

23% of teens who had smartphones in 2011 (Madden, Lenhart, Duggan, Cortesi, & Grasser,

2013). This surge in the presence of smartphones is also apparent in the adult population:

Duggan (2013) found that 91% of American adults own cell phones, where 60% of adults use

their phone to access the internet and 50% of adults use their phone to download apps.

While such devices are used primarily for entertainment and social networking purposes,

researchers are looking into ways to use these devices to confront a major obstacle in the mental

health industry: unmet need for treatment. A recent epidemiological study examined the

prevalence of mental health disorders among American adolescents and compared the rates of

mental health services sought, and the results were striking: Less than one in five adolescents


8

suffering from disorders involving anxiety, eating, or substance use received services

(Merikangas et al., 2011). In a separate study examining a sample of individuals in the United

States with major depression, 61.7% reported unmet need for treatment, 46.0% of whom listed

financial concern as the reason for not seeking treatment (Mojtabai, 2009). Converting mental

health services and treatments into forms applicable to the internet and mobile devices would

provide easy and affordable access to many individuals with unmet need for treatment.

Gamification has been one of the processes by which researchers have sought to create

online and mobile versions of mental health treatments. For example, Dennis & O’Toole (2013)

created a mobile gamified version of the attentional bias modification training task based on the

dot-probe paradigm. They designed a game on the iPod Touch where players, through tracing

certain characters on the screen with their finger, receive either attentional bias training or

placebo training. Their study involved 78 individuals high in trait anxiety, and results showed

that long training conditions (45 minutes of gameplay) but not short training conditions (25

minutes of gameplay) showed significant reductions in cognitive processes implicated in

attentional bias to threat (Dennis & O’Toole, 2013). A separate study by Enock, Hofmann, &

McNally (2014) created a mobile version of the dot-probe task and sought to determine whether

it was possible to administer attentional bias modification via smartphones. Their study

succeeded in demonstrating that it was feasible to conduct attentional bias modification via

smartphones.

Toward a Gamified Version of the Matrix Visual Search Task

The present study aims to analyze the effects of delivering via internet a gamified version

of the matrix attentional bias training task presented by Dandeneau, Baldwin, Baccus,

Sakellaropoulo, & Pruessner (2007) in order to determine whether increased engagement


9

strengthens the effects of the attentional training. We created a novel modified version of the

attentional training matrix by adding more game-like features and characteristics, such as

alternating matrix arrays, image flips, and level rotations. The goal of gamifying the attentional

matrix was to increase the engaging properties of the task. We hypothesized that the gamified

version would be more engaging than the standard version, based on participants’ perception of

how engaged they felt during the task. This was measured by a novel self-report questionnaire

of various items related to engagement. We expected to find this effect for both types of stimuli

of the task; specifically, that the gamified version of the matrix would be more engaging in both

the experimental stimuli and control stimuli conditions. We also hypothesized that the gamified

version, through being more engaging, would lead to more powerful effects of attentional bias

modification. This was determined by measuring the changes in attentional bias modification

and comparing the outcomes of each version of the task. Specifically, we expected that the

gamified version of the experimental stimuli matrix would show greater effects of attentional

bias modification than the standard version of the experimental stimuli matrix. No differences

were expected in either the gamified or standard version of the control stimuli matrix, as the

control stimuli have no effects in training attentional bias.

Methods

Participants

Participants were 78 undergraduate students from McGill University. 61 students

participated in the study to receive course credit, and 17 additional students not registered in the

subject pool were also recruited offering financial compensation of $10 CDN per participant who

completed the study. Of the 78 students who participated, 9 were male, 68 were female, and 1


10

preferred not to disclose their gender. The mean age of the participants was 20.65 years (range =

18-25).

Participants were randomly assigned to one of four matrix conditions. The final

breakdown included 20 assigned to the static flowers condition (19 female, 1 preferred not to

disclose); 15 to the dynamic flowers condition (3 male, 12 female); 19 to the static faces

condition (2 male, 17 female); and 24 to the dynamic faces condition (4 male, 20 female).

Measures

A number of questionnaires were implemented to measure various personality

characteristics. The study was broken up into 5 sections: (a) pre-training questionnaire1; (b) pre-

training visual probe task; (c) attentional bias training matrix; (d) post-training visual probe task;

and (e) post-training questionnaire1,2.

Engagement scale. The engagement scale was a novel measure created for this study to

measure participants’ perceived levels of engagement during the matrix attentional training task.

Questions were written based on the components of being immersed in a task, such as a sense of

time being distorted and a balance between perceived efficacy and task challenge

(Csikszentmihalyi, 1997), as well as the characteristics which make a game engaging, such as

clear rules and goals in addition to a sense of enjoyment (Prensky, 2001). The scale consisted of

10 items, each ranked on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly

agree). Participants were asked to think back to when they were performing the matrix task, and

to indicate how much they agreed or disagreed with each item. Items included statements such

as “I found it easy to do” and “I felt I was doing poorly at it” for measuring perceived efficacy,

1 No significant effects of any of the measures included in the questionnaires were found; as such, they will not be

further discussed. 2 Anagrams were used as a threat manipulation, but no significant effects of the task were found, so they will not be

further discussed.


11

and statements such as “I had fun doing it”, “it was boring” (reverse-scored item), and “I felt

completely involved in it” for measuring how immersed participants felt in the task.

Visual Probe Task

The visual probe task (VPT) was based on the well-established method of examining

attentional biases in anxious individuals (Bradley, Mogg, Falla, & Hamilton, 1998). The stimuli

included 32 images of faces which were procured from 16 different individuals. For each

individual, two images of different facial expressions were obtained: one neutral pose and one of

either a smiling (acceptance) or a frowning (rejection) pose. The two images presented on the

screen were of the same person and included either one of each expression or two copies of the

neutral expression. This could happen in one of five arrangements: frowning-neutral, neutral-

frowning, neutral-neutral, neutral-smiling, or smiling-neutral. This resulted in 8 acceptance-

neutral pairs, 8 rejection-neutral pairs, and 16 neutral-neutral pairs. All pictures were in color

and were presented against a white background.

The VPT consisted of 8 practice trials and 80 experimental trials that were presented in

random order for each participant. For each trial, the fixation symbol (the plus sign, +) was

presented in the center of the screen for 500 ms. Following the fixation period, a pair of faces

was displayed for 500 ms, after which a directional probe (an arrow pointing either up or down)

appeared behind either the picture on the left or the picture on the right. The probe remained on

the screen until the participant responded by pressing a key on the keyboard to indicate the

appropriate orientation of the probe (up arrow key for a probe pointing up; down arrow key for a

probe pointing down). Throughout the task, each of the 16 emotional-neutral pairs were

presented four times, differing by the location of the emotional face (left or right) and by the

orientation of the probe (up or down). Each neutral-neutral pair was presented only once. Each


12

probe orientation replaced an equal number of emotional and neutral pictures on each side of the

screen. Figure 1 illustrates the sequence of images for one trial.

As participants completed the study at their own convenience, it was assumed that screen

size and resolution would vary between participants. For this reason, prior to commencing the

task, participants were instructed to modify the size of a line in the middle of the screen by either

shortening or lengthening it so that it was approximately 4 inches long. This served to modify

the size of the display so the image sizes would conform to the indented parameters based on the

design of the VPT conducted by Dandeneau et al. (2007). Images were 45 x 70 mm with a

distance of 115 mm between their centers. Following this, participants were shown a screen with

instructions explaining that they were to fixate on the plus sign (+) and then indicate, as quickly

as possible, the orientation of the probe. Lastly, participants were instructed to set themselves up

such that their eyes were approximately 60 cm from the screen. Once ready, participants then

went through the 8 practice trials. Following this, a short instruction page was given to inform

participants that faces would be shown following the fixation sign, but that participants should

do their best to ignore the faces and to focus on correctly identifying the orientation of the probe.

Attentional Bias Training Matrix

The attentional bias training matrix (ABTM) was based on previous research examining

the attentional training effects of the matrix task (Dandeneau & Badwin, 2004).

Levels and Steps. The ABTM contained 12 different levels. Each level had a predefined

sequence of steps, which included code defining how the images would be displayed in the

matrix. The code included the following variables:

Pattern. This indicated which pre-defined array pattern would be used (e.g. 1 row by 3

columns).


13

Smiles. This indicated the number of smiles to be populated within the matrix.

Smiles_reset. This indicated the number of smiles to be clicked in order to advance to

the next step.

Reset_time. This indicated the amount of time, in seconds, after which the matrix would

advance to the next step if the participant had not yet responded (i.e. a smile had not yet

been clicked).

Flip_chance. This indicated the percent chance that any image in the pattern would

perform a rotating flip action.

Cycle_time. This indicated the amount of time, in seconds, between chances for images

to flip.

Flip_smile. This indicated the percent chance that any image, after flipping, would

change into a smiling face.

Each level lasted 30 seconds, during which its sequence of steps would be repeated until the time

limit was reached. After reaching the time limit, the matrix would automatically switch to and

load the following level. Levels were presented in continuous blocks of 4 with a break of 10

seconds in between each block.

Matrix stimuli: faces and flowers. The experimental and control conditions were based

on the type of stimuli displayed in the matrix: faces or flowers. Images of faces were used in the

experimental condition. The stimuli consisted of 144 images of faces procured from 48 different

individuals. For each individual, two images of different facial expressions were obtained: a

smiling (acceptance) pose and a frowning (rejection) pose. The task was to search for and click

on a smiling face presented among frowning faces.


14

Images of flowers were used in the control condition. The stimuli consisted of 144 images

of flowers. Flowers consisted of either 5 petals or 7 petals, and differed based on how many

petals were colored in. The task was to search for and click on the 5-petaled flower presented

among other 7-petaled flowers.

Matrix design: static and dynamic. Two different types of matrices were employed:

static and dynamic. The static design was a replication of the matrix visual search task in

Dandeneau et al. (2007). In the static condition, each level comprised only one type of step. The

pattern was set to 4 rows by 4 columns. Of the 16 images, 15 were non-target images

(neutral/frowning face or 7-petaled flower), and only 1 was a target image (smiling face or 5-

petaled flower). The step would be completed only when the participant successfully located

and clicked on the target. Participants would repeat the step for as long as they clicked on the

target image, until the level duration was over and the next level was loaded.

In the dynamic condition, levels 1 and 12 consisted of one type of step containing a 4 x 4

matrix (see Figure 2), whereas levels 2 through 11 consisted of multiple steps, each with unique

sequences of different array arrangements ranging from 1 x 2 to 5 x 5 (see Figures 3 and 4).

Each level had different features regarding flips, cycle time, and reset time, with most steps

automatically advancing after 5 seconds. The goal of the dynamic matrix was to make the task

more engaging by adding flips to images and through automatically advancing to the next step

after the participant had not made a response after 5 seconds.

Procedure

Participants were given the URL of the online study and instructed to complete the study at

their convenience, with the requirement of a computer with a working keyboard and mouse. The

URL directed participants to the pre-training questionnaire. After completing the pre-training


15

questionnaire, participants were directed to the pre-training VPT. Once having completed the

VPT, participants were randomly assigned to one of 4 ABTM conditions. Following completion

of the ABTM, participants were directed to the post-training VPT. After completing the second

VPT, participants were directed to the post-training questionnaire. At the end of the second

questionnaire, participants were thanked for their time and redirected to a debriefing page.

Results

Data analysis

Calculation of attentional bias scores. The VPT was used to measure the participant’s

attentional bias – specifically, acceptance bias and rejection bias. Only the trials in which the

participant correctly identified the probe would contribute to the bias, thus all trials in which the

participant made an error were discarded (2.96% of all data). A maximum cut-off RT was

calculated for each participant by calculating 2 standard deviations above their mean RT, and

any RT above the cut-off was replaced by the cut-off instead. In addition, any reaction time

lower than 200 ms was removed. This was done to remove the effect of any outliers in which the

participant had an extraordinarily short or long reaction time. One participant was excluded

from the study due to having numerous trials with RTs above 4000 ms.

Rejection scores were calculated for each VPT by subtracting the mean score of valid

rejection trials (where the probe was behind the rejecting face) from the mean score of invalid

rejection trials (where the probe was behind the non-rejecting face). A positive rejection bias

score indicates an attentional bias toward rejecting faces, whereas a negative rejection bias score

indicates a bias of disengaging (looking away) from rejecting faces. A change in rejection bias

was calculated by subtracting the rejection bias score of the second VPT from that of the first.

Similarly, acceptance scores were calculated by subtracting the mean score of valid acceptance


16

trials from the mean score of invalid acceptance trials. A positive acceptance bias score indicates

an attentional bias toward accepting faces, whereas a negative acceptance bias score indicates a

bias of disengaging from accepting faces. A change in acceptance bias was calculated by

subtracting the acceptance bias score of the second VPT from that of the first.

Engagement was measured by a novel scale created to encompass various aspects of what

makes a task engaging. Preliminary analyses suggested that the scale was not measuring a single

construct. A factor analysis was performed on the 10-item scale, which yielded two separate

factors, one loading on two of the questions, and another loading on the remaining 8 questions.

Two engagement scores were computed based on these two factors. An ANOVA yielded a

significant difference in the 2-item engagement factor between the static (M = 2.62, SD = 0.90)

and dynamic (M = 3.26, SD = 0.91) conditions of the matrix, F(1,74) = 8.344, p = .005 (see

Figure 5). No significant differences were found in the 8-item engagement factor between the

static (M= 2.89, SD = 0.75) and dynamic (M = 3.06, SD = 0.63) conditions, F(1,74) = 1.148, p =

.287. These results illustrate that the dynamic condition was found to be more engaging than the

static condition only in relation to the two-item subscale of the engagement score.

It is important to note that the two questions in this subscale were related to the difficulty

of the task as perceived by the participant (“I found it easy to do” and “I felt I was doing poorly

at it”), whereas the remaining 8 questions measured other concepts such as the amount of fun

and involvement experienced during the task. Thus the dynamic condition was more engaging

than the static condition, but only in the sense that the dynamic condition was perceived as easier

than the static condition.


17

Matrix Scores

Two matrix-performance scores were calculated: the total number of hits (targets clicked)

and the total number of errors (non-targets clicked). Two participants were excluded from the

study for having a total number of errors that exceeded 100. An ANOVA was performed to

compare the differences in number of hits, and a significant difference was found between the

static (M = 88.08, SD = 26.69) and dynamic (M = 199.78, SD = 27.22) conditions, F(1,74) =

8.822, p = .004. A second ANOVA yielded a significant difference in the number of errors

between the static (M = 5.52, SD = 6.48) and dynamic (M = 9.63, SD = 8.06) conditions, F(1,74)

= 7.408, p = .008. Thus participants in the dynamic condition were performing a greater number

of visual searches overall, as illustrated by a higher number of both hits and errors compared to

the static condition. Correlations were performed to examine the aforementioned differences. A

positive correlation was found between the 2-item engagement factor and hit score, r(76) =

0.347, p = .002. This supports participants’ self-reports of level of difficulty of the task, as the

participants who found it easier were attaining higher scores.

Changes in Acceptance Bias

An ANOVA was performed to compare changes in rejection bias, and contrary to

expectations, no significant differences were found between the faces and flowers groups,

F(1,74) = 0.71, p = .401. A second ANOVA was conducted to examine changes in acceptance

bias, yielding a significant difference between the static (M = 15.62) and dynamic (M = -5.56)

conditions of the matrix, F(1,74) = 5.26, p = .025 (see Figure 6). The same ANOVA yielded an

interaction between matrix design and stimuli that fell just short of significance, F(1,74) = 3.94,

p = .051. Independent samples t-tests were conducted to compare the change in acceptance bias

for flowers and faces in the dynamic and static matrix designs. There was a significant


18

difference in acceptance bias change between the dynamic face (M = -12.15, SD = 36.81) and

static face (M = 24.28, SD = 40.41) conditions, t(41) = 3.097, p = .004. There was no significant

difference in the flowers conditions, for which acceptance change scores were very close to zero,

t(33) = 0.217, p = .830. These results indicate that the flowers condition yielded no change in

attentional bias, whereas the static faces condition did yield a significant change in attentional

bias, specifically in acceptance bias. To more directly assess change in bias, paired samples t-

tests were conducted to compare the acceptance bias values preceding and following the matrix

task. There was a significant change in acceptance bias in the static face condition, t(18) = -

2.636, p = .017. No significant changes in acceptance bias were found in the other conditions.

Altogether, the results illustrate that only the static faces condition showed significant

improvement in acceptance bias.

Correlations were performed to further analyze the significant improvement in acceptance

bias observed in the static faces condition. A positive correlation was found between the 2-item

engagement factor and change in acceptance bias, r(17) = 0.595, p = .007. Thus, in the static

faces condition, the easier the participants perceived the task, the greater the change in their

acceptance bias. This correlation was nonsignificant in the other three conditions.

Discussion

The present study aimed to examine the effects of a gamified version of the matrix

attentional bias modification task (Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner,

2007). It was hypothesized that the gamified version would be more engaging than the standard

version, and that the gamified version, through greater engagement, would yield more powerful

effects of attentional bias modification. The gamified version was found to be significantly more

engaging, but only relative to self-reported perceptions regarding the ease of the task. In


19

addition, changes in attentional bias were only observed in the standard face condition, which

contradicted expectations of engagement improving effects of the attentional bias training.

Matrix Score and Engagement

Participants in the dynamic conditions of the matrix were found to have significantly

higher scores in the matrix than those in the static conditions. This is presumably due to the fact

that in the static condition, each trial had only one level which displayed a ratio of 1 smile to 16

frowns, whereas in the dynamic condition, trials had different levels with ratios of smiles to

frowns varying from 1:1 to 1:15 with an average ratio of 1:4.89. Thus participants in the

dynamic condition, having fewer stimuli to search, could more quickly and more easily locate

the target stimulus and advance to the next step. Higher hit scores correlated positively with

both overall engagement, the 2-item engagement factor, and the 8-item engagement factor, and

higher error scores correlated negatively with the 8-item engagement factor. These results show

that the better the participants did on the task (i.e. getting higher hits and less errors), the more

they perceived the task as both easier and as being more involving and fun.

How Engagement Affects Change in Acceptance Bias

We found that, overall, the dynamic condition was perceived as easier than the static

condition. We also found that the static face condition was the only condition in which

participants showed improvements in acceptance bias. This would suggest that the easiness

factor of engagement may actually serve to hinder attentional training. However, within the

static face condition, the 2-item engagement factor correlated positively with change in

acceptance bias, and acceptance bias correlated positively with the number of errors. This

suggests that the effects of the visual search task do not rely solely on feelings of success, as it

was not sufficient for participants in the static faces condition to simply score more hits without


20

any errors. Instead, a sense of accomplishment in the face of challenge appears to be one of the

important mechanisms underlying the effects of the training task. This would also explain why

the dynamic face condition did not experience any change in acceptance bias; despite the fact

that participants in the task perceived the task as easy, the task itself was of a lower level of

difficulty, and thus there was no element of challenge to overcome. However, as there was no

effect of attentional training in the dynamic flowers condition, overcoming a challenging task

and gaining a sense of accomplishment cannot be the only relevant factor underlying the effects

of attentional training. It is clear that both a challenging nature and emotional stimuli are critical

features in order for the visual search task to be effective in modifying attentional bias.

The dynamic version of the matrix was not found to be engaging in the manner assessed by

the 8-item engagement factor. The dynamic characteristics of the matrix were created to involve

the participant more through the use of moving images and alternating patterns of stimuli arrays.

A lack of self-reported engagement suggests that increased attention alone is not sufficient to

engage participants in the task. It may prove effective to alter the characteristics and features of

the dynamic matrix so as to increase the various subtypes of engagement (for example, adding a

social component, such as working with a team, to increase emotional engagement). It is also

possible that our engagement scale was not a valid measure of engagement. The scale was

untested prior to this study, and thus had no evidence of being an accurate measure of task

engagement. Additional research is needed to explore alternative methods of increasing the level

of engagement of the matrix task in addition to improved methods to measure task engagement.

Challenging Critiques of Mechanisms Underlying Attentional Bias Training

The mechanisms underlying the effects of attentional training are unclear; however, results

from attentional training studies continue to provide insight into potential explanations. For


21

example, Dandeneau et al. (2007) sought to examine the mechanisms behind the matrix

attentional training task and to determine whether the effects of the attentional training might be

desensitization to negative stimuli as opposed to modification of attentional bias. Subjects were

randomly assigned to three different conditions: (a) attentional bias modification; (b) exposure

(in which the subjects were shown the same stimuli, but not required to perform any action); and

(c) control. The researchers hypothesized that if the effects of the attentional training were due

to desensitization, the attentional bias modification and exposure conditions would show similar

results. Their results showed that individuals with low self-esteem in the attentional training

condition showed a decrease in attentional bias to rejection information, whereas individuals

with low self-esteem in the exposure condition displayed an increase in this bias. They

concluded that the effects of the attentional training task is not due to desensitization, and an

active engagement is required to produce the modification of attentional bias.

The design of our study, using two faces conditions and two flowers conditions, allow us to

address certain questions that have been raised concerning attentional bias modification. For

example, the matrix task has been criticized for the use of flower stimuli as a control group, with

the assumption that the use of facial stimuli is what accounts for the change in attentional bias in

the dot probe task. Our findings disprove this assumption, as participants in the dynamic face

condition did not experience improvements in acceptance bias change (and in fact showed

tendencies toward reduced acceptance bias). This provides evidence against the belief that the

flowers serve as a poor control group, as it is not simply the facial characteristics of the stimuli

that account for the change in attentional bias. On a similar note, one might criticize the

mechanism underlying the attentional training task, suggesting that the change in attentional bias

could be an effect of reinforcement; the more targets successfully clicked, the more the search


22

for that specific target is reinforced. Again, the fact that participants in the dynamic face

condition did not experience a positive change in acceptance bias disproves this theory. Thus

reinforcement alone is not a mechanism underlying attentional bias modification.

Limitations and Conclusions

A few limitations should be considered when interpreting the results of this study. First, the

study was conducted entirely online, thus participants completed the study at their own

convenience. As such, there was no way to observe whether participants completed the study in

one sitting or whether participants were multi-tasking and performing other tasks while

completing the study. Distractions or prolonged periods of breaks in the midst of the study may

have potentially affected participants’ results. A second potential limitation involved the

duration of the attentional training task. Participants completed 12 levels of 30 seconds each,

with two 10-second breaks in between, for a total of six minutes of attentional training. The

length of the attentional training session mirrored the training in the study by Dandeneau et al.

(2007) and was kept relatively short in order to prevent participants from growing bored of the

repetitive task; however, this length is still much shorter than the typical attentional training dot-

probe paradigm. Additional research is needed to explore the effects of longer training sessions

in order to determine whether the length of the training makes a difference in terms of producing

stronger effects of attentional training.

The gamified version of the matrix visual search task was engaging in the sense that it was

easy, but it did not appear to succeed in engaging participants in terms of a general sense of task

involvement and enjoyment. The gamified version also contradicted our expectations in terms of

the effects it would have in bolstering the effects of the attentional bias training. Not only did

the gamified version fail to enhance the attentional bias training, but it also suggested tendencies


23

towards reductions in acceptance bias. However, the results did serve to analyze the underlying

mechanisms of attentional bias training. Perceiving the task as challenging and attaining a sense

of accomplishment appear to be important factors that drive the effects of attentional training.

Future gamification attempts might look to combine these two factors with features adapted to

support the various subtypes of engagement in order to produce a truly engaging and effective

task.


24

References

Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and

psychological engagement: Validation of the student engagement instrument. Journal of

School Psychology, 4(5), 427-445. doi:10.1016/j.jsp.2006.04.002

Beard, C., Weisberg, R. B., & Primack, J. (2012). Socially anxious primary care patients’

attitudes toward cognitive bias modification (CBM): A qualitative study. Behavioral and

Cognitive Psychotherapy, 40(5), 618-633. doi:10.1017/S1352465811000671

Bradley, B. P., Mogg, K., Falla, S. J., & Hamilton, L. R. (1998). Attentional bias for threatening

facial expressions in anxiety: Manipulation of stimulus duration. Cognition and Emotion,

12(6), 737-753. doi:10.1080/026999398379411

Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips.

CBE-Life Sciences Education, 6(1), 9-20. doi:10.1187/cbe.06–12–0205

Csikszentmihalyi, M. (1997). Flow and the psychology of discovery and invention. Harper

Perennial. New York. Retrieved from http://www.bioenterprise.ca/docs/creativity-by-

mihaly-csikszentmihalyi.pdf

Dandeneau, S. D., & Badwin, M. W. (2004). The inhibition of socially rejecting information

among people with high versus low self-esteem: The role of attentional bias and the

effects of bias reduction training. Journal of Social and Clinical Psychology, 23(4), 584-

602. Retrieved from

http://www.psych.mcgill.ca/perpg/fac/baldwin/dandeneaubaldwin.pdf

Dandeneau, S. D., Baldwin, M. W., Baccus, J. R., Sakellaropoulo, M., & Pruessner, J. C. (2007).

Cutting stress off at the pass: Reducing vigilance and responsiveness to social threat by


25

manipulating attention. Journal of Personality and Social Psychology, 93(4), 651-666.

doi:10.1037/0022-3514.93.4.651

Dennis, T. A., & O'Toole, L. J. (2014). Mental health on the go: Effects of a gamified attention-

bias modification mobile application in trait-anxious individuals. Clinical Psychological

Science, 2167702614522228. doi:10.1177/2167702614522228

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to

gamefulness: Defining "gamification". In Proceedings of the 15th International

Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9-15).

doi:10.1145/2181037.2181040

Duggan, M. (2013). Cell phone activities 2013. Pew research internet project. Retrieved from

http://www.pewinternet.org/files/old-

media//Files/Reports/2013/PIP_Cell%20Phone%20Activities%20May%202013.pdf

Enock, P. M., Hofmann, S. G., & McNally, R. J. (2014). Attention bias modification training via

smartphone to reduce social anxiety: A randomized, controlled multi-session experiment.

Cognitive Therapy and Research, 38(2), 200-216. doi:10.1007/s10608-014-9606-z

Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59(2), 117-142.

doi:10.3102/00346543059002117

Finn, J. D. (1993). School engagement & students at risk. Washington, DC: National Center for

Education Statistics. Retrieved from http://files.eric.ed.gov/fulltext/ED362322.pdf

Fredricks, J. A., Blumenfield, P. C., & Paris, A. H. (2004). School engagement: Potential of the

concept, state of the evidence. Review of Educational Research, 74(1), 59-109.

doi:10.3102/00346543074001059


26

Freeman, S., O'Connor, E., Parks, J. W., Cunningham, M., Hurley, D., Haak, D., . . . Wenderoth,

M. P. (2007). Prescribed active learning increases performance in introductory biology.

CBE Life Sciences Education, 6(2), 132-139. doi:10.1187/cbe.06-09-0194

Goasduff, L., & Pettey, C. (2011). Gartner says by 2015, more than 50 percent of organizations

that manage innovation processes will gamify those processes. Gartner Press Release.

Retrieved from http://www.gartner.com/newsroom/id/1629214

Hughes, J., & Kwok, O.-m. (2007). Influence of student-teacher and parent-teacher relationships

on lower achieving readers' engagement and achievement in primary grades. Journal of

Educational Psychology, 99(1), 39-51. doi:10.1037/0022-0663.99.1.39

MacLeod, C., & Clarke, P. J. (2015). The attentional bias modification approach to anxiety

intervention. Clinical Psychological Science, 3(1), 58-78.

doi:10.1177/2167702614560749

MacLeod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal

of Abnormal Psychology, 95(1), 15-20. Retrieved from

http://psycnet.apa.org/journals/abn/95/1/15.pdf

MacLeod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective

attention and emotion vulnerability: Assessing the causal basis of their association

through the experimental manipulation of attentional bias. Journal of Abnormal

Psychology, 111(1), 107-123. doi:10.1037//0021-843X.111.1.107

Madden, M., Lenhart, A., Duggan, M., Cortesi, S., & Grasser, U. (2013). Teens and technology

2013. Retrieved from http://www.pewinternet.org/files/old-

media//Files/Reports/2013/PIP_TeensandTechnology2013.pdf


27

Meece, J. L., Blumenfield, P. C., & Hoyle, R. H. (1988). Students' goal orientations and

cognitive engagement in classsroom activities. Journal of Educational Psychology, 80(4),

514-523. doi:10.1037/0022-0663.80.4.514

Merikangas, K. R., He, J.-p., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., . . . Olfson, M.

(2011). Service utilization for lifetime mental disorders in U.S. adolescents: Results of

the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the

American Academy of Child & Adolescent Psychiatry, 50(1), 32-45.

doi:10.1016/j.jaac.2010.10.006

Mojtabai, R. (2009). Unmet need for treatment of major depression the United States.

Psychiatric Services, 60(3), 297-305. doi:10.1176/ps.2009.60.3.297

Muntean, C. I. (2011). Raising engagement in e-learning through gamification. In The 6th

International Conference on Virtual Learning ICVL (pp. 323-329). Retrieved from

http://icvl.eu/2011/disc/icvl/documente/pdf/met/ICVL_ModelsAndMethodologies_paper

42.pdf

Prensky, M. (2001). Fun, play, and games: What makes games engaging. In Digital Game-Based

Learning. McGraw-Hill. Retrieved from

http://www.autzones.com/din6000/textes/semaine13/Prensky(2001).pdf

Prensky, M. (2005). "Engage me or enrage me": What today's learners demand. Educause

review, 40(5), 60-65. Retrieved from

https://web.elcosd.org/sandbox/users/dnoll/weblog/5e07f/attachments/536f0/Engage%20

Me%20or%20Enrage%20Me.pdf


28

Rideout, V. J., & Roberts, D. F. (2010). Generation M2: Media in the lives of 8- to 18-year-olds.

Menlo Park, California: Henry J. Kaiser Family Foundation. Retrieved from

https://kaiserfamilyfoundation.files.wordpress.com/2013/04/8010.pdf

Zvielli, A., Bernstein, A., & Koster, E. H. (2014). Dynamics of attentional bias to threat in

anxious adults: Bias towards and/or away? PLoS ONE, 9(8), e104025.

doi:10.1371/journal.pone.0104025


29

Figure 1. An illustration of the three steps in one trial of the visual probe task. In the first step,

participants must focus their attention on the cross in the center of the page. In the second step,

two stimuli appear very briefly on each side of the screen. In the final step, a directional probe

(up or down arrow) appears behind either the left or right stimulus.


30

Figure 2. Examples of a static face matrix (left) and a static flower matrix (right).


31

Figure 3. Examples of two different types of steps in a dynamic flower matrix.


32

Figure 4. Examples of two different types of steps in a dynamic face matrix.


33

Figure 5. Mean difference values in self-reported engagement between the four matrix

conditions. A significant difference was found in the level of engagement reported by

participants from the dynamic design compared to participants from the static design, the latter

finding the task as significantly less engaging. The error bars attached to each column in the

figure represent standard errors.


34

Figure 6. Mean changes in acceptance bias in each of the matrix conditions. A significant

difference was found between the stimuli, where faces yielded significant changes while flowers

did not. A significant design by stimuli interaction was also found, where static faces yielded a

significant increase in acceptance bias, while dynamic faces yielded a slight decrease in

acceptance bias. The error bars attached to each column in the figure represent standard errors.


35

Statement of Contribution

For this project I was responsible for numerous tasks. I conducted extensive research of

the literature with input and suggestions from my supervisor. I was responsible for the design of

the experiment, which involved creating and ordering the questionnaires (using FluidSurveys),

designing the matrix levels, and creating the list of anagrams (which involved creating a survey

and recruiting volunteers to test my anagrams for level of difficulty). I also managed the

implementation and running of the experiment, which involved assigning credit to participants

who participated for course credit, recruiting and distributing financial compensation to external

participants, and collecting regular backups of the data. With help from my supervisor, I

conducted multiple analyses of the data using SPSS software. Finally, I was responsible for the

writing of my thesis, with feedback from my supervisor.

There were a number of aspects of the study provided for me to which I did not

contribute. All aspects of the visual probe task were provided to me by my supervisor, which

included all images used throughout the task as well as the coding for the web application.

Various aspects of the matrix task were also provided to me by my supervisor, including all

images used throughout the task as well as the coding for the web application (involving the help

of an external programmer). The methods of storage of data for both tasks were also provided by

my supervisor.

Documents

Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL … · Title: The Effects of Engagement and Gamification on Attentional Bias Modification Author: Veronique Servant Date: June 2015