Quest for the Golden Rule: An effective social skills promotion and bullyingprevention program

Alice Rubin-Vaughan a,*, Debra Pepler b, Steven Brown c, Wendy Craig d

aGraduate Psychology Department, York University, 4700 Keele Street, Toronto, Ontario, Canada M3J 1P3b York University and The Hospital for Sick Children, 555 University Avenue, Toronto, Ontario, Canada M5G 1X8c Practi-Quest Corp, CanadadQueen’s University, 99 University Avenue, Kingston, Ontario, Canada K7L 3N6

a r t i c l e i n f o

Article history:Received 28 April 2010Received in revised form4 August 2010Accepted 9 August 2010

Keywords:Bullying preventionProgram evaluationE-learningEducational computer gamingElementary school age

a b s t r a c t

Everyday many students face bullying situations that they are ill equipped to manage. E-learning hasrecently emerged as a potentially effective tool in teaching children social skills, in addition to academicsubject matter. Quest for the Golden Rule is one of the !rst bullying prevention e-learning programsavailable, designed by the Practi-Quest Corporation, for children in grades 2 – 5. The purpose of thecurrent study was to explore data collected as part of standard program quality assurance practices toevaluate the impact of the gaming modules on how much children learned through interacting with themodules. Sample sizes ranged from 226 to 438 depending on the module; with approximately equalnumbers of boys and girls. Following their interactions with each module, children’s knowledge ofbullying and their identi!cation of strategies to prevent bullying improved signi!cantly. The majority ofchildren reported that they enjoyed the game and felt con!dent that they could help solve bullyingproblems. Quest for the Golden Rule is an engaging, effective, and ef!cient means of raising awareness,fostering positive attitudes, and promoting effective problem-solving for bullying prevention in schools.

! 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Everyday many students face bullying situations that they are often ill equipped to manage. Bullying has repercussions, not only forchildren’s social and emotional well-being (e.g., Arseneault et al., 2006), but also for academic achievement and school absenteeism (e.g.,Kshirsagar, Agarwal, & Bavdekar, 2007; Nansel et al., 2004; Ybarra & Mitchell, 2004). Adults must support children in managing thesesituations, as they often require skills exceeding children’s developmental capacities.

Teachers and caregivers alike are charged with the responsibility of educating children in both academic and social domains. AlthoughCanadian children are performing well on the international stage in terms of academics, ranking 3rd, 4th, and 7th out of 57 countries onscience, reading, and mathematics, respectively (Bussiere, Knighton, & Pennock, 2007), the same is not true in terms of children’s socialexperiences at school. Based on responses to the World Health Organization–Health Behaviors in School Aged Children survey, Canadiangirls ranked 26th and boys ranked 21st out of 40 countries on measures of bullying and victimization, respectively (Craig et al., 2009). Giventhe discrepancy between students’ academic and social experiences it is clear that we must focus on further supporting children’s socialdevelopment at school.

There are several ways in which adults can support children individually and in the context of their peer groups. According to Pepler(2006), adults can provide support to individual children through scaffolding or coaching, which children require to achieve skillsbeyond their developmental level. Scaffolding is a critical element of bullying prevention initiatives for childrenwho bully, childrenwho arevictimized, and children who witness bullying (Pepler, 2006). Scaffolding can be provided in innovative and engaging ways througheducational gaming. The current study provides a preliminary evaluation of a pioneering suite of web-based bullying prevention games,called Quest for the Golden Rule: Bullying Prevention Software. These games are designed to engage children in bullying prevention exercises

* Corresponding author. Tel.: !1 416 678 6661.E-mail addresses: alicev@yorku.ca (A. Rubin-Vaughan), pepler@yorku.ca (D. Pepler), steven@practiquest.com (S. Brown), wendy.craig@queensu.ca (W. Craig).

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

0360-1315/$ – see front matter ! 2010 Elsevier Ltd. All rights reserved.doi:10.1016/j.compedu.2010.08.009

Computers & Education 56 (2011) 166–175

through computer-based gaming modules, and are based on the principle of scaffolding through the provision of tailored and progressivesupport for each student.

Although many students experience bullying and can bene!t from education and support in addressing these challenging socialdynamics, there is a small group of children (10%) who reports consistently high levels of bullying over time andwho require more intensivesupport including practice and coaching to learn essential social skills (Pepler, Jiang, Craig, & Connolly, 2008). Because public schools tend tobe under-resourced, it is often beyond the capacity of classroom teachers to provide the intensive and unique learning opportunities thatsome students require. Quest for the Golden Rule may provide an alternative means to provide intensive support to students in schoolthrough individualized experiences, an opportunity not typically provided through traditional approaches to bullying prevention.

Early prevention and intervention for bullying problems are crucial to support children in developing healthy academic, social, andemotional coping skills (e.g., Mitchell, Ybarra, & Finkelhor, 2007). In a comprehensive review of thirty stringent studies of bullyingprevention programs, Tto! and Farrington, (2009) demonstrated the effectiveness in reducing bullying and victimization by an average of20–23%. There is modest evidence to support the effectiveness of bullying prevention programs for elementary school students (e.g., Craig,Pepler, & Shelley, 2004; Smokowski & Kopasz, 2005); however, in their meta-analysis, Tto! and Farrington, (2009) found that programs forchildren older than elevenweremore effective. Because of the importance of prevention and early intervention, it is essential to increase theeffectiveness of bullying programs for young children to support them before early adolescence, which is a developmental period ofincreased risk for bullying (Pellegrini & Bartini, 2001; Pepler, Jiang, Craig, & Connolly, 2008).

Quest for the Golden Rule was designed to incorporate many of the effective characteristics of established bullying prevention programs(as recommended by Tto! & Farrington, 2009). In addition to the primary elements, Quest for the Golden Rule provides an enhanced learningexperience through the use of interactive gaming technology, which is individualized, accessible, and attractive for younger children. Fewcomputer-based bullying prevention programs have been developed for students in elementary school. An objective of the current studywas to provide preliminary data regarding the effectiveness of interactive bullying prevention software designed to increase knowledge andchange attitudes.

2. Why use educational gaming?

The issue of “play”may be particularly salient inwork with children and pre-adolescents. At this developmental stage, the importance ofplay in facilitating learning has long been recognized. Vygotsky (1978) posited that play creates the opportunity for children to experimentwith acting more maturely than their developmental stages, thus creating a “zone for proximal development”. Children’s social develop-ment can be fostered through play and experimentation with new social skills. Despite recognition of the importance of play in teachingchildren, there is a dearth of software programs that are acceptable to both children (as being fun and engaging) and adults (as providing“serious” education) (De Castell & Jenson, 2003).

3. Learning and educational gaming

Active learning, metacognition, and transfer of knowledge have been identi!ed as crucial components of learning (Huffaker & Calvert,2003; Shih, Feng, & Tsai, 2008). Huffaker and Calvert (2003) reviewed the literature and concluded that e-learning may support the criticalcomponents of learning through: 1. children’s motivation to engage in the learning process; 2. self-direction such that children learn to planand monitor their learning; and 3. collaborative activities and entertainment features that encourage the application of learning to situa-tions outside the classroom. Educational gaming programs capitalize on experiential learning by actively engaging students with thecomputer program.

Recently, researchers have begun to address the gap in the literature regarding whether elementary and secondary school students farebetter in educational gaming environments versus traditional or combined programs. Much of this research has focused on secondaryschool students with mixed results regarding whether the students learned more from a traditional versus combined curriculum (Annetta,Minogue, Holmes, & Cheng, 2009; Chandra & Lloyd, 2008). Only engagement and motivation were consistently found to be higher with theinclusion of educational gaming experiences. Research on elementary school students’ acquisition of sun safety knowledge revealed thatlearning was heightened, for younger students, by a combination of traditional teacher presentations and computer programs compared totraditional presentations. In comparison, there was no difference between combined and traditional programs for older students (Bulleret al., 2008). Given that the development of language skills is an essential task for early elementary school students, the presentation oflearning materials through both traditional and audiovisual and graphic means may augment the learning process (Buller et al., 2008).

4. Bullying prevention through educational gaming

The detrimental consequences for those involved in bullying, as the child who bullies, the child who is victimized, or children whowitness bullying incidents provide the impetus for the development of bullying prevention programming through educational gaming.Although overall, traditional bullying prevention programs have been found to be effective, there is considerable variability in the results,particularly for younger students (Baldry & Farrington, 2007; Tto! & Farrington, 2009). Early prevention is critical because of the potentialduring early childhood to build skills at a developmental stage during which students tend to be more accepting of adult-directedcurriculum, more willing to talk to adults about bullying, and have more trust that adults are able to help with bullying problems (Craig,Pepler, & Blais, 2007; Rigby, 2002). Educational gaming may be one new direction to pursue in bolstering the effectiveness of bullyingprevention programs for young children.

There are several areas of social skills promotion and bullying prevention that are ripe for education programs geared for youngerchildren. Elementary school students bene!t from learning a variety of skills that are readily reinforced by teachers at school, including:1. empathy, 2. emotional and behavioral regulation (e.g., recognize emotions, stop to regulate emotions through strategies like counting ordeep breathing), 3. coping with feelings of sadness or anxiety, 4. social skills (e.g., joining group of peers, turn taking, getting positiveattention), 5. positive leadership skills – to engage power dynamics positively, 6. alternative problem-solving, and 7. withstanding peer

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175 167

pressure (awareness, stop and think, explore responses to peers) (Vaughan & Pepler, 2007). Skill-building computer games have beendeveloped to achieve both bullying and more general violence prevention goals for elementary and secondary school students.

Speci!c educational lessons are required in considering bullying, which represent a unique social problem involving challengingdynamics and a pattern of behavior that signify a power imbalance in the relationship with multiple players in different roles (Olweus,1993). The current review yielded only two published articles with data regarding bullying prevention programs including multimediacomponents. McLaughlin, Laux, and Pescara-Kovach (2006) designed a unique experimental study to explore the impact of addingmultimedia interventions (CD-ROM with relaxation exercises and a quiz show) to the traditional counselor/teacher bullying preventionprogram for grade three children. The !ndings suggested that the interventions were effective in signi!cantly reducing bullying andvictimization; however, were inconclusive regarding whether the degree of change was related to the addition of the multimediacomponents (McLaughlin, Laux, & Pescara-Kovach, 2006). Although multimedia programming was used, the format failed to meet therecommendations of Huffaker and Calvert (2003) in terms of how e-learning may facilitate the educational process. The multimediacomponent was not animated nor was it interactive or self-directed, therefore, it may not have led to increased motivation for students toengage in the learning process.

FearNot! (Fun with Empathetic Agents to Reach out Novel Outcomes in Teaching) is a virtual role playing bullying prevention program(Hall, Woods, Hall, & Wolke, 2007). The FearNot! Program, developed for students aged 8–12, focuses on helping children to developa deeper understanding of bullying issues and coping strategies, through their ability to empathize with the virtual characters and act as aninvisible friend to a child who was victimized (Paiva et al., 2005). Students were found to bene!t from this program, such that FearNot!signi!cantly increased the likelihood that students reporting victimization at baseline would escape victimization by the !rst follow-upassessment compared to the control group, particularly those who interacted more with the characters (Sapouna et al., in press).Furthermore, greater levels of empathy and story comprehension occurred when animated characters were of the same gender as theparticipant, particularly for boys (Paiva et al., 2005; Woods, Hall, Dautenhahn, &Wolke, 2007). Evaluation of the FearNot! program providesevidence for the utility of educational gaming in addressing bullying, though it does not extend our understanding of the contribution ofeducational gaming over the traditional bullying prevention approach.

Based on the immense success of the gaming industry and the clear appeal that gaming holds for children, the use of gaming mayfacilitate engagement with educational content. Children are tech-savvy, with 94% having access to computers with the Internet at home.Furthermore, online games are a favorite pastime for younger children – particularly among grade four students, 89% of whom reportplaying games online (ERIN Research & Wings, 2005). Despite the prominent societal role of technology in education, recreation, andbusiness, and the subsequent opportunity to explore the utility of e-learning for intervening in serious social problems such as bullying,innovation into bullying prevention gaming has been slow to emerge. Although there is a relative dearth of research, emerging evidencefrom social skills promotion and bullying prevention programs using interactive computer software provides support for the potentialeffectiveness of an educational gaming approach in teaching students social skills and coping strategies tomanage bullying situations. In thepresent study, we expected that Quest for the Golden Rule would be effective in teaching children bullying prevention messages.

5. Program description: QUEST for the Golden Rule

The current study provides a preliminary evaluation of a new set of innovative prevention and intervention tools designed by the Practi-Quest Corporation (www.practiquest.com), in consultation with leading researchers in the area of bullying from PREVNet (PromotingRelationships and Eliminating Violence Network). Quest for the Golden Rule uses principles of effective bullying prevention, translated intoa modality of particular interest for children and youth – interactive, animated web-based games. The games are designed in such a way asto encourage experiential learning within a virtual and interactive format, facilitated through a safe and private virtual environment free ofpeer pressure. Children interact individually with animated characters in virtual role-plays, which provide them with the opportunity tolearn and practice social skills and try out different strategies to cope with bullying. Students are unable to move on from a social problem-solving situation until they are able to provide a prosocial solution, ensuring that children are supported in learning skills that they may belacking.

Quest for the Golden Rule provides a solution to the typical problems of providing bullying prevention education within traditionalclassroom settings in which teachers are often overwhelmed, resources are not available to identify students who may need additionalsupport, or teachers lack the time to provide the intensive coaching and practice required by some students. Companion guides for teachersaccompany the software and include targeted curriculum and follow-up activities. Students are encouraged to participate in additionalactivities at home. Three modules have been designed for students in grades two to !ve each addressing a different topic related to bullyingprevention.

5.1. Bark Academy

This module provides an introduction to social justice, safety and fairness in school, introduces bullying and depicts three forms ofbullying behavior (social, verbal, and physical). Students are invited to attend Bark Academy, a dog school where bullying manifestsitself in a variety of unfair behaviors, where they help solve cases of bullying by using the concept of The Golden Rule. Captain Fairness,a super-dog who helps address bullying and creates fair environments for kids, assists students as they work through the module. In the!rst case, Farid, a much bigger and stronger student, is bullying Kate and Kerri. Upon closer investigation, students learn that Farid isupset at being disrespected and excluded from play by Kerri and Kate. The students must help Kerri and Kate realize they wereparticipating in social bullying and help Farid discover more appropriate ways to respond when he feels left out. In the second case,Jermaine is physically bullying Hershey while a number of bystanders watch and laugh. Jermaine is also verbally bullying Hershey bymaking comments about his weight. The students must help Jermaine and the bystanders understand how they contributed to thebullying situation and explain the importance of following The Golden Rule. The module ends as the bystanders are confronted withtheir roles in the bullying situation. Students are left to question what impact the bystanders have in Hershey’s case and are encouragedto return for part two of Bark Academy to !nd out what the bystanders should have done differently. In the !nal case, Bobby

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175168

participates in social and cyber-bullying by threatening to send an email to the entire school that teases Yong about his ugly coat. Thestudents must help Bobby realize that he should not send the email by using the concept of The Golden Rule. Moreover, the studentshelp Yong learn that reporting bullying is not tattling.

5.2. Mission to Mars

This module teaches students about the social skills involved in making friends. The role-play begins as the students, as space cadets,travel to Mars to be interviewed by a group of astronauts called the “eXplorers”. If the students impress the eXplorer astronauts, Wynona,Adam and Jessie, with qualities such as kindness, and respect, they are invited to become the newest eXplorer and walk on Mars. Once thestudents arrive on Mars, they are introduced to Lionel, an assistant whom the students must mentor. Lionel and the students build a spacehelmet together. Before being interviewed by the eXplorers the students watch Reggie, another candidate, complete his interview. Reggie isunsuccessful in becoming an eXplorer because he tries to impressWynona, Adam and Jessie with his talents rather than listening and askingquestions about the eXplorers. The eXplorers decide his self-centeredness could be dangerous on Mars. The eXplorers then interview thestudents, andwhen the correct options are chosen, they are invited to become the newest eXplorer. During the interview Lionel interrupts toshow how well he can blow bubbles. He accidentally drops a pile of bubble gum on one of the eXplorer’s desks. The students must treatLionel with kindness during this incident in order towin the respect of the eXplorer team. Immediately after becoming an eXplorer, studentsare given the task of rescuing Reggie, who had decided to exploreMars on his own and fell into a giant hole. Once the students decide to usethe help of the eXplorer team to save Reggie, the team thenworks together to explore a strange golden light seen on the planet. The studentsand the team discover this light to be The Golden Rule.

5.3. Ghoul School

This module addresses the important issue of how to respond when faced with bullying situations and empowers students to safelyrefuse and report bullying. In this module there is ample opportunity for students to explore, discuss, and practice appropriate andinappropriate responses to bullying. Students are taught three important strategies for responding to bullying: “Lead with Respect”, “Let anAdult Help” and “Leavewith Respect”. Students are also taught to monitor their decisions to determine whether their actions make bullyingproblems bigger or smaller.

In the second part of Ghoul School, students are introduced toMousey. Mousey is a two-headedmonster that enjoys telling silly jokes. Heis often bullied byMolnar. Mousey has trouble staying calm and the students must !rst help Mousey become calm before he can respond tothe bullying. As Mousey calms down, the bullying nearly stops and Mousey is free to respond to his situation by either leading throughassertiveness, letting an adult help, or leaving. The students help the monsters decide to incorporate The Golden Rule into their own lives,and in the end, they all seek out new professions where they are accepted for who they are.

6. Current study

Three key research questions were explored in this study, with corresponding hypotheses.

1. Were there any signi!cant gender or grade differences in the children’s initial levels of knowledge demonstrated in the Bark Academy,Mission to Mars, and Ghoul School modules?" We expected that boys and girls would have similar levels of knowledge but that average knowledge of bullying would increase witheach grade.

2. Did children’s knowledge of how to cope with bullying improve signi!cantly following interaction with the Bark Academy, Mission toMars, and Ghoul School and was this related to gender or grade?" We expected that students would demonstrate signi!cant knowledge gains following interactionwith the games, and that the degreeof learning would be related to gender and grade.

3. Did children enjoy the Bark Academy, Mission to Mars, and Ghoul School modules?" We expected that the majority of children would favorably rate their subjective enjoyment of the software.

7. Method

Quest for the Golden Rule was investigated independently of Practi-Quest by PREVNet researchers. The current study representsa preliminary investigation with plans for subsequent formal evaluation in the upcoming years.

7.1. Participants

The participant pool constituted children who received this interactive computer programming in fall 2008 and winter 2009. AlthoughQuest for the Golden Rule was designed for children in grades two to !ve, grade six students were also included in this study. Several sixthgrade teachers in schools that were adopting the modules were interested in having their classes participate, thus it was left to theirdiscretion to determine if the modules were developmentally appropriate for their students. Schools at arms-length from the softwaredevelopers were targeted for the sale of the software. Schools were recruited to participate in preliminary testing of these modules throughthe endorsement of school board representatives or willingness of the principal to adopt this new form of bullying prevention program-ming. The modules were offered freely in a small portion of schools, with the remaining schools having purchased, for a small fee, the rightsto use the software in their classrooms as part of their educational curriculum. Access to the program was provided to each studentindividually – with teachers having online access, enabling them to log in to see how their students were doing.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175 169

Although data about ethnicity were not collected systematically, the schools from which data were collected represented a largeCanadian urban centre, populated by cultural, religious and ethnically diverse groups. Because students often only completed one module,we had different participants and sample sizes for each game. The number of students who completed each module was 307 for BarkAcademy, 226 forMission to Mars, and 438 for Ghoul School. The breakdown of participants with data at pre- and post-module by gender andgrade follows in Table 1.

7.2. Measures

Practi-quest staff and PREVNet researchers worked collaboratively to develop questions to address the speci!c knowledge and attitudesrelated to the skills taught within each module. Children were asked to respond to the questions for each module prior to, and immediatelyfollowing, their use of the module. For the majority of questions, the response options were based on a !ve-point Likert scale (ranging from“1 – strongly disagree” to “5 – strongly agree”). Because the Bark Academy andGhoul Schoolmodules were found to be too long for children tocomplete in one class session, they were broken into two sections that could be completed at different times. For these two modules, thequestions were similarly divided into two sections depending on the information contained in each section. For the purpose of the currentstudy, only students who completed both sections of a module were included in the analyses to understand what students learned from thefull module. To assess engagement with each of the modules, students were asked at the end of each module to rate their agreement withthe statement “This computer game was fun” on a !ve-point Likert scale.

7.2.1. The Bark AcademyCommensurate with topic areas covered by the modules, the Bark Academy–Fairness Scale (BA: Fairness Scale) questions addressed

attitudes about fairness and social justice and about bullying and coping strategies. The fairness and social justice items include ratings onstatements such as: I should always be fair even to people who I do not like; I am willing to work harder to make my school fair foreverybody; and I believe there is never an excuse to bully another person. The attitudes about bullying and coping strategies items includeratings on statements such as: Watching other students bully someone is always taking part; If I see someone getting bullied I would keepaskingmore adults for help until the problem is solved; and If I am being bullied I will always ask an adult to keepme safe. Seven items werecombined to make a scale, labeled the BA: Fairness Scale; however, the reliability was best when one statement was removed (It is alwaystattle-telling to tell an adult if someone is being bullied). Although this question was reverse coded so that higher scores re"ected a morepositive response, the question still may not have !t with the other items because students were more likely to respond negatively to thisquestion than the others.With the remaining six items (maximum scale total of 30) the internal consistencywas still low, but adequate, withthe Cronbach’s alphas being .55 and .68 at pre- and post-module, respectively.

7.2.2. The Mission to MarsFriendliness Scale (MM: Friendliness Scale) questions addressed attitudes and knowledge about respectful treatment of friends and

peers, and strategies to stay safe from bullying. Eleven items rated on a !ve-point Likert scale were combined to make a total score for theMM: Friendliness Scale and included the following items about friendship: If a student is mean to me I should be mean back to them; Ifa student won’t talk to me it means they don’t like me; I would like to learn how to be more respectful and friendly; I should learn tocompliment others more often; I always deserve to be treated with respect and friendliness; I have thought about joining a new group offriends soon; I have thought about inviting a new person to join my group of friends; I will always treat someone with friendliness even if Idon’t like him/her; The best way to stay safe from bullying is staying near good friends; and I feel comfortable making new friends. Thereliability was best when one statement was removed (My friends could be more friendly and respectful to me). This item may not have !twith the other items because students were more likely to respond negatively. The ten remaining items from this module were combinedinto a scale (maximum scale total of 50), found to have good internal consistency, with Cronbach’s alphas of .70 and .72 for pre- and post-module scales, respectively.

7.2.3. The Ghoul SchoolGhoul School was assessed using questions in several domains including children’s knowledge of how to identify bullying and strategies

to respond when faced with bullying situations, including safely refusing and reporting bullying. Fifteen items, each rated on a !ve-pointLikert scale were combined to make the Ghoul School: Attitudes Scale (GS: Attitudes Scale), with a maximum scale total of 75. Items on thisscale capture the attitudes of students toward their right to be treated respectfully, and strategies they might use in the face of bullyingsituations and included statements such as: When being bullied, the most important thing for me to do is to make the problem smaller; My

Table 1Number and Percentage of Participants for each Module, by Gender and Grade.

Variable Number and Percentage of Participants

Bark Academy Mission to Mars Ghoul School

GenderGirls 142 (46.4%) 113 (50.0%) 220 (50.3%)Boys 165 (53.6%) 113 (50.0%) 218 (49.7%)

Grade2 30 (9.6%) 1 (0.5%) 25 (5.7%)3 57 (18.5%) 49 (21.5%) 96 (21.8%)4 108 (35.3%) 76 (33.6%) 144 (32.8%)5 108 (35.3%) 99 (43.9%) 148 (33.9%)6 4 (1.4%) 1 (0.5%) 25 (5.7%)

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175170

friends should always treat me with respect and fairness; My friends always treat me with respect and fairness; I am a unique and specialperson; I would let an adult help me if a bullying problem is too big for me to handle on my own or if I feel unsafe; I think that respectful,calm people are less likely to be bullied; Showing that I am upset will make the bullying worse; I can avoid being bullied if I stay in a safeplace or stay near respectful friends; I canmake bullying problems smaller; I would like to practice staying calmwhen being bullied; I wouldlike to practice ways to respectfully say no to bullying; I have the right to always be safe; To leave with respect I should make eye contactthen calmly and proudly say I don’t accept being bullied and leave; Acting weak will always make the bullying worse; and Being aggressivewill always make the bullying worse. This scale was found to have good internal consistency, with Cronbach’s alphas of .88 and .91 for pre-and post-module scales, respectively.

The Ghoul School: Strategies Scale (GS: Strategies Scale), which assessed strategies to prevent and cope with physical victimization, wasbased on the number of strategies endorsed through discrete responses (“yes” or “no”) of students to the item “If someone has unfairlyshoved me many times this week, and just shoved me again, I will.”. Students were required to select all the correct answers given the tenpossible options including: make myself calm before I do anything, push them back to show them that I am not afraid, ignore the pushingand quietly wait for the problem to go away, if I feel upset not show that I am upset, make sure I stay safe and even run away if that’s what’sneeded, leave with respect, let and adult help if I can’t !x the problem alone, stand tall, shoulders back and use a calm clear voice while Imake eye contact, calmly and proudly say I don’t accept being bullied, or leave before the problem becomes bigger. The Ghoul SchoolPrevention scale had a maximum scale total of ten items endorsed, and was found to have good internal consistency with Cronbach’s alphasof .89 and .91 for pre- and post-module scales, respectively.

Students’ recognition of four key bullying prevention strategies were assessed through their responses to Item 9, “To make a bullyingproblem smaller, I can.”. The potential responses students could select, given the instruction to select all that apply, included: leave withrespect by maintaining my own respect and respect for others, be with others who treat me with respect, let a trusted adult help, lead withrespect, and say that bullying is not cool. Finally, students’ identi!cation of elements of the bullying de!nition was evaluated through theirresponses to Item 10, “Veronica and Ibolya always point and laugh when Ivan gets on the school bus. How do we know if Ivan is beingbullied?” Students were instructed to select all of the correct answers out of three options: it’s not fair to Ivan, it keeps happening, and ithurts Ivan. Because these two items were comprised of a small number of discrete responses, with maximum scale totals of four and three,respectively, they were explored descriptively.

7.3. Procedure

Practiquest software was adopted by whole schools, or classrooms within a school, to be used as part of the educational curriculum.Students within each classroom were assigned usernames and passwords to access the modules. Although the program was delivered atschool, some students took home their passwords and later accessed the games and continued to play from their home computers. As part ofparticipation in this program Practiquest collected data for program research and development as well as for quality control. Data fromevery student who used the program were accessible to the program developers and these data were used in the current study.

8. Results

8.1. Data Screening

From the descriptive statistics, it was clear that before and after the games most children responded in a manner re"ecting ampleunderstanding of the de!nition of bullying, and with knowledge of some skills required to cope with bullying situations (see Table 2 fordescriptive statistics). On average, before using the modules students “agreed” with prosocial attitudes and knowledge on items containedwithin the BA: Fairness Scale, MM: Friendliness Scale, and GS: Attitudes and Strategies Scales. Children had some understanding of thede!nition of bullying, correctly identifying 1.8 of 3 elements of the de!nition. They were also able to identify, on average, 5.5 out of 10strategies to cope with physical bullying and 2.4 out of 4 strategies to address bullying problems.

Diagnostics were completed to explore each variable prior to statistical analyses. On the BA: Fairness Scale and GS: Attitudes andStrategies Scales, square root and natural log transformations were used due to concerns regarding violations of normality and equalvariances. Analyses were run with and without these transformations. Because no substantive differences emerged for these scales, andgiven that ANOVA is robust despite violations of the assumption of equal variances (particularly given roughly equal numbers in each group),we decided to use to the original variables. To control for type 1 error, given the number of analyses conducted, we used a Bonferronicorrection with a criterion of p< .01.

8.2. Were there gender or grade differences in children’s initial levels of knowledge and attitudes?

8.2.1. Bark AcademyNo signi!cant gender differences were found in knowledge on the BA: Fairness Scale prior to using the module, t(346)#$.61, p# .545.

There were signi!cant grade differences on pre-module scores F(4)# 4.39, p# .002. Post hoc tests revealed that our hypothesis that

Table 2Descriptive Statistics: Means, Standard Deviations, and 95% Con!dence Intervals.

Variables Pre Mean (SD) Con!dence Interval Post Mean (SD) Con!dence Interval

BA: Fairness Scale (n# 307) 26.32 (3.04) (26.00, 26.64) 26.88 (3.30) (26.51, 27.25)MM: Friendliness Scale (n# 226) 40.37 (5.61) (49.62, 41.13) 42.20 (5.56) (41.45, 42.95)GS: Attitudes Scale (n# 438) 63.70 (8.82) (62.88, 64.53) 66.24 (9.03) (65.40, 67.09)GS: Strategies Scale (n# 438) 5.66 (3.33) (5.34, 5.97) 6.04 (3.47) (5.72, 6.34)

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175 171

knowledge would be greater among student in higher grades was only partially supported. Grade six students scored signi!cantly higherthan students in grades three and !ve but there were no other signi!cant differences among grades (see Table 3 for descriptive statistics bygrade).

8.2.2. Mission to MarsNo signi!cant gender or grade differences were found in knowledge on the MM: Friendliness Scale prior to using the module, t

(212)#$.93, p# .356 and F(4)# 1.54, p# .192, respectively.

8.2.3. Ghoul SchoolAcross both GS: Attitudes and GS: Strategies Scales there were statistically signi!cant gender differences such that the girls’ pre-module

knowledgewas greater than that of the boys, t (436)#$2.23, p# .026 and t (436)# 3.17, p# .002 respectively. Each scale showed signi!cantgrade differences, F (3)# 3.76, p# .011 and F (3)# 7.92, p< .000, respectively. Post hoc tests revealed that although grade !ve students hadthe lowest average GS: Attitudes Scale score, they were only signi!cantly lower than students in grade four. Conversely, grade !ve studentsscored highest on the GS: Strategies Scale and the mean score was signi!cantly higher than for students in grades two and three. Grade fourstudents, likewise, had the second highest mean score and were trending in the direction of knowing signi!cantly more strategies toprevent bullying than students in grade two (see Table 3 for descriptive statistics by grade). Our hypothesis that knowledge increasedcorresponding to advancing grade was not supported for the GS: Attitudes Scale but was somewhat supported for the GS: Strategies Scale,suggesting that knowledge of bullying prevention strategies increased with development.

8.3. Were there improvements in children’s knowledge following interaction with each of the modules, and was this related to gender or grade?

8.3.1. Bark AcademyOn the BA: Fairness Scale, a signi!cant time by gender interaction was found, F (1, 307)# 4.29, p# .039. Effect size was estimated by

partial h2# .014 with observed power# .542. Girls’ knowledge improved signi!cantly more than boys’ knowledge (see Fig. 1). Grade wasincluded as a covariate in this analysis but was not signi!cant, F (1, 307)# .20, p# .657.

8.3.2. Mission to MarsThe MM: Friendliness Scale revealed a signi!cant main effect of time, F (1, 214)# 32.37, p# .000. Effect size was estimated by partial

h2# .132 with observed power# 1.000. Because there was no time by gender interaction, F (1, 214)# .031, p# .861, the bullying knowledgeof boys and girls improved similarly through this module (see Fig. 2). Grade was not included as a covariate because no signi!cantdifferences were found between grades in the pre-module scores.

8.3.3. Ghoul SchoolTrends were identi!ed for both the GS: Attitudes Scale and GS: Strategies Scales, in terms of a main effect of time, F (1, 430)# 3.01,

p# .084 and F (1, 430)# 2.76, p# .097, respectively. Effect size was estimated by partial h2# .007 with observed power# .409, and partialh2# .006 observed power# .383, respectively. No time by gender interactions, F (1, 430)# .09, p# .763 and F (1, 430)# 1.79, p# .193,

Table 3Descriptive Statistics for Pre Data Across Grade Level: Mean (Standard Deviation).

BA: Fairness Scale MM: Friendliness Scale GS: Attitudes Scale GS: Strategies Scale

Grade2 10.92 (4.25) 30.00 (n/a)a 27.80 (12.34) 4.05 (3.46)3 9.01 (2.51) 18.57 (7.58) 26.38 (9.99) 5.03 (3.63)4 10.21 (3.24) 19.90 (5.35) 27.52 (8.46) 5.65 (3.21)5 9.17 (2.77) 19.87 (4.52) 24.38 (6.78) 6.55 (2.97)6 11.33 (3.55) 15.00 (n/a) a 28.00 (10.15) 4.60 (3.91)

a Standard deviation was not applicable because there was only one participant in the group.

Fig. 1. BA: Fairness Scale changes in knowledge from pre to post-module.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175172

respectively, nor covariate effects of grade emerged. These trends are suggestive that students’ attitudes towards empowerment andknowledge of bullying prevention strategies improved after interacting with the module.

The additional items were explored using z-test of proportions. Knowledge of four strategies students could use to make bullyingproblems smaller was measured in the Ghoul Schoolmodule. Childrenwere able to identify signi!cantly more bullying prevention strategiesafter the module, compared to before (z#$6.45). After using the module, 315 (71.9%) of the childrenwere able to identify at least 3 out of 4strategies compared to 222 (50.7%) before. Children’s knowledge of the elements of the bullying de!nition was also measured in the GhoulSchool module. Children were able to identify signi!cantly more elements of the bullying de!nition after the module, compared to before(z#$6.23). After using the module, 299 (68.3%) of the children were able to identify at least 2 out of 3 elements of the bullying de!nitioncompared to 208 (47.5%) before.

8.4. Did children enjoy using the interactive computer modules?

The majority of children reported that they enjoyed the modules. When asked if the computer game was fun, 93.4% who played BarkAcademy, 93.0% who playedMission to Mars, and 83.6% who played Ghoul School “strongly agreed” or “agreed”. Of those childrenwho playedGhoul School, 84.5% “strongly agreed” or “agreed” that they could make bullying problems smaller, re"ecting con!dence in the skills thatthey learned or were reinforced through the computer game.

In summary, there is evidence that children’s knowledge of bullying and their identi!cation of strategies to prevent bullying improvedfollowing their interactionwith each module. By the typical conventions of observed power analysis (greater than .8 representing adequatepower), we had relatively low power for these statistical tests to detect signi!cant differences. With larger samples or over a longer follow-up period, the differences detected may have been stronger, and we may have had more power to detect gender or grade differences.Furthermore, the estimates of effect size (partial h2) were small for all the scales examined, suggesting that although we found statisticallysigni!cant improvements, they were relatively modest, thus we must be cautious in our interpretation.

9. Discussion

Bullying is a relationship problem that is challenging for both children and adults to manage. Early education and bullying preventionprograms are crucial to provide children with opportunities to learn and practice skills so they are better equipped to manage bullyingproblems. Educational gaming, such as Quest for the Golden Rule, provides a fun and interactive way for students to learn individually and forclasses to have a collective experience that promotes a safe and positive climate. Because educational gaming – as a method of teachingsocial skills or bullying prevention – is fairly new, there is limited research available to explore whether these programs are effective and forwhom.

Evidence from the preliminary investigation of the suite of games in Quest for the Golden Rule shows signi!cant improvements inknowledge and attitudes across each of the three modules. Children gained knowledge about issues of fairness and safety in their schools(Bark Academy), social skills (Mission to Mars), and strategies to refuse and cope with bullying (Ghoul School). The vast majority of studentsreported that they enjoyed using the games; further evidence of their enjoyment and engagement was provided through log-on recordsindicating that many students took their passwords home and played the games from their home computers.

Based on research suggesting that boys and girls interact differently with online gaming environments (e.g., Paiva et al., 2005), wehypothesized that gender would be a factor in understanding students’ learning through the modules. As children completed the games theonly evident gender difference in girls’ and boys’ responses was found on the Bark Academy, with girls’ knowledge of fairness and safetyimproving signi!cantly more than that of boys. On the MM: Friendliness Scale, knowledge of social skills increased for both boys and girlsand, despite initial gender differences on the GS: Attitudes and Strategies Scales a trendwas found for both boys and girls toward an increasein knowledge of empowerment and physical victimization prevention strategies. Although gender differences were evident on the BA:Fairness Scale in gains of knowledge and changes in attitude related to respect and fairness, overall there were many commonalitiesindicating that Quest for the Golden Rule modules are effective in teaching both boys and girls about issues related to bullying and bullyingprevention strategies.

Before the children had experienced these educational games, there were signi!cant grade differences in children’s scores on the BA:Fairness Scale, GS: Attitudes and Strategies Scales. It is notable, however, that grade was not statistically signi!cant in understanding how

Fig. 2. MM: Friendliness Scale changes in knowledge from pre to post-module.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175 173

much girls and boys learn about bullying prevention through these two games. Because of the relatively low power in these analyses, it mayalso be that the power to detect grade-based variations was limited. Another interpretation of these !ndings is that because Quest for theGolden Rule is an interactive computer game that children complete in a self-directedmanner, it is equally effective across elementary schoolgrades in leveling out differences in understanding of bullying.

These !ndings highlight the universal applicability of the program for students in grades two to !ve, such that childrenwork at their ownpace, attend to elements that are of greatest interest to them, experience success in resolving dif!cult social situations, and spend additionaltime acquiring skills and understanding in areas in which their skills are under-developed. As a result, children’s experiences with Quest forthe Golden Rulemodules are tailored in amanner that supports optimal learning. This type of individualized support provides the scaffoldingrecommended by Pepler (2006) that is an integral element of bullying prevention. Given that teachers are often overburdened and do nothave time to provide the individualized coaching that some children require, the design of this computer program enables students in needof additional support to receive more opportunities to practice their skills and develop social competence. This fun learning experience isconsistent with Vygotsky’s (1978) observation that through play children are able to develop skills and understanding beyond their existinglevels of competence.

9.1. Limitations

Although the current study was not conducted using a full experimental design, results are generally consistent with those of theFearNot! program (Sapouna et al., in press) in the sense that improvements in knowledge and attitudes were found for students who usedthe games. Cautionmust be taken in interpreting the results due to the low power to detect change andmodest effect sizes. These dif!cultiesmay be attributed, in part, to the post-testing immediately after the students had completed the modules. With a subsequent follow-upassessment after a period of time in which the lessons from the games can be reinforced by teachers, peers, and parents, we might !nd aneven stronger program effect.

An important aspect of program evaluation is to determine for whom programs work and why. In the present study, we did not haveinformation about students’ involvement in bullying, victimization, or witnessing bullying, which may have been a factor in successfullearning through the program. Given the overall high level of bullying prevention knowledge demonstrated by these students, it would behelpful to understand the nature of students’ involvement in bullying and their associated levels of knowledge. Conducting research usingan experimental designwould strengthen the analyses through provision of a control groupwith which tomake comparisons. An importantfuture direction will be to explore, through an experimental design, how bullying prevention games alone, traditional bullying prevention,or a combination of gaming and traditional bullying programs compare to a control condition in terms of children’s acquisition of bullyingprevention knowledge and skills, as well as the longer-term impact on the bullying experiences of students.

10. Conclusion

Educational gaming appears to be a promising area through which to bolster the effectiveness of early bullying prevention programs forelementary school students. Bullying prevention computer games complement a whole-school approach by providing cumulative andindividualized learning of prosocial knowledge, attitudes, and skills within the classroom context that may help shift the social normswithin the class and decrease bullying behavior. This preliminary investigation of Quest for the Golden Rule provided a unique opportunity toexplore the effectiveness of gaming in bullying prevention. One of the greatest strengths of this program is the individualized nature of thegames, and subsequent support given to students in the form of scaffolding to help them develop skills above their existing levels ofcompetence. This program shows great promise in teaching elementary school students about bullying and strategies to cope with thesedif!cult social situations. Supported through whole-school policies and adult guidance, Quest for the Golden Rule contributes importantbullying prevention education that will help set children up for healthy social development and healthy relationships throughout their lives.

Acknowledgments

First and foremost, wewould like to thank the students and teachers who participated in this study. We are grateful to the staff membersat Practi-Quest Corp who have engaged in this important and innovative effort to make appealing and educational bullying preventiongaming software. This project was also greatly supported by PREVNet (Promoting Relationships and Eliminating Violence Network), throughthe researchers who collaborated in developing Quest for the Golden Rule, and in providing graduate student funding for research on thissuite of bullying prevention games.

References

Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics.Computers & Education, 53, 74–85.

Arseneault, L., Walsh, E., Trzesniewski, K., Newcombe, R., Caspi, A., & Mof!tt, T. E. (2006). Bullying victimization uniquely contributes to adjustment problems in youngchildren: a nationally representative cohort study. Pediatrics, 118(1), 130–138.

Baldry, A. C., & Farrington, D. P. (2007). Effectiveness of programs to prevent school bullying. Victims & Offenders. Special Issue on Early Intervention, 2, 183–204.Buller, M. K., Kane, I. L., Martin, R. C., Giese, A. J., Cutter, G. R., Saba, L. M., et al. (2008). Randomized trial evaluating computer-based sun safety education for children in

elementary school. Journal of Cancer Education, 23, 74–79.Bussiere, P., Knighton, T., & Pennock, D. (2007). Measuring up: Canadian results of the OECD PISA study, the Performance of Canada’s youth in Science, Reading and Mathematics.

Ottawa: Human Resources and Social Development Canada & Statistics Canada.Chandra, V., & Lloyd, M. (2008). The methodological nettle: ICT and student achievement. British Journal of Educational Technology, 39(6), 1087–1098.Craig, W., Harel-Fisch, Y., Fogel-Grinvald, H., Dostaler, S., Hetland, J., Simons-Morton, B., et al. (2009). A cross-national pro!le of bullying and victimization among adolescents

in 40 countries. International Journal of Public Health, 54, S1–S9.Craig, W. M., Pepler, D. J., & Blais, J. (2007). Responding to bullying: what works? International Journal of School Psychology, 28, 15–24.Craig, W., Pepler, D., & Shelley, D. (2004). Summary of interventions to address Bullying problems at school. Report prepared for the Ministry of Education. Government of

Ontario.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175174

De Castell, S., & Jenson, J. (2003). OP-ED: serious play. Journal of Curriculum Studies, 35(6), 649–665.ERIN Research Inc., & Wings, C. (2005). Young Canadians in a wired world – Phase II: Student survey. Canada: Media Awareness Network.Hall, L., Woods, S., Hall, M., & Wolke, D. (2007). Children’s emotional interpretation of synthetic character interactions. Affective Computing and Intelligent Interaction, 4738,

642–653.Huffaker, D. A., & Calvert, S. L. (2003). The new science of learning: active learning, metacognition, and transfer of knowledge in e-learning applications. Journal of Educational

Computing Research, 29(3), 325–334.Kshirsagar, V. Y., Agarwal, R., & Bavdekar, S. B. (2007). Bullying in schools: Prevalence and short-term impact. Indian Pediatrics, 44(1), 25–28.McLaughlin, L., Laux, J. M., & Pescara-Kovach, L. (2006). Using multimedia to reduce bullying and victimization in third-grade urban schools. Professional School Counselling, 10

(2), 153–161.Mitchell, K. J., Ybarra, M., & Finkelhor, D. (2007). The relative importance of online victimization in understanding depression, delinquency, and substance use. Child

Maltreatment, 12, 314–324.Nansel, T. R., Craig, W., Overpeck, M. D., Saluja, G., Ruan, W. J., & Health Behavior in School-aged Children Bullying Analyses Working Group. (2004). Cross-national consistency

in the relationship between bullying behaviors and psychosocial adjustment. Archives of Pediatrics & Adolescent Medicine, 158(8), 730–736.Olweus, D. (1993). Bullying at school: What we know and what we can do. Oxford: Blackwell.Paiva, A., Dias, J., Sobral, D., Aylett, R., Woods, S., Hall, L., et al. (2005). Learning by feeling: evoking empathy with synthetic characters. Applied Arti!cial Intelligence, 19,

235–266.Pellegrini, A., & Bartini, M. (2001). Dominance in early adolescent boys: af!liative and aggressive dimensions and possible functions. Merrill-Palmer Quarterly, 47, 142–163.Pepler, D. (2006). Bullying Interventions: a binocular perspective. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 15, 16–20.Pepler, D., Jiang, D., Craig, W., & Connolly, J. (2008). Developmental trajectories of bullying and associated factors. Child Development, 79, 325–338.Rigby, K. (2002). A meta-evaluation of methods and approaches to reducing bullying in pre- schools and in early primary school in Australia. Canberra: Commonwealth Attorney-

General’s Department.Sapouna, M., Wolke, D., Vannini, N., Watson, S., Woods, S., Schneider, W., Enz, S., et al. Virtual learning intervention to reduce victimization to primary school: a controlled

trial. Journal of Child Psychology and Psychiatry, in press.Shih, M., Feng, J., & Tsai, C. (2008). Research and trends in the !eld of e-learning from 2001 to 2005: a content analysis of cognitive studies in selected journals. Computers and

Education, 51, 955–967.Smokowski, P. R., & Kopasz, K. H. (2005). Bullying in school: an overview of types, effects, family characteristics, and intervention strategies. Children & Schools, 27, 101–110.Tto!, M. M., & Farrington, D. P. (2009). What works in preventing bullying: effective elements of anti-bullying programmes. Journal of Aggression, Con"ict and Peace Research, 1

(1), 13–24.Vaughan, A., & Pepler, D. (2007). Bullying. In L. Beal (Ed.), The ABC’s of Mental Health Teacher Resource. The Hincks-Dellcrest Centre. http://www.brocku.ca/teacherresource/

ABC/.Vygotsky, L. (1978). Mind in Society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.Woods, S., Hall, L., Dautenhahn, K., & Wolke, D. (2007). Implications of gender differences for the development of animated characters for the study of bullying behavior.

Computers in Human Behavior, 23, 770–786.Ybarra, M. L., & Mitchell, K. J. (2004). Youth engaging in online harassment: associations with caregiver-child relationships, Internet use, and personal characteristics. Journal

of Adolescence, 27(3), 319–336.

A. Rubin-Vaughan et al. / Computers & Education 56 (2011) 166–175 175