Research Review Chapter1

8/2/2019 Research Review Chapter1

1/28

1

Copyright 2011, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Chapter 1

DOI: 10.4018/978-1-60960-495-0.ch001

EDUCATIONAL POTENTIALS OFCOMPUTER GAME PLAY

Researchers have long suggested that science in-struction should provide students with opportuni-ties to explore the world, and to make connectionsbetween these explorations and their personal lives(e.g. Aikenhead, 2007; Linder et al., 2007; Zeidler,2007). Science educators in many countries have,accordingly, worked for decades to infuse inquiry

into the school, but good scientific inquiry seemsto be hard to implement in classrooms (Ekborg etal., 2009; Linder et al., 2007). Evaluations report,that given the constraints of classroom settings,real world data collection and laboratory experi-ments are difficult to conduct, meaning that thereare limited opportunities for teaching higher orderinquiry skills in the ordinary classroom (Linder etal., 2007). The contextual clues offered to teach-ers by textbooks tend to lead away from inquiry(Phelps & Lee, 2003). Game-based learning ap-proaches, on the other hand, are constructed to

Gunilla Svingby

Malm University, Sweden

Elisabet M. NilssonMalm University, Sweden

Research Review:Empirical Studies on ComputerGame Play in Science Education

ABSTRACT

The interest for game-based learning is growing among science educators. A range of research reviews

have been published regarding the educational potentials of using computer games as a tool for learning

and mediation, but on a general level. This research review focuses on empirical studies conducted on

computer game play specically used to enhance science learning. 50 publications published during the

last decade were found that met the criteria of presenting empirical data from students using games forlearning science in school contexts. The studies are reviewed and analysed according to: type of game,

research design, research interests and research methodology, school subject and content, number and

age of students, time spent on the intervention, gender, and teacher roles. The scope and quality of the

studies are also discussed.


2/28

2

Research Review

situate learners in complex and authentic tasks.Given the widely acknowledged lack of studentinterest in school science, and the downward trend

in results (e.g. Jidesj & Oscarsson, 2006; Linderet al., 2007; Osborne, 2007), the educationalpotential of computer game play (e.g. Aiktin,2004; Gee, 2003; Klopfer, 2008; Williamson,2009) might be of interest to science educators.To achieve this, authentic problems, conceptsand processes are embedded in the narrative, thatprovide scope for scientific inquiry (Barab et al.,2007; 2007a, 2007b; Ketelhut, 2007; Magnussen,2008; Neulight et al. 2007).

When comparing scientific literacy or science

education standards (e.g. NRC, 1996; OECD,2003) with the characteristics of computer games,some striking correspondences can be found.Squire and Jan (2007), for example, identify fivecore features pertinent to designing computergames for learning. (1) Games ask students toinhabit roles. Players are encouraged to createidentities that blend the game player role and therole as a scientific professional. All information,experiences and rewards occur within this role,leading to the development of specific skills andcompetencies mediated by digital tools (e.g. digitallab equipment). (2) The activities in the gameare organised around challenges and rewards,designed to support engagement, collaboration,and learning. (3) The games offer opportunitiesto tie goals to particular places, particularly sitesof contested spaces. (4) The games allow forembedding authentic resources and tools that en-able acting on higher levels. Digital tools, such asresearch labs and calculators, both mediate playand provide opportunities for players to interactwith the environment in new ways. (5) Playinggames is fundamentally social, and producessocial interaction. After having been met withscepticism at the outset, and seen as only play,public interest in computer games as learning toolsseems to be spreading internationally (Van Eck,2006, p. 2). On the other hand, even if theoreticalassumptions ascribe computer games great poten-

tial for learning, strict empirical research is stilllacking, to explain if and why computer gamesare effective in practice, and if so, under which

conditions (e.g. Egenfeldt-Nielsen, 2007; Hang-hj, 2008; Linderoth, 2009; Williamson, 2009;Wong et al., 2007). Aitkin (2004) compares thenature of todays science research with simula-tion games, and points to simulation as the coreof most scientific research today.

To meet the need for more empirical observa-tion on the possibilities gaming can provide, anumber of educational computer games have beendeveloped, in various university projects. In thesegames, researchers have attempted to combine

socio-cultural or constructivist approaches tolearning, with the affordances of contemporarycomputer games, with the aim to engage students inauthentic, deep forms of inquiry (e.g. Barab et al.,2007a, 2007b; Beckett & Shaffer, 2005; Squire &Jan, 2007). The focus is on offering affordances ofmaking observations, posing questions, gatheringdata, experimenting, examining books, and so on,as tools for scientific inquiry processes (Ketelhut,2007). Using educational games, students areinvited to explore and negotiate contested spaces.Besides the computer games which have specifi-cally been developed for educational purposes inthese projects, the possibility of using commercialoff-the-shelf (COTS) games for science learninghas also been investigated in a few studies (Nils-son & Jakobsson, Accepted; Nilsson & Svingby,Accepted; Steinkuhler & Chmiel, 2006). The aimof the present review is to review computer gameprojects intended to enhance the teaching andlearning of science, also including COTS games.

EARLIER REVIEWS OFCOMPUTER GAME PLAY INRELATION TO LEARNING

Even the first review studies of computer gameplay and learning (e.g. Van Sickle, 1986; Randelet al., 1992) showed that computer games could


3/28

3

Research Review

promote learning and reduce instructional time,across a row of disciplines. Referring to studiesfrom the eighties and nineties, Van Eck (2006)

concludes that, on the whole, these studies reportof learning gains. However, he also points to thelow number of studies that report empirical stud-ies undertaken according to established researchcriteria.

In a recent meta-analysis, the effects of com-puter games on learning and attitudes to learningwere summarised for various subject matters (Vo-gel et al., 2006). The authors evaluated a total of248 studies, covering the time period 19862003,over all sorts of subject content and spanning all

ages. They found that only 32 of 248 studies metstrict experimental criteria, formulated as eachstudy must have identified cognitive gains or at-titudinal changes, and reported statistics assessingtraditional classroom teaching versus computergaming or interactive simulation teaching (p.232). Because of what the authors call method-ological flaws, a majority of the articles couldnot be included in the meta-analysis. Among thestudies included, most involved adults and highereducation, and more studies dealt with medicineor similar content, rather than with school sub-jects. The scientific criteria applied conform totraditional experimental design criteria for certainquantitative research and thus, exclude studiesusing for example anthropological or ethnologicalmethods. Neither does the meta-analysis includestudies developed according to a design-basedresearch tradition (Barab & Squire, 2004; Wang& Hannafin, 2005). Being a recent research field,the study of computer games for science learningstill has a focus on the exploration of the tool, andthe methodological criteria applied for excludingthe majority of studies examined in the meta-analysis are not necessarily entirely relevant to thefield. Given these restrictions, however, Vogel etal. (2006) showed that persons using interactivegames attained significantly higher cognitive gainsand better attitudes toward learning, comparedto subjects exposed to traditional instruction.

The authors conclude that across all situations,games or interactive simulations will most likelyinstruct subjects with better cognitive outcomes

and attitudes toward learning when compared totraditional teaching methods (p. 235). The resultswere shown to hold across all age groups.

In a recent meta-study (Boyle, 2009), that in-cluded studies of all kinds of content and all typesof computer games, but was restricted to studiesengaging persons above 16, Boyle found that ofmore than 2,000 articles published since 2004, only52 met the criteria of including empirical data onlearning, and/or engagement. The outcome of thestudies that did include empirical data was reported

as positive impacts on learning and engagement.The situation today is similar to the situation inthe 1980s: even if much has been written aboutcomputer game play, relatively few studies reporton empirical data. Van Eck (2006, p. 4) suggests asan explanation the difficulty of measuring effectsof computer game play on learning by standardisedtests. There is thus the problem of comparing withthe effects of traditional school teaching. He arguesthat the demands in traditional evidence-basedresearch of narrowly defined variables and tightlycontrolled conditions, may lead to narrow claimsabout learning, that is, the accurate reproductionof facts and simple concepts, whereas computergames embed potentials of more complex learning.

RESEARCH FOCUSOF THIS REVIEW

Several research reviews consider computer gameplay in relation to learning and motivation to learn(see e.g. De Freitas, 2007; Kebritchi & Atsusi,2008; Kirriemuir & McFarlan, 2004; Linderoth,Lantz-Andersson & Lindstrm, 2002; Susi et al.,2007). However, we have not found any studyfocusing the relations between computer gameplay and science learning, more specifically.

The present article reviews empirical studiesconducted on computer game play specifically


4/28

4

Research Review

used at school to enhance science learning, andto favour more positive attitudes towards scienceand science learning.

Criteria and Limitations

The studies included in this review meet the fol-lowing criteria:

1. the computer game is used for science learn-ing in formal school settings, which excludescomputer games used only for entertainment,and studies undertaken outside of schoolcontexts,

2. the analytical focus lies on students aged618 (K-12),

3. empirical data is included,4. the study is peer-reviewed,5. the study was published between 1999 and

2009.

Studies with strict experimental design us-ing statistical analyses, and studies with a moreexplorative design using qualitative data are bothincluded.

Search Methods

In order to find publications that met the criterialisted above, several search strategies were used. Ina first round, databases and other archive resourceswere explored. The following databases, and ar-chive resources were used to find relevant articles:Digiplay, DiGRA Digital Library, Games Studies,Google scholar, ERIC, Innovate, Science Direct,SAGE. The following search terms were appliedin different combinations: computer games,video games, digital games, educational games,simulation games, serious games or gaming, andlearning, education, instructional, and science,physics, chemistry, biology or technology.

The database search was supplemented bysearches in conference proceedings, earlierreviews, web sites, and literature lists. In this

way, more than 200 publications were foundthat related to the use of computer game play atschool in science instruction. Of these, 50 pub-

lications met the criteria listed above. Many ofthe publications found in the first search round,dealt with theoretical assumptions concerningthe educational potential of computer game play.Since the review had a focus on empirical stud-ies, these publications were sorted out. The mostfrequent reason for excluding a text from thisreview, thus, was absence of empirical data. Themajority of the articles included were publishedin peer-reviewed journals. Some were publishedin peer-reviewed conference proceedings (8 of 50

items), as book chapters (3), or as peer-reviewedbut unpublished doctoral theses (4). A majorityof the articles selected were published between2005 and 2008 (43 of 50 items). As the samestudy in some cases was presented in more thanone publication, the number of empirical studiesis smaller than the number of articles.

Besides directly applying the criteria listedabove, a number of studies were excluded in orderto maintain the specific focus of this review. Quitea few articles dealt with the educational potential ofletting students design their own computer games.Such studies were excluded, since designinggames is a different type of activity than playinggames. Articles dealing with ways to improveprogramming skills through computer game play,or game design activities, or computer games usedin computer science/programming courses werealso sorted out. Finally, since the analytical focuswas on the gaming students, articles dealing solelywith pedagogical methods or practical barriers inthe classroom were excluded.

Definition of a Computer Game

The definition ofgame presented some problemswhen choosing studies to be included. The litera-ture on games research provides a range of differentdefinitions of a game. A recent definition viewsgames as a system in which players engage in


5/28

5

Research Review

artificial conflict, defined by rules, and resultingin a quantifiable outcome (Salen, 2008, p. 268).Games are further often described as transmedial

phenomena, implying that the same game can betransmitted through different kinds of media: onpaper, via computers, digital networks, consoles,handhelds, mobile phones etc. (Juul, 2005). Here,only games presented by some sort of digitaltechnology were included.

The demarcation between a game and a simula-tion also called for reflection. Some of the gameapplications used in the studies included, may beplaced on the borderline, between what wouldbe defined as computer games within the games

research community, and what may instead bedescribed as asimulation. However, with time, thisdistinction has blurred, as is shown by (Sauv etal., 2007), and as science simulations and gamestend to merge, it may no longer be relevant forthis subject area (Aitkin, 2004).

The concept ofmulti-user virtual environment(MUVE) may cause another problem of defini-tion. A MUVE can be described as a computerisedenvironment that enables multiple participants tosimultaneously access virtual contexts, to interactwith digital artefacts, with other participants, aswell as with computer-based agents, and to engagein collaborative learning activities (Dede et al.,2002; Murray, 1997). A MUVE does not neces-sarily contain gaming activities. The MUVEsincluded in the review do, however, offer suchactivities.

We have on the whole solved the dilemmaof game definition by accepting the definitionsmade by the authors, and/or the publishing jour-nals (revealed via key words used, forum forpublication, or expressions used in the text). Thereview, thus, includes empirical studies involvingany kind of game sustained by digital technology.Games may in the various articles be signified bythe terms: alternate or augmented reality game,immersive game, interactive game, interactivelearning environments, multi-user virtual envi-ronments (MUVE), simulation game, simulation,

video game, or virtual reality game. Mainly forthe sake of simplicity, the term computer gameis used in the following as an umbrella term for

all kinds of games, whatever digital platform theyare played on.

Structuring the Article

The studies reviewed are analysed according to thefollowing criteria, which also structure the article.

A. Type of computer gameB. Research designC. Research interests

D. School subject and contentE. Number and age of studentsF. Time spent on the interventionG. GenderH. Teacher roles

Publications Included in the Review

FIRST ROUND OF RESULTS: THERESEARCH SETTING

Computer Game Genre

The studies are categorised according to whichof the following types of computer game thestudents played.

Research-Based EducationalComputer Games

The category has three sub-categories: (1) Edu-cational MUVEs (played on PC), (2) Augmentedand alternate reality games (played on handheldsor PC), and (3) Educational single-player games(played on PC). In the following, the variouscomputer game genres are presented.


6/28

6

Research Review

Table 1. Overview of the 50 publications that met the listed criteria, and are included in the review

Author(s) Year Computer game Type of game

Annetta, Mangrum, Holmes, Collazo, Cheng 2009 Dr Friction MUVE

Barab, Arici, Jackson 2005 Quest Atlantis MUVE

Barab, Dodge, Tzn, Job-Sludwer, Jackson, Arici, Job-Sluder,Carteaux, Gilbertson, Heiselt

2007 Quest Atlantis MUVE

Barab, Ingram-Goble, Warren 2009 Quest Atlantis MUVE

Barab, Sadler, Heiselt, Hickey, Zuiker 2007 Quest Atlantis MUVE

Barab, Zuiker, Warren, Hickey, Ingram-Goble, Kwon 2007 Quest Atlantis MUVE

Beckett, Shaffer 2005 Madison 2200 AR gameEpistemic game

Cai, Lu, Zheng, Li 2006 The protein game Educational single-player game

Cameron, Dwyer 2005 Metalloman Educational single-player game

Clarke, Dede 2005 River City MUVE

Colella 2000 The virus game AR game

Facer, Joiner, Stanton, Reid, Hull, Kirk 2004 Savannah AR game

Hansmann, Scholz, Francke, Weymann 2005 Simulme Educational single-player game

Hickey, Ingram-Goble, Jameson 2009 Quest Atlantis MUVE

Jenkins, Squire, Tan 2004 Supercharged! Educational single-player game

Kafai 2008 Whyville MUVE

Kafai, Feldon, Fields, Giang, Quintero 2007 Whyville MUVE

Kao, Galas, Kafai 2005 Whyville MUVE

Ketelhut 2006 River City MUVE

Ketelhut 2007 River City MUVE

Ketelhut, Dede, Clarke, Nelson, Bowman 2007 River City MUVEKlopfer, Yoon, Rivas 2004 The virus game AR game

Lim, Nonis, Hedberg 2006 Quest Atlantis MUVE

Magnussen 2005 Homicide AR gameEpistemic game



Marsh, Wong, Carriazo, Nocera, Yang, Varma, Yoon, Huang,Kyriakakis, Shahabi

2005 Metalloman Educational single-player game

Mathevet, Le Page, Etienne, Lefebvre, Gigot, Prorol, Mau-

champ

2007 Butorstar AR game (RPG game supported

by computer simulations)Mellor 2001 Build your own time machine Educational single-player game

Miller, Moreno, Estrera, Lane 2004 MedMyst Educational single-player game

Nelson 2005 River City MUVE

Nelson 2007 River City MUVE

Nelson, Ketelhut 2007 River City MUVE

Nelson, Ketelhut 2008 River City MUVE

Nelson, Ketelhut, Clarke, Bowman, Dede 2005 River City MUVE

continued on following page


7/28

7

Research Review

Educational MUVEs

Educational MUVEs are a form of socio-construc-tivist and situated cognition-based educational

software. Educational MUVEs incorporate 2Dand 3D virtual worlds, in which learners controlcharacters that represent them in the world (Cobbet al., 2002; Nelson et al., 2005; Nelson & Ketel-hut, 2007). The content varies widely; each virtualworld can have its own theme.

A growing body of research indicates that basedon this relatively new form of technology, educa-tional MUVEs can be designed to support highlyinteractive scientific inquiry learning, thus offeringa safe approach to scientific inquiry (Nelson &

Ketelhut, 2007). Advocates of MUVEs argue thatthey open up new possibilities for creating learningexperiences that are authentic, situated and dis-tributed, and also provide a context for changingthe standards by which students accomplishmentsare assessed (Dieterle & Clarke, 2008).

We have found the following educationalMUVEs designed for science learning.

River City

The large-scale project River City, at HarvardUniversity (funded by the US National ScienceFoundation) was created by a team of researchers

representing a variety of expertise, and specifi-cally designed for use in formal school settings(Clarke & Dede, 2005; Ketelhut, 2007; Nelson &Ketelhut, 2007). It builds on highly immersive 3Dtechnology. TheRiver City virtual world consistsof a city with a river running through it. The learn-ers themselves populate the city in teams of three,along with computer-based agents. Students workin teams to develop hypotheses regarding one ofthree strands of illness in the city. At the end ofthe project, students compare their research with

other teams, to discover the range of potentialhypotheses and possibilities to explore. The basicidea is that students learn to behave as scientiststhrough they collaboratively identifying problemsvia observation and inference, forming and test-ing hypotheses and deducing evidence-basedconclusions about underlying cause (Nelson etal., 2005, p. 3). Nearly 10,000 students in the US

Author(s) Year Computer game Type of game

Neulight, Kafai, Kao, Foley, Galas 2007 Whyville MUVENilsson, Svingby 2009 Agent O AR game

Price 2008 Unreal Tournament COTS game

Rieber, Noah 2008 Computer-based simulationof acceleration and velocity

Educational single-player game

Rosenbaum, Klopfer, Perry 2007 Outbreak @ The Institute AR game

Squire 2006 Supercharged! Educational single-player game

Squire, Barnett, Grant, Higgenbotham 2004 Supercharged! Educational single-player game

Squire, Jan 2007 Mad City Mystery AR game

Squire, Klopfer 2007 Environmental Detectives AR game

Svarovsky, Shaffer 2006 SodaConstructor Educational single-player game

Svarovsky, Shaffer 2007 SodaConstructor Educational single-player gameToprac 2008 Alien Rescue III Educational single-player game

Tzn 2007 Quest Atlantis MUVE

Wong, Shen, Nocera, Carriazo, Tang, Bugga 2007 Metalloman Educational single-player game

Wycliffe, Muwanga-Zake 2007 Zadarh Educational single-player game

Table 1. continued


8/28

8

Research Review

and internationally are reported to have completedthe computer lab-basedRiver City curriculum, aspart of their middle school science classes.

Quest Atlantis

The Quest Atlantis project at Indiana Universityfocuses on the ability to support authentic scien-tific inquiry and collaboration in realistic virtualcontexts (Barab et al., 2007, 2009). The narrativetakes students to a city with a great deal of envi-ronmental problems. Students take part in a largenumber of quests, to save the people of the virtualAtlantis from destruction through environmental,moral and social decay.

Whyville

Whyville is a MUVE designed to support scientificlearning and inquiry, but outside the formal school(Kafai, 2008; Kao et al., 2005). It is a 2-dimensionalMUVE to be used in informal settings, relatingto biology, physics and chemistry. When used inschools, students were confronted with a diseaseoutbreak in theWhyvillevirtual community, whichmanifested itself as red spots on the faces of theavatars. The illness soon spread through thecommunity. Students tracked the spread of theoutbreak on charts in their classroom, used aninfection simulator, and gathered informationin a virtual centre for disease control. Galas(2006) found that students could conduct au-thentic, collaborative scientific inquiry, and gotdeeply involved in gathering data and forminghypotheses. Neulight et al. (2007) focused on theimpact on students understanding of the causes ofreal-world diseases. Using pre- and post-surveysshowed a significant improvement in the numberof students moving from pre-biological to bio-logical understandings.

Augmented and AlternateReality Games

One approach to creating stronger connectionsbetween students experience of the real world,

and students actions in a virtual model of a com-plex ecological system is to link real and virtualelements, in augmented reality learning environ-

ments. In these environments, participants areexposed to both a physical and virtual reality, thusproviding students with multiple representationsfor solving complex problems. While virtual real-ity attempts to replace the real world, augmentedreality seeks only to supplement it (Klopfer,2008). Augmented reality learning environmentsenable students to take the technology out of theirclassrooms, and use it to explore the environmentaround them (Klopfer & Squire, 2008; Squire &Klopfer, 2007). Learning environments designed

with augmented reality technologies are seen asenabling students to participate in the processof scientific investigation, because they providestudents with the opportunity to act as scientists,in a situation that blends the real and the virtual.Such learning environments may help to resolvethe existing dichotomy between indoor technol-ogy environments and outdoor experiences, byusing mobile technologies in the context of natureexploration. It is argued that augmented or alter-nate reality bridges reality and virtual reality byletting students play in the reality or using realdata (Beckett & Shaffer, 2005; Klopfer, 2008;Rosenbaum et al., 2007).

Educational Single-PlayerComputer Games

This kind of computer games are primarilydesigned for individual play, and do not allowstudents to interact outside of the game, or onlinewith other gamers. They represent an earlier andless advanced type of educational game, comparedto the previous. Mostly, the games of this type arenot built on the same stable ground of researchersrepresenting different disciplines, including thelearning sciences.


9/28

9

Research Review

COTS Games

COTS games are designed to be entertaining, not

primarily educational, and not for school use.Teachers say, however, that students are allowed toplay such games at school (e.g. Williamson, 2009).Studies reporting empirical results on the use ofCOTS games for science learning are includedamong the studies reviewed, but not studies thatonly write of the potential of a game. In relationto type of game, the publications were distributedas follows in Table 2.

The majority of the games reported of wereresearch-based educational games, most of them

developed by researchers in a design-based re-search process. It is striking that even if COTSgames are reported as the type of game mostoften used in schools (e.g.Williamson, 2009),little research seems to have been conducted oneffects of these games on learning and attitudes,judged by the number of publications. Thedominance of studies using educational MUVEs,compared with the total number of games, maybe explained by the support of grants offered bythe US National Science Foundation for this re-search. The difference between the MUVE andthe other games is demonstrated by the numberof research articles published on each game. Thefour MUVE games have resulted in 22 publica-tions, whereas studies of the 7 educational single-player computer games are reported in 15 publi-cations.

Research Design

As mentioned above, some researchers have

criticised studies on computer game play for notbeing undertaken according to certain researchcriteria, including standardised measurements,control groups and randomisation (e.g.Vogel,2008). Other researchers argue that to apply suchcriteria is not adequate, in relation to the complexcompetences that many of the games are built topromote, and may in fact be counterproductivein the development of a potent tool for sciencelearning (Squire & Klopfer, 2007). In our experi-ence, explorative studies are of great value in the

process of developing an educational game. Thesame is true for the study of students interactionsand explorations when playing. Other measuresthan standardised tests and standard scales arealso needed to investigate the more advancedcompetences, as defined by the US National Sci-ence Standards (NRC, 1996), for example.

As a consequence, we have here categorisedthe studies as experimental or explorative.A study categorised as experimental includes theuse of pre- and post-test, standardised measure-ments of learning and attitudes, or other types ofcontrolled evaluation, as well as adequate statisti-cal analyses. Some but not all such studies alsoemploy control groups, randomisation and otherexperimental arrangements. Explorative stud-ies focus on observation of gaming behaviour,and of interactions in and outside of the game.Interview, log book, game activity tracking, and

Table 2. Computer games played in the studies

Computer game genre Number of publications Game title

Educational MUVE 22 Dr Friction, River City, Quest Atlantis, Whyville

AR games 11 Agent O, Butorstar,Environmental detectives, Homicide,Mad City Mystery, Madison 2200, Outbreak @ the Institute,Savannah, The Virus game

Educational single-player games 15 Alien Rescue III, Build your own time-machine,, MedMyst,Simulme, SodaConstructor, Supercharged!, Zadarh

COTS games 1 Unreal Tournament 2004


10/28

10

Research Review

various observation methods are used. Some ofthe publications combine the two designs.

Experimental and Explorative Design

Table 3 shows the distribution between the twodesigns.

About half of the publications reported ofstudies with an experimental design including theuse of pre- and post-test. Eleven of these alsoincluded explorative data. The assessment toolsmostly consisted of both standardised multiplechoice/short answer questions and of contentrelated tasks. Control groups were used in a third

of the studies. These studies used statisticalanalyses in order to determine effect sizes andreliability. They were mostly designed accordingto design-based research methodology, with theaim to introduce innovations into real worldclassroom contexts, study the activities and inter-actions emerging, and at the same time scientifi-cally research the impact on learning and derivescientifically grounded claims (Nelson et al., 2005;Hickey et al., 2009).

Another group of studies were designed withan ethnographic, and/or design interest, exploring,for instance, how students related to the computer

game, which activities they engaged in, how theycollaborated, how the game was included into theclassroom, and teachers roles. In other words,the majority of the experimental studies focusedmore on the processes of playing the game andless on the strict measuring of effects.

To sum up, roughly two thirds of the stud-ies included in the present review apply one ofthe standard elements of an experimental study,indicating that published studies on computergames used to enhance science learning meet the

expectations and criteria that are usual in quan-titative research studies within various scientificdisciplines. Taking into account the position ofeducational computer games for science learn-ing, the amount of studies focusing on studentsactivities and interactions when playing, as wellas the effects of various parallel technical andeducational solutions, is promising. The researchfield is in need, not only of studies undertaken ac-cording to strict empirical rules, but also of studiesthat seek to understand the medium.

Research Interest

The research questions of the experimental stud-ies mainly focused one or more of the variables:learning, engagement, and attitude to the subjectcontent. In addition, the explorative and some ofthe experimental studies posed questions concern-ing the relations between playing the game and

Table 4. Research interests of the studies. Some studies take an interest in all or several of these ques-

tions, thus the total number of research interests differs from the actual number of studies.

Research interest Number of studies

Learning 25

Engagement 15

Attitudes to the subject 12

Other aspects 17

Table 3. Research designs

Type of study

Number

of studies

Experimental only 16

Explorative only 20

Combining experimental and explorative 11

Pre- and post-test 28

Pre- and post-test plus control group, randomisation 13


11/28

11

Research Review

the use of inbuilt guidance systems, of continuingmotivation, of students attendance to classes, ofcollaborative work, of use of representations, of

teachers roles, or other similar issues. Such re-search interests are summed up by Other aspects.

Both quantitative and qualitative methods wereused to evaluate the research interests. Learningwas mostly measured by standardised tests, butin some studies also by other means, aiming tocapture more genuine and complex socioscien-tific qualities. Engagement and attitudes wereassessed by surveys, interviews and by observa-tions. The quality of the methodology used andanalyses applied corresponded to established

methodological practice for these types of researchinterests.

School Subject and Content

Many of the computer games played in the studieswere designed to enhance scientific inquiry andscientific practice, without explicitly implyingspecific school subjects. Others were more specific

about the subject content explored. Our analysisfound that the studies had the following focus.

The science content of most of the games can

be classified as environmental studies, and health.In combination with such content, a row of stud-ies explored the theoretical assumption thatcomputer games help to make science inquiryattainable at school. Games focusing on chemis-try and physics are relatively few. In addition,there is a relationship between the age of thestudents and content focus, with chemistry andphysics primarily included in games for olderstudents.

Number and Age of Students

The studies vary substantially in the number ofstudents involved: from 8 to 1,666. The variation isdue to research interest and design with explorativestudies involving fewer students, as a rule, andexperimental studies involving a moderate to verylarge number of students. However, as mentionedabove, some of the experimental studies withmany students also include explorative methods.

We have constructed a variable with fourlevels: few students (


12/28

12

Research Review

that contained only few students. The number ofstudents involved is largely related to the type ofstudy undertaken, with few students involved inthe explorative studies.

The age of the students varies from 9 to 18years. We have constructed an age variable with

four levels: 16. The studiesare distributed on age as shown in Table 7.

With respect to the age of the students, themiddle school ages (1114) are overrepresented.

Time Spent on the Intervention

Students spent from 1 hour (60 minutes) to severalmonths on the intervention. The distribution onfour levels is shown here.

In the majority of the studies, students spend

relatively little time with the game. 18 publicationsreport of an intervention that takes only a few hourswhereas 10 studies report of an intervention thatgoes on for one or more months. There is anotherfactor involved in this; in the longer studies, gameplay is integrated more or less into the curriculumand students do not play all the time. It is obviousthat the time of exposure to the game play interven-

tion has an impact on what can be learned andexperienced. Another aspect is the way the gameis played; in groups or alone. In more than half ofthe studies, the students worked in groups, and in

a few other studies the students worked in pairs.

SECOND ROUND OFRESULTS: EFFECTS

We will now turn to the question of possible ef-fects of game play on science learning, attitudes,and engagement.

Science Learning and Engagement

Of the 19 studies that report on learning effects,16 report positive changes in students resultson tests or other forms of assessment, as a resultof the game play. In the eight studies includingcontrol groups, all but one showed the experimentgroup to gain significantly better results than thecontrol group on standardised tests. In a so-calledevidence-based study, more than 1,000 studentsplayed River City for three weeks in authenticschool settings. Significant gains were demon-

strated with respect to scientific inquiry skills(Nelson et al., 2005; Nelson & Ketelhut, 2007). Inan experimental study, Squire et al. (2004) showedthat the game Supercharged! enhanced studentsunderstanding of a complex physics phenomenon.By introducing a virtual disease game in Whyville,Neulight et al. (2007) demonstrated that studentsgained scientific knowledge relating to the cau-

Table 7. Age of students. Since some studies include more than one age group the numbers exceed the

numbers of reviewed publications (50).

Age of students Number of studies

16 13

Table 8. Time spend on intervention

Time Number of studies

14 hours 18

14 days 13

13 weeks 6

4 weeks or more 10


13/28

13

Research Review

sality of natural infectious diseases. The studentsperceived the game simulations as similar to anatural infectious disease, and used the experi-

ences gained during game play to deepen theirunderstanding of natural diseases. Engagementin the game play was shown to support studentsconceptual understanding. The results confirmthe theoretical assumptions that when designedin according to the criteria set forward in theoriesof educational game play, educational computergames do in fact immerse students in activitiesthat help develop deep learning qualities, whilepracticing skills relevant to scientific inquiry.

The problem of valid assessment of deep

learning qualities in science was, however, alsoilluminated in the studies. General standard assess-ments do not cover all of the knowledge qualitiesaimed for by the games. To assess qualities ofscientific inquiry defined as a multifaceted ac-tivity that involves making observations; posingquestions; examining books and other sources ofinformation to see what is already known; planninginvestigations; using tools to gather, analyze, andinterpret data; proposing answers, explanations,and predictions; and communicating the results(NRC, 1996) other types of tests are needed (Jns-son, 2008; Jnsson & Svingby, 2008).

In a study reported by Ketelhut et al. (2007),it was shown that on a set of standard questionswith multiple-choice/short answers, students play-ingRiver City got the same results as the controlgroup students. On a content-related task, however,that involved suggesting measures to improve theenvironment of a city, the experiment group per-formed far above the control group students. Thestandard tests were, in other words, shown unableto provide information concerning the inquiryskills developed by the students who had playedthe game. In consequence, some of the subsequentstudies by Ketelhut focused on alternate measuresof effects. A number of studies were designed ac-cording to the design-based research model. Thesestudies include more specific experimental stud-ies. As part of a US National Science Foundation

funded project that implementedRiver City withnearly 3,000 students, Ketelhut (2007) studied thedata gathering behaviour of 96 seventh graders.

Data gathering behaviour included: (1) makingobservations by visiting various information sites,(2) using tools, by accessing water sampling andbug-catching stations, (3) gathering evidencefrom book material, from library books, andhospital records, (4) accessing information fromother sources, students interacted with guidancemessages in the individualised guidance system;and (5) posing questions, students asked for in-formation from virtual agents (p. 104).

Students were randomly assigned to one of

two guidance treatments (Nelson et al., 2005)that offered one or three guidance hints to helpdevelop students understanding. Ketelhut (2007)studied students of the high guidance group intheir second, third and fourth visit to the city. Sheassumed that students with more knowledge andhigher science self-efficacy would be gatheringmore data, as well as data from different sources,compared to students with less knowledge andlower science self-efficacy. Material concerningstudents behaviour was processed using fit-ted multi-level models, to address the researchquestions:

is growth in total scientic inquiry behav-iour related to self-efcacy,

does gender affect this relationship, does the data gathering behaviour develop

over time?

The results initially confirmed the assumption,showing that self-efficacy and students contentpre-test score both impacted their scientificdata-gathering behaviour. Furthermore, studentsscientific self-efficacy measured by a Likert scalecorrelated significantly with students pre-testscore. From the start, there was a significantdifference between students with high and lowcontent score/self-efficacy and data-gathering.Boys and girls with high self-efficacy gathered


14/28

14

Research Review

more data than students with low self-efficacy andalso data from more different sources. Over timethe results changed. During visit four, scientific

data gathering was not affected by students initialcontent score/self efficacy at all. Girls with initiallow self-efficacy showed the strongest positiverate of change. A slightly different pattern waspresented for different data sources assessed. Touse the information offered by the game leadsto better science learning. Initially, girls withhigh pre-test scores benefitted from the potentialsources offered by the game by gathering muchmore information than girls with low scores.Gradually, this difference disappeared.

The results contradict the argument that noveltywas the prime reason why students were engagedinRiver City (Ketelhut, 2007). Had novelty beenthe reason for student engagement, the engage-ment would have diminished over time. Ketelhutfurther argues that the complexity of the gameworld adapts to all kinds of students, meaning forexample, that low self-efficacy regarding sciencebecame irrelevant in the game world thus helpingsuch students to engage in science learning. Theresults, further, suggest that embedding scientificinquiry in the MUVE might act as a catalyst forchanging students self-efficacy and learning pro-cesses. The results give support to Lemkes (1990)analysis of science education as being presented asadifficultsubject resulting in that [w]hen studentsfail to master it, they are encouraged to believe itis their own fault: they are just not smart enoughto be scientists (p. 138). Ketelhut (2007) argues,in the same line, that if schools cannot convincestudents that science is achievable, there is a riskof confirming the idea that science is only for afew elite students. Lack of interest, frequentlyreported for science teaching, is part of the sameproblem. In this perspective, the results of the 15studies reporting on engagement can be seen asencouraging. Twelve of the studies reported posi-tive engagement, or even enjoyment. Students say,for example, that they appreciated the dynamicnature of the game world, seeing that their actions

had an effect on the outcome of the game in arealistic way (Rosenbaum et al., 2007, p. 43).Others stress that the inquiry component of the

game play was not only experienced as a scien-tific process, but also very social and involvedunderstanding people, which deeply engagedthe students and led to rich social negotiations(Barab et al., 2007a, p. 71). Engagement is in somestudies measured by standardised attitude scales,and in other studies by interviews.

Three studies, however, reported negativeresults on student engagement. Firstly, in a studyof 76 students aged 1618, only half of the stu-dents said that they enjoyed playing the physics

gameBuild your own time machine, which has afocus on astronomy (Mellor, 2001). Secondly, ina study of 8 students in a Singapore school, whoplayed Quest Atlantis, initial interest diminishedover time, when students found that the game didnot exactly match the assessment test they wouldbe taking (Lim et al., 2006). Students immersedin the game lost focus on the learning task, im-plying that the game play served as a distractingelement. The study points to the dilemma of aneducational game built to be almost as interest-ing as a COTS game, but implemented in schoolsystems that follow quite different principles.Finally, observations of a similar kind are reportedby Nelson (2005). The game had an inbuilt guid-ance system designed to help students learn theembedded scientific formalism. Nelson (2005)showed that the students who used the systemperformed better. However, only 25 percent ofthe students, mainly girls, used the teaching helpoffered. Nelson concludes that for playing thegame, the guidance system was not a necessity. Touse it was thus a function of students ambitionsto learn, and not primarily to play.

That limited assessment practices may be aproblem when assessing engagement was illus-trated in a study by Toprac (2008), on the motiva-tion to continue studying science. Students aged14 years played the gameAlien Rescue IIIfor ninehours. On a standardised questionnaire, students


15/28

15

Research Review

answered in a way that implied that they were notmotivated to further study the subject. However,when observed working with the game and when

students were interviewed, they expressed a totallydifferent view. They said that they enjoyed playingthe game, and it was observed that they continuedto discuss issues from the game in their free time.

Taken as a whole, the studies examined in thepresent review indicate that game play as part ofscience education frequently results in positiveengagement. Science education through gameplay appears to be much more enjoyable thantraditional science classes.

Gender

It has been assumed that girls are less engagedby gaming technologies than boys (e.g. Krotoski,2004). A handful of studies report on girls andboys engagement in the games, and of learn-ing effects related to gender. Van Eck (2006a)showed that girls and boys gaming strategiesvary substantially. A simulation game was playeddifferently by the two groups: the girls tended todiscuss and build dwellings complete with bath-rooms, hot tubs, and pools; boys, on the other hand,tended to discuss and create swamps, crocodiles,and jaguars to the exclusion of everything else(p. 5).The assumption of girls being less engagedwas not confirmed by these studies. The study byKetelhut (2007), for example, showed that girlswith low self-efficacy gained most by the game,and showed the greatest changes in scientific datagathering behaviour.

The positive impact of games on girls be-haviour was also shown by Mellor (2001), whodemonstrated girls to be more able to approacha physics-based computer game from a lesspredetermined position and seemed to be happyviewing physics as an imaginative and speculativeexercise (p. 287). The boys, on the other hand,tended to act as collectors of facts, rather thanexplorers. The researcher concludes that the boysare vulnerable to feeling patronized and have

difficulty in negotiating situations in which theusual boundaries between the classroom and thehome between education and leisure have been

broken down (p. 289). The issue of how genderrelates to computer game play for science learningis, in other words, far from straightforward. Evenif a computer game may appeal equally to girlsand to boys, they do not necessarily appreciatethe same things about the game.

THIRD ROUND OF RESULTS:RESULTS RELATED TOCOMPUTER GAME GENRE

When analysing the studies, some interestingvariations in results could be noted, that mightdepend on the type of computer game played.

Studies Using Educational MUVEsCompared to Studies UsingEducational Single-Player Games

Most of the studies using these two types of gamesare designed as experimental studies, with controlgroups, pre- and post-tests, and strict statisticalanalyses. A number of differences are revealedwhen comparing studies that use these two typesof games.

Many more MUVE-based studies report onstudents engagement in the game, as well asdiscussing various aspects of students interactionwith the game, with each other and with the teacher.These interests are related to the overall designof the studies adopted: design-based research. Inline with the design, such studies make use ofa variety of evaluation measures and methodsof analysis in an iterative circle of evaluativeand explorative research elements based in realworld settings (Barab & Squire, 2004; Wang &Hannafin, 2005).

When the MUVE-based studies are comparedto other studies, a range of differences are revealed:number of students involved, the age of the stu-


16/28

16

Research Review

dents, the time spent with the intervention, andthe subject matter of the game. The educationalsingle-player game studies all involved many

students, but on the other hand, lasted over ashort period of time. The MUVE-based studiesincluded both experimental studies with verymany students (more than 1,000), and explorativestudies with a small number of students involved.Six of the studies lasted for two or more weeks.No study based on educational single-playergames lasted that long, and the majority lastedonly a few hours. The students playing the edu-cational single-player game studies were olderthan the students playing the MUVEs, and the

content of the games was different. A majorityof the MUVEs concentrated on general science,specified as scientific inquiry of socio-scientificissues for middle school students. The specificcontent concerned questions relating to ecologicaland health issues. The educational single-playergame studies, on the other hand, focused on coreaspects of the subjects of chemistry, biology andphysics for students aged 1516.

The findings also show different patterns. Theeffects on learning and engagement reported fromthe MUVE studies were overwhelmingly positive.When control groups were used, students gotbetter results than the control group students, andwhen pre- and post-test scores were compared,the gaming students developed more positively. Inall but one study, the results hold for standardisedknowledge tests as well as for alternative waysof assessing students learning. By contrast, onlyhalf of the educational single-player computergame studies reported learning results, and insteadfocused on other aspects. In those studies wherelearning was assessed, the results were positive.Furthermore, the two types of games resulted indifferent levels of student engagement. All butone of the educational MUVEs demonstratedstrong engagement and enjoyment, whereas theeducational single-player game studies showedstudents to be less engaged.

We have found only one exploratory studyreported on COTS games (Price, 2008). Thesestudies point to possible learning effects and

positive attitudes, but also to problems of keep-ing students focused on the science content. Noexperimental studies concerning COTS gameswere found.

Example of Studies

To illustrate the results we will present a fewstudies in depth.

Outbreak @ the Institute

A series of augmented reality games played onhandheld computers have been developed byresearchers at MIT Teacher Education Program(e.g. Klopfer, 2008; Klopfer & Squire, 2007). Anexample of an exploratory study of such a gameis Outbreak @ the Institute (Rosenbaum et al.,2008). The players are presented to a fictionalscenario: the outbreak of a new form of bird fluthat has become transmissible between humans.Players may encounter both bird flu, as well as thecommon seasonal flu. Some students are alreadyexhibiting symptoms. Players worked as a teamto gather information, using the tools available tothem in order to try to stop the outbreak. Playersjoined the game in one of three possible roles, eachhaving specific competences and tools. As thereare no specific criteria for winning the game,players must decide what their goals should bethroughout the game. No teachers were involvedin the game play. The study involved twenty onestudents aged 17 years, playing for two hours.Data included observations, as well as a pre- andpost-survey. The survey asked for a list of five fac-tors in the game that would influence the numberof sick people. Students appeared to perceive thegame as authentic, in several ways. Observationsdemonstrated that students behaved in ways sug-gesting that they felt personally embodied in thegame. The roles felt authentic, and the possibilities


17/28

17

Research Review

of communication and collaboration were seen asimportant. During the course of the game, studentsgoals shifted from initially knowledge-based, to

more personal and team-based. The results wereused to re-design the game.

Mad City Mystery

The augmented reality gameMad City Mysterywas developed by researchers at the Universityof Wisconsin-Madison. They explored studentsdevelopment of scientific argumentation skillswhen were playing a location-based augmentedreality game on handheld computers (Squire &

Jan, 2007).The study is an example of studiesthat adopt an ethnological type of qualitativeresearch design. These studies concentrate onexploration, using ethnographic methods, withmostly qualitative measures. The research focusis not on learning effects, but on understandinghow and why learning may occur.

The game story is about a person found dead ina lake close to the University campus. The task ofthe player is to investigate the case, and to piecetogether an explanation. This is done in groups.Players interview virtual characters, gather data,and examine documents. Playing the game takes90120 minutes. The study reports of 28 students,aged 916 years, playing the game with no teachersinvolved. The study aims to explore the hypothesisthat a game designed according to establishedprinciples of contemporary computer games canbe a useful tool, for supporting students in develop-ing scientific argumentation skills (Squire & Jan,2007). The study investigated whether the gamecould engage students in scientific thinking, inspecific hypothesis formation, as well as reasoningfrom evidence. Qualitative methods were used,including observations and interviews, but also aquestionnaire on students attitudes toward scienceand gaming. All student groups were observed toengage in argumentation cycles, similar to thoseadvocated by science educators, and thought to bedifficult to produce in classrooms. The research-

ers concluded that playing the game requiredstudents to weigh evidence, develop hypotheses,test them against evidence, and generate theories

based on the evidence, thus engaging students inthe practice of scientific argumentation.

Quest Atlantis

The educational MUVE Quest Atlantis exempli-fies the high quality of the research-based projectsfunded by the US National Science Foundation.The game was developed by a team of researchersand designers at Indiana University.Quest Atlantisis intended to engage children aged 912 in a form

of dramatic play. At the core is the narrative aboutAtlantis, a world in trouble. Sheltered within adigital game, Quest Atlantis introduces studentsto inquiry processes, as well as to practices ofsocial commitment. Through the Quest Atlantiscommunity, students can complete quests, talkwith other children, and build their virtual persona.Completing quests requires children to participatein academically meaningful activities, either inthe real world, or through simulation. Two-weekcurricula composed of 58 quests with a commonfocus (e.g. water quality) are presented to teachersas unit plans. Teachers may also create their ownquests, through a Teacher Toolkit (Barab et al.,2007, p. 158). Quest Atlantis has more than 3,500registered members, from all over the world.

The project has a well developed theoreticaland educational basis. Quest Atlantis is describedas a situationally embodied curriculum, thatrelates scientific formalisms to contexts (Barab etal., 2007b). The researchers underline their inter-est to use gaming principles to embed standards-based science concepts, without underminingthe situative embodiment. The project group hasdesigned a context for learning, in the intersectionof education, entertainment, and social action. Theauthors refer to it as a context for participation(Barab et al., 2007, p. 159), underlining that thisform of learning context is not to be seen simplyas a technological innovation (Barab et al., 2005,


18/28

18

Research Review

2007). The project was developed using design-based research methodology. A team of researchersand other experts, representing various disciplines

and expertise, progressively refined the projectin ongoing cycles of research, design, tests, andexperiments. A series of mainly explorative studieshave been reported from the project. Data werederived from students from Australia, Denmark,China, Malaysia, Singapore, and the US. Theinterventions mostly lasted two weeks, but onestudy went on for two years, calculating variousdata from 3,279 students (Barab et al., 2005, 2007).Among the 3,279 registered users, approximately2,500 were elementary school children, aged 912

(in public and private schools), 450 undergraduatestudents, and 50 teachers.

By means of the computerised records ofchildrens choices and accomplishments in thequests, as well as by participant observation andinterviews, students participation has been infocus. In building qualitative interpretations, theresearchers used multiple raters, and wheneverpossible, calculated inter-rater reliabilities. Learn-ing gains were assessed by pre- and post-tests(Tzn, 2007). The research design also includedexperimental studies on sub-samples, wherestudents were randomly assigned to either theQuest Atlantiscondition, or to a worksheet controlcondition, to test the effectiveness of the QuestAtlantis context (Barab et al., 2009). Statisticallysignificant learning gains were documented forscience. For example, elementary students takingpart in a unit on plant and animal cells demon-strated significant gains in their understanding ofthe cell (Barab et al., 2007).

Simulme

Hansmann et al. (2007) report of two experimentalstudies using the educational single-player com-puter gameSimulme. This game is built to developstudents knowledge of ecological and economiceffects of food consumption, and give them theopportunity to reconsider their own consumption

patterns. During the game play, the gamers hadto make six virtual purchases of meat and veg-etables. Each purchase represented the players

typical consumption pattern. Within the game, theplayers choices were said to influence the nationaland global ecological and economic situation.After each purchase, the player received feedbackon the consequences of his or her consumptionpattern. To achieve good performance, the playershould increase the consumption of food that wasproduced according to organic farming standards,regionally produced, and decrease the share ofmeat. The game involved answering questionson the origin, type of cultivation, and mode of

production of various provisions. The study reportsof a 90 minutes intervention, with six experimentclasses, and six control classes (aged 17 years),as part of biology curriculum. The same teach-ers taught in both conditions. Both students andteachers judged the lessons using the game morefavourably, than the lessons not using the game.Statistical analysis showed that, regardless ofteacher, students profited more from instructionusing Simulme compared to instruction withoutthe game. In a second experiment with the samegame, 212 students were randomly assigned to theexperiment (98) and control situations (114). Theexperiment group playedSimulmebefore shoppingin an online store, whereas the control group madetheir shopping at once. The experimental groupspurchasing behaviour significantly differed fromthe control group, in the sense that the studentswho had played the game bought more organicproducts and more regionally produced food thanthe control group. No differences were found inthe buying of meat.

As is shown above, the design, scope and focusof research varies substantively with game genre.The most consistent and varied research involvesthe educational MUVEs (e.g. River City, QuestAtlantis). Extensive research is also reportedon the augmented reality games (e.g.Mad CityMystery, Outbreak @ the Institute). Each of therest of the games included in this review (close


19/28

19

Research Review

to ten games) has only been subjected to one ortwo studies, many which represent the exploratoryphase. Nevertheless, even if the number of stud-

ies is limited, certain tendencies related to typeof game may be noted.

The research studies published on the MUVEsRiver City and Quest Atlantis undoubtedly showthat carefully designed computer games can suc-cessfully support real world inquiry practices,that are equally compelling for girls and boys.The games are characterised by the use of mostof the characteristics of COTS games that makesuch games immersive. They build on a storylinewhich appeals to the age group. The narrative

unfolds as a result of students activities. Aloneor together, students have to solve a row of is-sues and tasks, using tools, skills and processestypical of scientific practice. The student acts viaan avatar, representing her/himself in the game,and can also interact with virtual agents. Attemptsare made to imbed scientific formalisms into theissues that students work with. The games areeffective as tools for learning, because learningtakes place within a meaningful context, and isapplied in this context. Feedback is integratedinto the game play.

The designers of some games attempted tocross the border between the virtual and the realworld, by locating the game in a real place, andhaving students play the fictional story in a placein the physical world. These so-called augmentedreality games were played on handheld comput-ers, in teams of students who had to collaborate inorder to gather data, and find plausible solutionsto the problem at hand. Two gamesMadison 2200and Homicide, referred to as alternate realitygames and epistemic games, used real data,as well as the epistemology of professionals inthe real world. The games thus put students in afictional story, working according to the practicesof the real profession, and with data belonging tothe profession. The games were studied in smallexploratory studies, mostly using ethnographic

data. Very positive results on learning and engage-ment were reported.

COTS Games for Science Learning

In contrast to the above studies, that all report oneducational computer games, carefully developedby researchers and teachers, the review can saylittle about the possibilities for science learningembedded in COTS games. Amazingly, veryfew studies have reported results of this kindof games used in school contexts. Even thoughsuch games are reported to be most frequentlyplayed in classrooms (e.g. Williamson, 2009),

few researchers have studied learning outcomes,and/or interactions with teaching. The few stud-ies available report of some possible learningbenefits, but also of many problems that emergefrom the way the games treat science contentand processes. Still, as other researchers havesuggested (e.g. Gee, 2003) it might be possibleto integrate some COTS games in the sciencecurriculum, taking advantage of their immersivequalities and potentials for inquiry and problemsolving. To realise this, teachers need to occupya much more central position.

SUMMARY AND CONCLUSION

It has been argued that computer games can affordlearning contexts where students are supported inthe practice of scientific inquiry and experimen-tation. This is achieved by providing scientifictools (authentic resources) that are used in thegame play, a process that is claimed to be similarto the process of scientific inquiry. The reviewconfirms the theoretical assumptions, that whendesigned in according to the criteria set forwardin theories of educational game play, educationalcomputer games do offer students the possibilitiesof practicing skills relevant to scientific inquiry.


20/28

20

Research Review

The review is based on 50 publications pub-lished during the last decade (19992009) thatwere found to meet the criteria of presenting em-

pirical data from students using games for learningscience in school contexts. Compared to the overallamount of studies reporting of empirical studiesof science education and learning, this is a sur-prisingly small number, and illustrates the limitedinterest in computer games for science learning.The majority of the games in the studies weredeveloped by researchers, mostly at universitiesin the US. The number of games developed andreported on, for example, by European research-ers is strikingly low (9 out of 50). The number of

publications that report on COTS games used forscience learning is extremely low. It can thus, beconcluded that reports on computer games usedfor educational purposes in the science curriculumare likely to be based on games that were fromthe start designed to be educational, since so littleresearch on COTS games exists.

The studies reviewed apply either one, or amixture of the following two research designs.(1) A type of evidence-based design which iscarried out in authentic school settings. Thesestudies applied standard methodology includingcontrol groups, pre- and post-test and conformedto experimental criteria, such as having identi-fied cognitive gains/attitudinal changes, usingstandardised assessment scales and analysing thedata by adequate statistics, in some studies even onadvanced level. (2) An ethnographical field studydesign, focusing on observations and recordingsof what happens in the interactions between stu-dents and the game, as well as between studentsand teachers or other students.

Many of the studies combined elements ofthe two designs. Only a small number of studiesmet the criteria adopted by Vogel et al. (2006) ofcomparing traditional classroom teaching withoutthe game to teaching with the game (e.g. Ketelhut,2007; Hansmann, 2007; Toprac, 2008). Most ofthe games were designed for elementary andmiddle school students. The analysis revealed

that the games built for these age groups focusedmore on enhancing general science inquiry skills,and less on developing the use of formal science

concepts. Specific content addressed was further,more often ecology and biology than physics andchemistry. Specific science concepts and rigidscientific simulations were more often addressedby studies aimed for older students.

When learning effects were assessed, play-ing the computer game was found to contributepositively to students science learning. In studiesincluding control groups, the gaming students gotbetter results, both on standardised and content-based tasks, compared to the non-gaming students.

The importance of using various types of assess-ment and evaluation methods and the advantagesin combining quantitative and qualitative datawere also effectively demonstrated. An impor-tant aspect is the enjoyment and engagement inscience learning that the games brought about.On the whole, the studies reviewed indicate thateducational games for science learning have thepotential to favourably influence students attitudetowards science learning and, as a consequence,to accelerate students learning. Even if almost allstudies report on positive effects and experiences,it is, as pointed to by Egenfeldt-Nielsen (2007,p. 8) unrealistic to expect the computer games toalways facilitate a desired educational outcome,since such outcomes seldom is a part of the gameculture. Computer game play is an activity ofgreat variation that can take many directions, andoutcomes may therefore correspond to teachersexpectations in some cases, while leading to quitedifferent outcomes in others.

An important notion, thus, is the situatednessof game play. Playing computer games in an insti-tutionalised setting is clearly a different activity,compared to playing computer games in times ofleisure (Linderoth, 2009).To solve problems in acomputer game as a part of an assignment in aschool context, is a different matter compared toplaying the game outside school.


21/28

21

Research Review

Learning processes are intertwined with thesurrounding culture, and constitute a situatedpractice that cannot be extracted from the context

in which they occur (Lave & Wenger, 1991; Slj,2005). Computer games brought into a schoolsetting are thus cultural product, associated toexpectations based on students previous experi-ences of game play outside school. When computergame play is actively situated in a regular schoolsituation and made part of it, it can be shown thatstudents bring forth, and make use of scientificconcepts and theories (Barab et al., 2007a, 2007b;Svingby & Jnsson, 2007).

The ethnographical studies allowed for analy-

sis of what was going on when students used thegames for learning. Observation revealed that,even when they attained good results on the sci-entific standard tests, students only occasionallyapplied domain formalism while playing (Barabet al., 2007a, 2007b; Rosenbaum et al., 2007;Squire et al., 2004). It was further observed thatwhen students applied scientific reasoning skills,they sometimes displayed inconsistency betweenconclusions and solutions, made inaccurate sci-entific assumptions, and underestimated socialimpacts (Barab et al., 2007b; Squire et al., 2004).Rosenbaum et al. (2007) report, for example,that students understood the dynamics of thegame, but misunderstood some of the details ofthe scientific matter at stake in the game (that is,disease transmission mechanism). It seemed as ifstudents drew this mental model of disease fromtheir everyday experiences, and that the gamedid not provide enough feedback to challenge it(Nelson, 2007).

Lack of clear connection to the curriculum, orof follow up by the teacher in the classroom, seemsto result in students not reaching the potentialsfor learning offered (Nelson, 2005; Rehn et al.,2007). Such results suggest that students need toconsciously relate the game play to a learning pur-pose. Most of the studies reviewed, accordingly,take an interest in the part played by the teacher inthe intervention. Game development has followed

two main paths in designing educational computergames: (1) to include the support, structure andhelp by the game itself, or/and by other players

(which may include a teacher). In the educationalMUVEs, teachers are mostly not built into thegame. Instead, virtual agents play the role ofhelping, structuring and giving feedback. Teachersmay in any case take advantage of the game. (2)Designing a game to be administered, controlledand supplemented by the teacher. The designershave thus either from the outset calculated withteachers active participation, and given themadequate training; or the studies showed thatteachers integrated the game into the curriculum

by adding data, gathering students to reflect onactivities and solutions suggested by the studentsin the game, or other similar initiatives.

In studying the game HomicideMagnussen(2005, 2008) clearly demonstrates the importanceof the mediating activities brought forward by theteacher. In this game the teacher is further, given aspecific role, by acting as chief detective. As such,s/he can contribute new information, as well asgather teams of students for briefing and reporting.In a physics game, where students were acting aselectronic particles, Squire et al. (2004) furthernoticed that by prompting deeper reflection, theteacher added qualities to the game play. In stillanother study, Rieber and Noah (2007) showedthat the teacher played a significant role in help-ing students to make sense of and learn aboutthe science content. It was observed that in theabsence of such help, boys focused too much onthe competitive parts of the game. Teachers lowgaming competence and lack of computer gameplay experiences were in some studies shown tobe an obstacle to involvement.

According to the studies reviewed, both the twostrategies outlined above may work, suggestingthat the role of the teacher is a promising area forfurther research. This conclusion is strengthenedby recent studies of teachers attitudes to usingcomputer games at school. Teachers were reportedto hold mainly positive attitudes towards games,


22/28

22

Research Review

but also feelings of not knowing where to findgood computer games, and what to do withthem (e.g. Williamson, 2009).

This implies the importance for teachers andschools to consciously strive to integrate the gameinto the curriculum and for game developers toeither integrate educational guidance systemsinto the game, or/and integrate the game as partof the curriculum. Close collaboration betweendevelopers, designers and teaching experts arerecommended. If computer games are to be usedin the classroom appropriate instructional actionsare required. The studies suggest that use of in-formation, guidance etc. needs to be guaranteed.

To sum up, computer games hold a promisefor engaging students in science education and tohold their interest. To do so, the game play needto be integrated in the curriculum in a way thatguarantees that students are both engaged by thegame and made to stop and discuss and reflect onthe experiments, concepts and theories relevant tosolve the problems presented in the game world.

REFERENCES

Aikenhead, G. (2007). Expanding the researchagenda for scientific literacy. In Linder, C., stman,L. & Wickman, P.-O. (Eds.),Promoting scientificliteracy: Science education research in transition.

Proceedings of the Linnaues Tercentenary Sympo-

sium, Uppsala University.Uppsala: Geotryckeriet.

Aitkin, A. L. (2004).Playing at reality: Exploringthe potential of the digital game as a medium for

science communication. Unpublished doctoraldissertation, Faculty of Science, The AustralianNational University.

Annetta, L., Mangrum, J., Holmes, S., Collazo, K.,& Cheng, M.-T. (2009). Bridging reality to virtualreality: Investigating gender effect and studentengagement on learning through video game playin an elementary school classroom.InternationalJournal of Science Education, 31(8), 10911113.doi:10.1080/09500690801968656

Barab, S., Ingram-Goble, A., & Warren, S. (2009).Conceptual Play Spaces. In Ferdig, R. E. (Ed.),Handbook of Research on Effective Electronic

Gaming in Education. Hershey, PA: IGI Globalpublications.

Barab, S., Zuiker, S., Warren, S., Hickey, D., In-gram-Goble, A., & Kwon, E.-J. (2007b). Situation-ally embodied curriculum: Relating formalismsand contexts. Science Education, 91(5), 750782.doi:10.1002/sce.20217

Barab, S. A., Arici, A., & Jackson, C. (2005). Eatyour vegetables and do your homework: A design-based investigation of enjoyment and meaning in

learning.Educational Technology, 65(1), 1521.Barab, S. A., Dodge, T., Tzn, H., Job-Sludwer,K., Jackson, D., & Arici, A. (2007). The Quest At-lantis Project: A socially responsive play space forlearning. In Shelton, B. E., & Wiley, D. (Eds.), Theeducational design and use of simulation computer

games. Rotterdam: Sense Publishers.

Barab, S. A., Sadler, T. D., Heiselt, C., Hickey, D.,& Zuiker, S. (2007a). Relating narrative, inquiry,and inscriptions: Supporting consequential play.

Journal of Science Education and Technology,16(1), 5982. doi:10.1007/s10956-006-9033-3

Barab, S. A., & Squire, K. (2004). Design-basedresearch: Putting a stake in the ground.Journal ofthe Learning Sciences, 13(1), 114. doi:10.1207/s15327809jls1301_1

Beckett, K. L., & Shaffer, D. W. (2005). Augmentedby reality: The pedagogical praxis of urban plan-ning as a pathway to ecological thinking.Journalof Educational Computing Research,33(1), 3152.doi:10.2190/D5YQ-MMW6-V0FR-RNJQ

Boyle, L. (2009).Keynote lecture at the 3rd Eu-ropean Conference on Games Based Learning.University of Applied Sciences, Graz, Austria.

Cai, Y., Lu, B., Zheng, J., & Li, L. (2006).Immersive protein gaming for bio edutain-ment. Simulation & Gaming, 37(4), 466475.doi:10.1177/1046878106293677


23/28

23

Research Review

Cameron, B., & Dwyer, F. (2005). The effectof online gaming, cognition and feedback typein facilitating delayed achievement of different

learning objective.Journal of Interactive LearningResearch, 16(3), 243258.

Clarke, J., & Dede, C. (2005). Making learningmeaningful: An exploratory study of using multi-

user environments (MUVEs) in middle school.Paper presented at American Educational ResearchAssociation Conference, Montral, Qubec.

Cobb, S., Neale, H. R., Crosier, J. K., & Wilson,J. R. (2002). Development and evaluation ofvirtual learning environments. In Stanney, K. M.(Ed.),Handbook of virtual environments: Design,implementation, and applications. Mahwah, NJ:Lawrence Erlbaum Associates, Inc.

Colella, V. (2000). Participatory simulations:Building collaborative understanding throughimmersive dynamic modeling. Journal of theLearning Sciences, 9(4), 471500. doi:10.1207/S15327809JLS0904_4

De Freitas, S. (2007). The learning in immersiveworlds: A review of game based learning report.UK: JISC e-Learning Programme, Higher Educa-tion Funding Council for England (HEFCE).

Dede, C., Ketelhut, D. J., & Ruess, K. (2002, Octo-ber).Motivation, usability, and learning outcomesin a prototype museum-based multi-user virtual

environment. Paper presented at The Fifth Inter-national Conference of Learning Sciences, Seattle.

Dieterle, E., & Clarke, J. (2008). Multi-user virtualenvironments for teaching and learning. In Pagani,

M. (Ed.),Encyclopedia of multimedia technologyand networking. Hershey, PA: Idea Group, Inc.

Egenfeldt-Nielsen, S. (2007).Educational poten-tials of computer games. New York: Continuum.

Ekborg, M., Ideland, M. & Malmberg, C. (2009)Science for life-a conceptual framework foranalysis and construction of socio-scientific cases.NorDiNa, 10(1).

Facer, K., Joiner, R., Stanton, D., Reid, J., Hull,R., & Kirk, D. S. (2004). Savannah: Mobile gam-ing and learning?Journal of Computer AssistedLearning, 20(6), 399409. doi:10.1111/j.1365-2729.2004.00105.x

Galas, C. (2006). Why Whyville?Learning andLeading with Technology, 34(6), 3033.

Gee, J. P. (2003). What video games have toteach us about learning and literacy. New York:Palgrave Macmillan.

Hanghj, T. (2008).Playful knowledge: An explor-ative study of educational gaming. Unpublished

doctoral dissertation Copenhagen: Institute ofLiterature, Media and Cultural Studies, Universityof Southern Denmark.

Hansmann, R., Scholz, R. W., Francke, C.-J. A. C.,& Weymann, M. (2005). Enhancing environmentalawareness: Ecological and economic effects offood consumption. Simulation & Gaming, 36(3),364382. doi:10.1177/1046878105279116

Hickey, D. T., Ingram-Goble, A. A., & Jameson,E. M. (2009). Designing assessments and assess-

ing designs in virtual educational environments.Journal of Science Education and Technology,18(2), 187208. doi:10.1007/s10956-008-9143-1

Jenkins, H., Squire, K., & Tan, P. (2004). Youcant bring that game to school! Designing super-charged! In Laurel, B. (Ed.), Design research:Methods and perspectives. Cambridge, MA:MIT Press.

Jidesj, A., & Oscarsson, M. (2006). Studentsattitudes to science and technologyfirst results

from the ROSE-project in Sweden. In Janiuk, I.R. M., & Samonek-Miciuk, E. (Eds.), Scienceand technology education for a diverse world.

Dilemmas, needs and partnerships. Lublin, Po-land: Marie Curie-Sklodowska University Press.


24/28

24

Research Review

Jnsson, A. (2008).Educative assessment for/ofteacher competency. A study of assessment and

learning in the interactive examination for stu-

dent teachers. Unpublished doctoral dissertation,Malm University, Malm.

Jnsson, A., & Svingby, G. (2008). The use of scor-ing rubrics: Reliability, validity, and educationalconsequences. Educational Research Review,2(2), 130144. doi:10.1016/j.edurev.2007.05.002

Kafai, Y. B. (2008). Understanding VirtualEpidemics: Childrens Folk Conceptions of aComputer Virus. Journal of Science Educationand Technology, 17(6), 523529. doi:10.1007/s10956-008-9102-x

Kafai, Y. B., Feldon, D., Fields, D., Giang, M., &Quintero, M. (2007).Life in the times of Whypox:A virtual epidemic as a community event. Paperpresented at The Third International Conferenceon Communities and Technology, New York.

Kao, L., Galas, C., & Kafai, Y. (2005).A totallydifferent world: Playing and learning in multi-

user environments. Paper presented at DIGRA2005 Conference: Changing Views Worlds inPlay, Vancouver.

Kebritchi, M., & Hirumi, A. (2008). Examining thepedagogical foundations of modern educationalcomputer games. Computers & Education, 51(4),17291743. doi:10.1016/j.compedu.2008.05.004

Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratoryinvestigation in River City, a multi-user virtualenvironment.Journal of Science Education and

Technology, 16(1), 99111. doi:10.1007/s10956-006-9038-y

Ketelhut, D. J., Dede, C., Clarke, J., Nelson, B., &Bowman, C. (2007). Studying situated learning ina multi-user environment. In Baker, E., Dikieson,J., Wulfeck, W., & ONeil, H. (Eds.),Assessmentof problem solving using simulations. New York:Lawrence Erlbaum Associates.

Kirriemuir, J., & McFarlan, A. (2004).Literaturereview in games and learning. Bristol: FutureLabseries.

Klopfer, E. (2008).Augmented learning: Researchand design of mobile educational games. Cam-bridge, MA: MIT Press.

Klopfer, E., & Squire, K. (2008). Environmentaldetectivesthe development of an augmented real-ity platform for environmental simulations.Edu-cational Technology Research and Development,5(2), 203228. doi:10.1007/s11423-007-9037-6

Klopfer, E., Yoon, S., & Rivas, L. (2004). Com-parative analysis of palm and wearable computersfor participatory simulations.Journal of ComputerAssisted Learning, 20(5), 347359. doi:10.1111/j.1365-2729.2004.00094.x

Krotoski, A. (2004). Chicks and joysticks: An ex-ploration of women and gaming. Entertainment andLeisure Software Publishers Association. RetrievedJanuary 31, 2010, from www.elspa.com/assets/files/c/chicksandjoysticksanexplorationofwom-enandgaming_176.pdf

Lave, J., & Wenger, E. (1991). Situated learning:

Legitimate peripheral participation. New York:Cambridge University Press.

Lemke, J. (1990). Talking science: Language,learning, and values. Norwood, NJ: Ablex.

Lim, C. P., Nonis, D., & Hedberg, J. (2006).Gaming in a 3D multi-user virtual environment:Engaging students in science lesson.British Jour-nal of Educational Technology, 37(2), 211231.doi:10.1111/j.1467-8535.2006.00531.x

Linder, C., stman, L., & Wickman, P.-O. (Eds.).(2007). Proceedings of the Linnaues Tercente-nary Symposium, Uppsala University. Uppsala:Geotryckeriet.

Linderoth, J. (2009). Hur datorspelande kan ge enillusion av lrande. In Selander, S., & Svrdemo-berg, E. (Eds.),Didactic designs in digital envi-ronments: New learning opportunities. Stockholm:Liber.


25/28

25

Research Review

Linderoth, J., Lantz-Andersson, A., & Lindstrm,B. (2002). Electronic exaggerations and virtualworries: Mapping research of computer games

relevant to the understanding of childrens gameplay. Contemporary Issues in Early Childhood,3(2), 226250. doi:10.2304/ciec.2002.3.2.6

Magnussen, R. (2005). Learning games as aplatform for simulated science practice. Paperpresented at DiGRA 2005 Conference: ChangingViews Worlds in Play, Vancouver.

Magnussen, R. (2007). Teacher roles in learninggameswhen games become situated in schools.Paper presented at DiGRA 2007 Conference:Situated Play, Tokyo.

Magnussen, R. (2008).Representational inquiryin science learning games. Unpublished doctoralthesis, Danish School of Education, University ofAarhus, Copenhagen

Marsh, T., Wong, W. L., Carriazo, E., Nocera,L., Yang, K., Varma, A., et al. (2005). User ex-periences and lessons learned from developing

and implementing an immersive game for the

science classroom. Paper presented at The 11thInternational Conference on Human-ComputerInteraction, Portland, Oregon.

Mathevet, R., Le Page, C., Etienne, M., Lefebvre,G., Gigot, G., & Prorol, S. (2007). BUTORSTAR:A role-playing game for collective awareness ofwise reedbed use. Simulation & Gaming, 38(2),233262. doi:10.1177/1046878107300665

Mellor, F. (2001). Gender and the communicationof physics through multimedia. Public Under-

standing of Science (Bristol, England), 10(3),271291. doi:10.1088/0963-6625/10/3/302

Miller, L., Moreno, J., Estrera, V., & Lane, D.(2004). Efficacy of MedMyst: An Internet teach-ing tool for middle school microbiology.Journalof Microbiology & Biology Education, 5, 1320.

Murray, J. H. (1997).Hamlet on the Holodeck:The future of narrative in cyperspace. New York:The Free Press.

National Research Council. (1996). Nationalscience education standards. Washington, DC:National Academy Press.

Nelson, B. (2007). Exploring the use of indi-vidualized, reflective guidance in an educationalmulti-user virtual environment. Journal of Sci-ence Education and Technology, 16(1), 8397.doi:10.1007/s10956-006-9039-x

Nelson, B., & Ketelhut, D. J. (2007). Scientific

inquiry in educational multi-user virtual environ-ments.Educational Psychology Review, 19(39),265283. doi:10.1007/s10648-007-9048-1

Nelson, B., & Ketelhut, D. J. (2008). Exploring em-bedded guidance and self-efficacy in educationalmulti-user environments. International Journalof Computer-Supported Collaborative Learning,3(4), 413427. doi:10.1007/s11412-008-9049-1

Nelson, B., Ketelhut, D. J., Clarke, J., Bowman,C., & Dede, C. (2005). Design-based research

strategies for developing a scientific inquirycurriculum in a multi-user virtual environment.Educational Technology, 45(1), 2134.

Neulight, N., Kafai, Y. B., Kao, L., Foley, B., &Galas, C. (2007). Childrens participation in avirtual epidemic in the science classroom: Makingconnection to natural infectious disease.Journal o

Documents

Research Review Chapter1