www.dcs.shef.ac.uk/~daniela/publications.html

Serious Games & Virtual Environments for

Education and Entertainment

Daniela M. RomanoDaniela M. Romano

ContentContent

• Serious Games Definition and Concepts

• Virtual Reality Technology & Games

• AI & Games

• Believable Synthetic Characters

• Natural Interfaces Interfaces

GameGame

Playful*, Goal Driven Activity (with or without a computer)

Aspects:• Rules • Challenges• Choices • Competition & Goals • Fantasy & Engaging aspects

*lively and full of fun => Engaging

Benefits of Playing GamesBenefits of Playing Games• Whatever the content there are

benefits associated with video games playing.

• Research has shown improvements in mood, reductions in stress, and feelings of competence and autonomy, with positive impact on wellbeing

• Playing with others have revealed benefits in terms of social wellbeing and feelings of relatedness

• Co-operative video game play is associated with increased brain activity

Read more: http://www.smh.com.au/comment/the-brain-gain-why-video-games-can-make-people-happy-20131008-2v63o.html#ixzz2tl7HucBU

History of Computer Games

Computer Games have outgrown from computer research in fields such as:•Artificial Intelligence•Virtual Reality

Early arcade video games

(1971–1977)

Pong was the first arcade video game with widespread success

History of Computer Games

Pong: Past and Present

Brain Computer Interface PongPong original game

History of Computer GamesPackman: Past and Present

Now packman is used in AI research to test algorithm strength in a finite environment

Packman for Apple IIPackman Arcade

Science Fiction becomes RealityScience Fiction is influenced and influences our reality especially in the area of Games, Serious Games and Gamification

Science Fiction & RealityScience Fiction is influenced by Research advancements …… and in turn influences our reality especially in the area of Games, Serious Games and Gamification

ROBOTICSAI & Emotion

3D Graphics VRBrain Computer Interface

3D Graphics and VR

3D Graphics and VR

ABM & 3D graohics

The Serious Games Convergence

The Serious Games Convergence

Serious Games are simulations that use

entertaining and engaging aspects

of computer games with the aim to engage the users with

an educational content.

LearningEnvironments

Learning EnvironmentsLearning EnvironmentsEdutainment

– The fun is separated from the learning– Chocolate-covered broccoli (Jacob Hobgood, Sumo Digital)

– The user has fun and engages with the environment, learns,

has fun, engage, learns, ....

FUNFUN

ENGAGE learnENGAGE learn

Learning EnvironmentsLearning EnvironmentsSerious Game

– The fun is embedded in the learning activity, within the ‘gameplay’

– No apparent linear sequence and possibilities are unlimited (or at least the limitations are not apparent to the gamer)

– The user has fun while engaging in the activity of learning

– One learns though experience– The learner is an actor, s/he constructs

her/is own experience and acquisition of knowledge vs passive learning

FUN

ENGAGE learn

experience

Serious Games Serious Games Pedagogical ModelPedagogical Model

Why learn with a serious game?Why learn with a serious game?• Our present economic era is

referred as

Information or Digital Age to indicate the shift from the production of physical goods towards

the production and manipulation of information

• Second economic revolution

• The pedagogical model of the information age has to address the same shift

Why learn with a serious gamesWhy learn with a serious gamesBruner (1996) • Different models of the mind have been developer which have

shaped our pedagogical one

• Imitative learners - need to acquired the ‘know-how’ (e.g. artisan)

• Pupils learning from didactic exposure, a ‘tabula rasa’ to be imprinted, a passive receptacle of knowledge (e.g.TV watcher)

• Pupils are thinker in need of constructing their own model of the world, exchanging understanding trough discourse and collaboration with educators and peers, manipulating beliefs about the world and other his/her metacognitive skills (Piaget, proximal development idea). (e.g. Group work, classroom discussions) (Social learning & reasoning in a community of practice, networked learning).

Serious Games Pedagogical ModelSerious Games Pedagogical ModelKolb learning wheel • Learning trough experience and reflection (ARTISAN)

Chancley• Situated learning in a community of practice (GROUP

LEARNING IN CONTEXT)

Is this all it happens?

Serious Games Pedagogical ModelSerious Games Pedagogical ModelBrumer (1996)• New pedagogical model• Pupils grasp the distinction between:

– Personal knowledge and ‘what is to be known’ by culture

(normal teen-ager attitude as critical thinker)

• Pupils begin to understand how evidences are used to check beliefs, they form beliefs about beliefs, or hypothesis, that become a proven theory or a body of facts

• Learners are actors constructing their own knowledge

MOTIVATION for learning => they enjoy what they do

• They learn following their own interest• They search, access and produce their own material

(e.g. Google, U-tube, blogs, Wikipedia, etc.)• (e.g. Fail school, but extremely competent in game technology, communication

technology & Second life playing .... )

Blended Learning

Serious games have various Serious games have various shapes and formsshapes and forms

Virtual Learning EnvironmentsVirtual Learning Environments

Development Platforms– Bespoken 2D graphics

– Bespoken 3D graphics

– Game engine

Virtual Learning EnvironmentsVirtual Learning EnvironmentsVisualization Hardware

– 2D screen (PDA, 15”, .., Cinema)– 3D solution

Virtual Learning EnvironmentsVirtual Learning EnvironmentsInteraction Devices

– Pointer metaphor• Mouse & Keyboards• Gamepad / game hardware

– Intuitive/ natural

20/30 cm

Difference between films, 3D graphics, and VRDifference between films, 3D graphics, and VR

CreativeMedia

OUTPUT

Active interactions (actor, creator)Virtual Reality

Passive spectatorTV, 3D graphics in films

2D

VISUALIZATIONHARWARE

or

Images &sound

3D

2D

3D

2D

INTERACTION

Pre-Rendered,

Pre-defined, Video

Rendered at run time,

(un)limited possibilities

Example of Serious GamesExample of Serious Games

2D Desktop Serious Game: 2D Desktop Serious Game: T’rrific tales

• Learning Environment for Creative Writing for infants• To study the interaction between the human and the digital and the effect on

- teacher time,- classroom context and learning 2000

Agent Based Football Training

A. BinSubaih, S. Maddock and D.M. RomanoA collaborative domain independent platform to is used to train a police officer to deal with road accident in virtual environments. Desktop and Fully immersive solution, game platform independented (e.g. Torque)

3D Game Desktop/Immersive Serious Game:3D Game Desktop/Immersive Serious Game:DICE

Domain Independent Collaborative training Environment

AIMSTo develop a flexible collaborative

training environment that can be used to train police academy students in

different situations

PROPOSED SOLUTION A training environment has been built

that is domain independent.The logic of the scenario is also

independent from the virtual environments used and can be entered

by non programmers.The simulation can run using any of the game engine available (low cost & high

quality graphic).

STUDIESCan such environment replace on-site

training?Can it be integrated within the police

academy course material?Is there a transfer of knowledge

between virtual training and real life accidents?

Sponsored by Dubai Police

2004-2007

Tabletop vs CVE Police TrainingTabletop vs CVE Police TrainingAhmed BinSubaihSteve MaddockDaniela Romano

Current Training Dubai police force currently trains its accident investigators using theory (taught courses) followed by practice (on-the-job training)Issues:• A large gap exists between the theoretical and practical training• Current on-the-job practical training is not sufficient to bridge that gap

On-Site Training Accident ScenarioCrash between two vehicles; one severely injured passenger; no injuries to the drivers; both vehicles have leaked hazardous material at the scene. Roles (4 people collaborating):• Investigator (Attend emergency & establish faults)• Two drivers (Explain to the investigator what happened, answering any questions)• Control room operator (Provide resources)

CVE• CVE = three desktop machines networked together, reproducing the accident scene.

• On each desktop the users were equipped with a headset for free-speech communication with the others.

• The users use the joystick/mouse/keyboard to navigate the CVE.

• Users were represented by avatars in the environment.

Tabletop• A 2D map of the accident scenario• Movable stickers to represent: drivers, injured

person and vehicles.

• Two actors played the role of the drivers, sat across the table from the Police Officers

• The operator moved the stickers representing the resources as requested (i.e. ambulance, the towing-trucks, additional assistance, etc.)

ResultsPerformance measured: Time & No. of tasks completed.

Trainee improved task performance and time of the second run in both environments.

Performance below half of the mark that can be achieved => current training inadequate => further training needed in practical form.

Better spatial awareness related tasks for CVE subjects.

Better overall performance of the group that performed tabletop first and then CVE.

Subjects’ comments favoured the CVE method.

North

West S

outh

East

89

90

Al-Riga Road

60 KM/H

Glass5 m

10 m

25 m28 m

Petrol

Dodge

Oil

Taxi

Department ofComputer Science

Two groups(3+3)of Dubai Police Officers solved the incident, 2x 2 design (tabletop first, than CVE and vice versa)

Server/Host

VirtualEnvironment

Actor 2

Actor 1

Trainee

VirtualEnvironment

VirtualEnvironment

CR Operator

3D Game Consol Serious Game:3D Game Consol Serious Game:Interactive Trauma Trainer

2004

DTI

BLITZ games – game engine

Surf Simulator

Drug Market SimulatorDrug Market SimulatorEPSRC, DAATBuild a training environment for the Drug Alcohol Abuse Teams to rehearse interventions

Drug Market SimulatorDrug Market Simulator

2006

2D map into 3D world – ordinance survey dataACORN classification of residential neighbourhoodHalf-life game engine / Bespoken solution

EPSRC, DAATBuild a training environment for Drug Alcohol Abuse Teams

Automatic Construction of a CityAutomatic Construction of a City A row of Terrace houses in Sheffield automatically reconstructed from GIS data (above) and the real street (below)

GPU FLAMEGPU FLAME• Agent-Based Modelling with emergent behaviour on the GPU• Enhanced version of the Agent Framework FLAME

POLLy – POliteness in Language LearningPOLLy – POliteness in Language LearningDialogue and Animation generation system for teaching British

politeness to non-British English speakers Swati Gupta, Marilyn Walker, Daniela Romano

POLLy – POliteness in Language LearningPOLLy – POliteness in Language LearningDialogue and Animation generation system for teaching British

politeness to non-British English speakers

Architecture

• videos

D.Romano@sheffield.ac.uk

THANK YOU

Daniela M. RomanoDaniela M. Romano