Becoming a Gamer: Cognitive Effects of Real-Time Strategy Gaming Brian Glass The University of Texas at Austin, 2011

Becoming a Gamer: Cognitive Effects of Real-Time Strategy

Gaming

Brian Glass

The University of Texas at Austin, 2011

Overview

• Real-time strategy (RTS) gaming

• Why study gaming?

• Blitzy results from 40-hour training with novices

• Novel modeling technique

Real-Time Strategy Games

Games

Video Games

Action Adventure Simulation Strategy

Most ExistingResearch

e.g., First-personshooter

Real-Time Strategy Games

Games

Video Games

Action Adventure Simulation Strategy

Real-TimeStrategy

• God-like view / command• Rapid, fluid action• Small window into larger world (literal spotlight)• Resource management• War themes, many “fronts”

Why video games?

1. Popular

2. Lead to generalized learning

3. We can control them

Why video games?

Popular

• 72% of Americans reported playing video games in 2008

• Females are fastest growing demographic

• In 2010, 40% of gamers were female

• Of all gamers, 33% were adult women > 20% who were males under 17

• Striatal dopamine during video game was on par with methamphetamine injection (Koepp, et al., 1998)

• DSM-V committee found insufficient evidence for video game addiction (as of 2007)

Entertainment Software Association, 2010 Essential Facts Guide

Why video games?

Lead to generalized learning

• Transfer to new tasks and new contexts

• Prior work demonstrates that long-term training in attention-based tasks can lead to performance enhancement

• Most training procedures tap specific skill specific system

Green & Bavelier (2008)

Existing Research: Action Games

Players vs. Non-Players

• Self-report as avid action gamers (5+ hours per week)• Tested on various speeded-choice RT tasks

Dye, Green, & Bavelier (2009)


Training Non-Players

• Pretest – 50 hours of play over 9 weeks – Posttest



Training Non-Players

• Multiple-object tracking task

• Useful field of view task

• Attentional blink (10 Hz)

• Crowding tasks (identification among tightly packed distractors)

• Mental rotation

• Piloting training (flight simulator)

Feng, Spence & Pratt (2007)


Myers, et al. (2000)

StarCraft: “Gold Standard” RTS

Video

http://www.youtube.com/watch?feature=player_detailpage&v=BYcKz-WJLmM#t=1903s

Attention and StarCraft

Does RTS gameplay have more of an attentional component?

Strains on attention:

Multi-agent system

Resource management

Multiple “fronts”


• Alerting• Sensitivity to incoming stimuli

• Orienting• Selection of information from sensory input

• Executive• Monitoring and resolving conflicts

StarCraft

Sensitivity to enemy units which may appear

on the screen

Selecting which game elements to attend to

Determining priority of which enemy units to

attack

(Posner & Rothbart, 2007)

Training Novices

3. We can control them

1. Control Game (The Sims)

2. Low-Attention Starcraft

3. High-Attention Starcraft

Used in: Dye, Green, & Bavelier, 2009

Training Novices

Low-Attention(One Base)

High-Attention(Two Bases)

Training Novices

Intake / Screening Pretest 20 Hours of

GamingMidtest 20 Hours of

GamingPosttest

1. Control (The Sims)2. Low-Attention StarCraft3. High-Attention StarCraft

Training Novices

Battery of cognitive / perceptual tasks – Expect Change?

Stroop – Directed attention

Task Switching

Multitask Switching

Attentional Network Test (ANT)

Visual Search

Filtering

Digit Span – Working Memory

Operating Span (Ospan)

Balloon Analogue Risk Taking Task

Multimedia Multitasking Index

Yes

Maybe

No

Pretest only, possible covariate

Training Novices

Fan, McCandliss, Fossella, Flombaum, Posner (2005)



ANT, Time3-Time1

0.00

0.05

0.10

0.15

0.20

0.25

0.30

SC-Low SC-High SC Sims

Acc

urac

y

ANT, Time3-Time1

-200.0

-150.0

-100.0

-50.0

0.0

50.0

100.0

SC-Low SC-High SC SimsR

T

All groups faster, but StarCraft-High group more accurate

Training Novices

Multitask Switching

Switching between locations, remembering states

0 0

Odd?4

7

Odd?2

3

Odd? 6

1

8

Training Novices

Multitask Switching

Switching between locations, remembering states

Multitask Switching, Time3-Time1

-0.4-0.3-0.2-0.10.00.10.20.30.4


AC

C -

Sw

itch

Tri

als


-1200.0-1000.0

-800.0-600.0-400.0-200.0

0.0200.0400.0

SC-Low

SC-High

SC Sims

RT

- S

witc

h T

ria

ls *

Training Novices

Task Switching

Ophir, Nass, Wagner (2009)

43

51

Even / Odd?

pu

jo

Vowel / Cons.

E/O V/CE/O E/O V/C E/O E/O V/C V/C V/C V/C E/O V/C

SwitchSwitch Switch Switch Switch

Training Novices

Task Switching


E/O V/CE/O E/O V/C

Switch

Task Switching, Time3-Time1

-0.06

-0.04

-0.02

0.00

0.02

0.04

0.06

0.08

0.10

SC-Low SC-High

SC Sims

Acc

ura

cy

Switches Non-Switches


-500-450-400-350-300-250-200-150-100

-500

SC-Low SC-High

SC Sims

RT


Training Novices

Filtering


Information Filtering, Time3-Time1

-0.150

-0.100

-0.050

0.000

0.050

0.100

0.150

SC-Low

SC-High

SC Sims

Acc

ura

cy

Information Filtering, Time3-Time1

-120

-100

-80

-60

-40

-20

0

20

40

SC-Low SC-High

SC Sims

Re

act

ion

Tim

e

Training Novices

Balloon Analog Risk Taking (BART)

• Sensation seeking• Impulsivity• Constraint deficiencies• Self-reported:

• Addictive Risk Bx• Health Risk Bx• Safety Risk Bx

Lejuez, et al. 2002

Training Novices

Balloon Analog Risk Taking (BART)

Lejuez, et al. 2002

BART, Time2-Time1

-0.04-0.020.000.020.040.060.080.100.12

SC-Low SC-High

SC Sims

Acc

ep

t Le

ng

th

Ris

kier

Training Novices

Battery of cognitive / perceptual tasks

Sims SC-Lo SC-Hi

Stroop – Directed attention + +

Task Switching* -A / -RT +A / -RT +A / -RT

Multitask Switching* -RT

Attentional Network Test (ANT)* -RT -RT +A / -RT

Visual Search – Target/Distractors H<E

-RT

H<E

-RT

H~E

-RT

Filtering -RT -A / -RT +A / -RT

Digit Span – Working Memory* + +

Operating Span (Ospan)* - +

Balloon Analogue Risk Taking* +Risk ++Risk Risk

2

3

*NHST p<0.05

Results Summary

StarCraft Measures

• Games won

Duration

Final difficulty levels

• Feature Modeling

70+ features are being captured during game play

Continuing Analyses

Feature Modeling with RADAR

Feature Recording for Each UnitID unit ID

unittype type of unit

isselected is selected by the player

isabuilding is a building

isidle idle = 1, not idle = 0, being built = 2, newly built = 3

isloaded loaded into a transporter

disttocc distance to nearest command center

hasgas scv carrying gas

hasminerals scv carrying minerals

isrepairinganother is repairing another unit

tickssincelastunderattack milliseconds since last under attack

prophealth proportion health remaining

killcount other units killed

velocity movement velocity

healedbymedic being healed by a medic

blindedbymedic being blinded by a medic

targetDist distance to target

islifted is lifted (building)

cntfriendly friendly units in view

cntenemy enemy units in view

distfriendly distance to nearest friendly unit

distenemy distance to nearest enemy unit

cntfriendlybldg # friendly buildings in view

cntenemybldg # enemy buildings in view

distfriendlybldg distance to closest friendly building

distenemybldg distance to closest enemy building

cntfriendlyunderattack # friendly units under attack in view

cntenemyunderattack # enemy units under attack in view

distfriendlyunderattack distance to closest friendly unit under attack

distenemyunderattack distance to closest enemy unit under attack

cntfriendlybldgunderattack # friendly buildings under attack in view

cntenemybldgunderattack # enemy buildings under attack

distfriendlybldgunderattack distance to clsoest friendly building under attack

distenemybldgunderattack distance to closest enemy building under attack

cntgas # of geysers in view

cntmineral # of mineral patches in view

distgas distance to closest gas

distmineral distance to closest minerals

propvisible proportion of possible tiles in view which are unobstructed and viewable

propbuildable proportion of possible tiles in view on which buildings can be built

ispatrolling is patrolling

istraining is being trained

remaintraintime remaining time to be trained

isresearching is a building which is researching

remainresearchtime remaining time to research

screenMoved_int has the screen moved since last recording

mmDragged_int has the minimap been dragged since last recording

sX screen x position on map

sY screen y position on map

x x position

y y position

critterinview critter in view

angle angle of rotation

xdivy x position divided by y position

ydivx y position divided by x position

mousedist distance from the mouse cursor

distscreentop distance from unit to top of screen

distscreenleft distance from unit to left of screen

distscreencorner distance from unit to screen corner

cntfriendlyunderattackodd # friendly units under attack is odd

cntenemyunderattackodd # enemy units under attack is odd

cntfriendlybldgunderattackodd # friendly buildings under attack is odd

cntenemybldgunderattackodd # enemy buildings under attack is odd

xodd x position is odd

yodd y position is odd

angleodd angle of rotation is odd

xgreatery x position is greater than y position

cntfriendlyunderattackmult3 count of friendly units under attack is multiple of 3

cntenemyunderattackmult3 count of enemy units under attack is multiple of 3

cntfriendlybldgunderattackmult3 # friendly buildings under attack is multiple of 3

cntenemybldgunderattackmult3 # enemy buildings under attack is multiple of 3

xmult3 x position is a multiple of 3

ymult3 y position is a multiple of 3

anglemult3 angle of rotation is a multiple of 3

random1 random number


Feature Recording for Each Unitunittype type of unitisselected is selected by the playerisabuilding is a buildingisidle idle = 1, not idle = 0, being built = 2, newly built = 3isloaded loaded into a transporterdisttocc distance to nearest command centerhasgas scv carrying gashasminerals scv carrying mineralsisrepairinganother is repairing another unittickssincelastunderattack milliseconds since last under attackprophealth proportion health remainingvelocity movement velocityhealedbymedic being healed by a medicdistenemy distance to nearest enemy unitcntfriendlybldg # friendly buildings in viewcntenemybldg # enemy buildings in viewdistfriendlybldg distance to closest friendly buildingcntenemybldgunderattack # enemy buildings under attackdistfriendlybldgunderattack distance to clsoest friendly building under attackcntgas # of geysers in viewcntmineral # of mineral patches in viewpropvisible proportion of possible tiles in view which are unobstructed and viewableyodd y position is oddxgreatery x position is greater than y positioncntfriendlyunderattackmult3 count of friendly units under attack is multiple of 3cntenemyunderattackmult3 count of enemy units under attack is multiple of 3ymult3 y position is a multiple of 3anglemult3 angle of rotation is a multiple of 3


Response Adaptive Display Anticipates Request (RADAR)


• StarCraft – Correctly predicts how likely a unit is to be selected – 75% of the time.

Compare Novice vs. Expert

Cross-validate over games to measure consistency of game play

Summary

• RTS gaming represents unique opportunity for long-term cognitive training

• Train 36 novices over three conditions• High-Attention RTS• Low-Attention RTS• Control (The Sims)

• StarCraft vs. Control (Sims)• When accuracy is high, RT improvement• When accuracy is moderate, accuracy + RT improvement• However, Control faster at location switching task

•StarCraft-High vs. StarCraft-Low• StarCraft-Low group is riskier on time 3• StarCraft-High group > SC-Low at information filtering

• Game factors such as success, and some offerings from RADAR to be taken into account

• 2-months follow up

Thank You!

Todd MaddoxBrad Love

Marc TomlinsonDevon GreerRusty Gomez


• Alerting• Sensitivity to incoming stimuli• Fronto-Parietal• Norepinephrine system, locus coeruleus

• Orienting• Selection of information from sensory input• Cholinergic system, basal forebrain• Reduce neural activity and RT for invalid cues

• Executive• Monitoring and resolving conflicts• Usual suspects: ACC and LatPFC• Targets of mesocortical dopamine system

Training Novices

Operating Span (OSpan)

Ospan, Time3-Time1

-20

-15

-10

-5

0

5

10


O-S

pa

n S

core


-0.06

-0.04

-0.02

0.00

0.02

0.04

0.06

0.08

0.10

SC-Low SC-High

SC Sims

Acc

ura

cy



-300-250-200-150-100

-500

50100150200

SC-Low SC-High

SC Sims

RT



-300

-250

-200

-150

-100

-50

0

50

100

150

200

SC-Low SC-High

SC Sims

RT

- S

witc

h C

ost


-0.060

-0.040

-0.020

0.000

0.020

0.040

0.060

0.080

0.100

SC-Low

SC-High

SC Sims

AC

C -

Sw

itch

Co

st

Task Switching


-600.0

-400.0

-200.0

0.0

200.0

400.0

600.0

SC-Low

SC-High

SC Sims

RT

- C

ost


-0.4-0.3-0.2-0.10.00.10.20.30.4


AC

C -

Co

st

Multitask Switching

Documents

Becoming a Gamer: Cognitive Effects of Real-Time Strategy Gaming Brian Glass The University of Texas at Austin, 2011