57
CS/INFO 4154: Analytics-driven Game Design Playtesting Class 14:

Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

CS/INFO 4154:Analytics-driven Game Design

Playtesting

Class 14:

Page 2: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Mon Wed Fri

9/29

Alpha Testing 2

9/25

Playtesting

9/27

Alpha Testing 1

10/2

Alpha Postmortems

10/4

Logging

10/6

Data Visualization

Page 3: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Alpha Prototype

Wednesday!

Three playable levels: easy, medium, hard

List of input methods/keys

Page 4: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Alpha Testing

Wednesday and Friday: testing / feedback

Monday: postmortem activity

Page 5: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Alpha Submission

Submit through CMS before class (10:10am)

Whatever we need to run the game

Can ZIP it up or just submit the compiled file

Page 6: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Playtesting: is the game good?

Page 7: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Today

Traditional techniques

Newer techniques

Group activity: playtesting planning

Page 8: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Resources

“Successful instrumentation: tracking attitudes and behaviors to improve games”

Ramon Romero, GDC 2008

“Valve’s approach to playtesting: the application of empiricism”

Mike Ambinder, GDC 2009

“Biofeedback in Gameplay: How Valve Measures Physiology to Enhance Gaming Experience”

Mike Ambinder, GDC 2011

Page 9: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Today

Traditional techniques

Newer techniques

Group activity: playtesting planning

Page 10: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Traditional playtesting methods

Q&A

Page 11: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Questions and Answers

What happened when you

went through the portal?

I can freeze water now

Page 12: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Pros and Cons

Targeted information

Players may not be able to articulate feelings

Biased: everybody lies

Page 13: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

The impact of bias

Dell et al. CHI 2012

Page 14: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Interviewer’s video Competitor video

Dell et al. CHI 2012

Page 15: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Dell et al. CHI 2012

Page 16: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Interviewer’s video Competitor video

Dell et al. CHI 2012

Page 17: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Without claiming ownership

Dell et al. CHI 2012

Page 18: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Traditional playtesting methods

Q&A

Direct Observation

Page 19: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Direct Observation

Page 20: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback
Page 21: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 22: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 23: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 24: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 25: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 26: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Key: reading emotions

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 27: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Comparisons

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 28: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Comparisons

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 29: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Comparisons

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 30: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Arousal vs. Valence

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 31: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Pros and Cons

Can see what players do

Less biased

Hard to interpret behavior

Player might not do what you want

Page 32: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Traditional playtesting methods

Q&A

Direct Observation

Think-alouds

Page 33: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Think-alouds

I don’t know

what to do

I keep catching

on fire and dying

Why are you

making me do this

Page 34: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Pros and Cons

Know what player is thinking

Hard to talk and play at the same time

Player may ramble

Page 35: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Traditional playtesting methods

Q&A

Direct Observation

Think-alouds

Page 36: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Today

Traditional techniques

Newer techniques

Group activity: playtesting planning

Page 37: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Newer playtesting methods

Surveys

Page 38: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Halo 3 Balance Interface

Are you having fun? Y/N

Page 39: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Halo 3 Balance Interface

Are you having fun? Y/N

Page 40: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Halo 3 Balance Interface

Are you having fun? Y/N

Page 41: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Halo 3 Balance Interface

Pistol:

Sniper Rifle:

3

4

Are you having fun? Y/N

Page 42: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Survey

Did you like the game? 1 2 3 4 5

(1 = not much, 5 = a lot)

Page 43: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Survey

Deciding where to go 1 2 3 4 5

Jumping on platforms 1 2 3 4 5

Defeating enemies 1 2 3 4 5

Solving puzzles 1 2 3 4 5

Using the controls 1 2 3 4 5

How stressful were each of the following?

(1 = not much, 5 = a lot)

Page 44: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

More specific survey

Freezing 1 2 3 4 5

Heating 1 2 3 4 5

Flash Freezing 1 2 3 4 5

Flash Heating 1 2 3 4 5

How useful was each ability?

(1 = not much, 5 = a lot)

Page 45: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Pros and Cons

Less biased

More precision

Loss of nuance

Time-consuming

Page 46: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Newer playtesting methods

Surveys

Physiological measurements

Page 47: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Physiological measurements

Heart rate Skin conductance

EEG Eye tracking

Page 48: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Left 4 Dead 2

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 49: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Skin Conductance: Left 4 Dead 2

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 50: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Skin Conductance: Left 4 Dead 2

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 51: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Skin Conductance: Left 4 Dead 2

M. Ambinder, “Biofeedback in Gameplay: How Valve Measures Physiology to

Enhance Gaming Experience”, GDC 2011

Page 52: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Heart rate

Page 53: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Heart rate

Page 54: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Direct, quantitative measures of engagement

Much less biased

Expensive

Nonscalable

Often invasive

Page 55: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Today

Traditional techniques

Newer techniques

Group activity: playtesting planning

Page 56: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Monday Postmortem Activity

Goal: summarize the results of your playtest

Discuss two design challenges and what you learned from the playtest

Report results from your survey

No slides

Page 57: Playtesting - Cornell University · Analytics-driven Game Design Playtesting Class 14: Mon Wed Fri 9/29 Alpha Testing 2 9/25 Playtesting 9/27 Alpha Testing 1 10/2 ... “Biofeedback

Group Activity: Playtesting plan

Plan for ~8 players

What will you do with each of them?

Revisit three (or more) playtesting questions

How, exactly, will you answer each of them?

Come up with a survey

~5 questions

Likert scale