Posture as an Indicator of Engagement in Handheld and Laptop Game-Play

Jennifer R. CaseThe Graduate Center, CUNY

Co-authors: Winslow Burleson, Elizabeth Hayward, Charles Hendee, Bruce Homer, Ken Perlin, Jan

Plass, & Jay Verkuilen

Games for Learning Institute• Our mission is to pursue the art and science of

designing effective and fun games for learning, by bringing together 14 game designers, computer and education scientists, and psychologists from NYU, Columbia, CUNY, Dartmouth, NYU-Poly, Parsons, Chile’s PUC, RIT, and Teacher’s College.  We use quantitative and qualitative methods to study game design elements and learning patterns, and apply this knowledge to prototype “mini games”, which we evaluate with middle school age learners. Our initial focus is on the teaching of science, technology, engineering, and math.

Video Games• The video game industry has grown at a

rate (>10%) larger than the U.S. economy (<2%) from 2005 to 2009 (ESA, 2010)

• 41% of all households with TVs have a video game console (Nielson, 2007)

• In 2009, Nearly as many video games units were sold (273.5 million) as there are people in the United States (308.7 million) (ESA, 2010; US Census, 2010)

• In 2007, nine games were sold every second of every day on average (ESA, 2007)

Games & Education• Educational games are a type of “Serious

Games”• Serious games often do not overlap with

entertainment games• 64% of parents think games are positive in

their children’s lives (ESA, 2010)

• Researchers identified 219 math educational video games, yet these types of games are not terribly popular in usage as pure entertainment games (G4LI @ NYU-Poly, 2010)

Engagement• Humans express emotion through

non-verbal cues (Rinskind, 1984; Duclos, 1989; Coulson, 2004; Mehrabian, 1969; Wilson, 2004)

• But what does engagement look like?

Posture• Posture has been used as a tool to

measure affective states such as engagement (Mota & Picard, 2003; D’Mello, Chipman, & Graesser, 2007, Arroyo, Cooper, Burleson, Woolf, Muldner, & Christopherson, 2009)

• The idiom “edge my seat” has not been confirmed in prior research

• We are looking to evaluate posture as an indicator of engagement during game play

Our Posture Pad• Sensor pads based on Interpolating Force Sensitive Resistance

(IFSR) have the desirable property of being able to detect even small variations in both position and pressure with great fidelity, because they capture true anti-aliased images of pressure at high frame rates. IFSR sensors are also inexpensive, flexible, low power, technically robust and insensitive to electrical interference.

• As such, it is an ideal solution for detecting subtle motion during game play.The pad we used as an 8.5" x 11" sensor, with a grid of 64 x 48 pressure detection (sensel areas at 12 bits of data).

• 2x2 experimental design:Task Types

Engaging Unengaging

Computer

Handheld

Research Design

Horizontal Movement

Vertical Movement

Future Analyses• Upcoming analyses on the data will utilize

more advanced statistical designs• Possible Methods:– General Additive Models– Functional Data Analysis

• Pool data for individuals after determining how to remove the unique individual effect

• Add other measures of engagement:– Galvanic Skin Response– Mouse Pressure

Questions?• Jcase@gc.cuny.edu

Thank You