Playing with Your Data: Towards

Personal Informatics Driven Games

Elizabeth L. Murnane

Information ScienceCornell Universityelm236@cornell.edu

Geri Gay

Communication,Information ScienceCornell Universitygkg1@cornell.edu

Mark Matthews

Information ScienceCornell Universitymark.matthews@cornell.edu

Dan Cosley

Information ScienceCornell Universitydrc44@cornell.edu

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s).UbiComp/ISWC ‘16 Adjunct, September 12–16, 2016, Heidelberg, Germany.ACM 978-1-4503-4462-3/16/09.http://dx.doi.org/10.1145/2968219.2968322

Abstract

Personal Informatics technologies and the quantified-selfmovement focus on helping people collect personally mean-ingful information to gain self-knowledge, which can gohand in hand with the drive to change behavior or improveoneself. The field of serious games examines how gamescan be used for purposes beyond entertainment, with com-mon applications in areas such as education, training,or health care. This workshop paper overviews our re-search aimed at bridging and expanding the scopes ofthese fields through the design of personal informaticsdriven games: gameful and playful approaches to data cap-ture, self-reflection, and behavioral intervention.

Author Keywords

Personal Informatics; Quantified Self; Games; Toys; Play

ACM Classification Keywords

H.5.m [Information Interfaces and Presentation (e.g., HCI)]:Miscellaneous; K.8.0 [Personal Computing]: Games

Introduction

Games are a powerful medium, and digital games in par-ticular have a captivating ability to create deeply engag-ing experiences. Recognizing this, designers of interactivesystems — including personal informatics (PI) tools — areincreasingly using games (or game elements, i.e., gamifi-

cation) to increase user motivation, adherence, and enjoy-ment. As gameful design continues to gain traction withinHCI, we see a tremendous opportunity for digital games toconnect more richly with our real-life behaviors and goals.

Potential impacts could be considered game-to-world orworld-to-game. Serious games yield the former as they areintended to provide a player with real-world takeaways (e.g.,knowledge, skills, or fitness improvements). The latter isless commonly encountered.

By developing prototypes that integrate real-world tasks(e.g., snapping photographs, taking public transportation, ordoing volunteer activities) with in-game actions, exploratoryHCI research has demonstrated the promise of world-to-game approaches for incentivizing pursuit of personal andsocial goals [5]. Today, a handful of commercial gamesdo incorporate social media data, for instance from Face-book (e.g., Who?1) or Twitter (e.g., Twirdie2). However,most games use sparse amounts of personal informationas part of play, player-generated data generally has no im-pact on game dynamics, and most importantly, few studieshave been undertaken to understand the range of uses andimplications this approach might have.

Our research examines ways that gameful designs can helpcapture personal information as well as ways that gamescan transform such data into personally meaningful ex-periences and positive real-life impacts. In this paper, weoverview our work exploring how games and toys can makemanual self-tracking more reliable and less burdensome,what it means to literally play with personal data, and howpersonal-data-driven games can support individuals’ self-management and behavior change goals.

Figure 1: Our pressure-sensitivetangible self-report device for painthat can be embedded into serioustoys in order to improve datacapture from children

Figure 2: Our prototype game forcollecting alertness data 1http://www.whothegame.com/

2http://www.twirdie.com/

Games for Data Capture

A central aspect of personal informatics is gathering in-formation about behaviors, experiences, or various otheraspects of oneself that are of personal interest. However,burdens associated with common techniques (e.g., diaries,surveys, or ecological momentary assessments) presentchallenges, especially when incorporated into personal in-formatics systems intended for long-term, in-situ use.

In response, HCI researchers are increasingly designingmore usable modalities such as more visually engaging in-terfaces or smart physical objects. Pursuing play-based ap-proaches that promote adherence, data quality, and positiveuser experiences, one strand of our work investigates theuse of toys and games as an alternative strategy for datacapture of subjective or objective measures.

Consider the case of children, where it is difficult to pre-cisely and reliably capture perceptions — for instance ofpain intensity — through standard instruments. Investigat-ing innovative forms of self-report, we have been develop-ing a novel pressure-based tangible user interface shownin Figure 1 that is inspired by the way those experiencingpain grasp nearby objects or a loved one’s hand [1]. For thepediatric context, we are designing ways to embed this sen-sor into toy-based form factors such as a squeezable teddybear. Going forward, we aim to explore additional designsand application areas for such “serious toys”.

In the context of alertness assessment, stimulus-responsetests such as the Psychomotor Vigilance Task (PVT) arecommonly used to objectively capture personal data. How-ever, such tools are known to be cumbersome and are par-ticularly susceptible to inadherence when used outsideof laboratory settings — for instance during everyday PIpractices. Indeed, in our own research that made use ofa smartphone version of the PVT [6], participants unani-

mously expressed its highly burdensome nature and theimprobability they would use it outside our study.

Further feedback from participants and follow-up designprocesses led us to explore game-based interactions asan input mechanism for collecting personal cognitive per-formance data. As first steps, we are prototyping (see Fig-ure 2) and working to validate mobile versions of familiarWhack-A-Mole style games for alertness assessment aswell as more novel games, such as a word stem generationgame based on established psychological instruments forthe evaluation of higher order cognitive functions.

Games for Discovery and Reflection

Our second area of work focuses on creating personal-data-driven games to support self-discovery and reflectionabout one’s self, behaviors, and past. Using personal in-formation extracted from sources such as Gmail and Face-book, we are building a series of “tailor-made” games andconducting user evaluations to investigate engagement andthe individual impact of these types of playful interactionswith data. To date, we have focused on creating person-alized adaptations of familiar games so that the differencefrom the norm is more apparent.

Figure 3: Variants of PlayMail, ourludic email game that supportsawareness of personal patterns bymapping email usage attributesonto gameplay dynamics

Figure 4: Our picture-matchinggame injected with personalphotographs to support selfreflection and social reminiscence

PlayMail uses a game metaphor to abstract personal emaildata into a ludic experience. Today, many people spendsignificant time tending to email and can struggle with effec-tively managing it. Email analytics tools like Gmail Meter3

provide statistics and graphs as a way to help people im-prove email practices. Communicating aspects of personalemail data that are harder to access through traditional vi-sualizations, PlayMail aims to help people discover andmake sense of their own email patterns as well as gain in-sights and empathy about other email users through modes

3http://www.gmailmeter.com/

for two players or alternate personas (e.g., “Play as an Ad-ministrative Assistant”).

We have created several PlayMail variants (e.g., adaptingclassic games like Asteroids, Space Invaders, and HighNoon), as illustrated in Figure 3. Consistent to each de-sign is a tight mapping between characteristics of the un-derlying data and gameplay elements. For example, in theAsteroids-style version, the number of enemy ships corre-sponds to the number of emails the player receives at thetime of day being played in the game, the types of those en-emy ships correspond to types of email senders, and theplayer’s number of available bullets corresponds to that indi-vidual’s number of sent emails.

Turning to ways to facilitate reflection, reminiscence, andmaintenance of social connections, we have injected so-cial media data into a memory game similar to Concentra-tion, where a player flips over pairs of face-down cards inan attempt to match cards’ photos and gain points. In ourpersonalized version, shown in Figure 4, game material isdrawn from a player’s photos in social media networks likeFacebook or from that person’s smartphone camera roll. Amulti-player variant also enables investigation of the use ofPI games as icebreakers, storytelling tools, or vehicles forcollaboratively making sense of personal information.

Games for Behavior Change and Intervention

For many individuals, the value in exploring personal datastems from a desire to translate gleaned self-knowledgeinto self-improvement strategies. Aiming to support PImotives related to behavior change, our final line of re-search deals with developing games that support self-experimentation, actionable feedback, and guided inter-vention.

Focusing on applications for mood and stress manage-ment, we are incorporating biofeedback passively collectedthrough off-the-shelf wearables into custom games thatdeliver relaxation interventions to players. Stress Fighter,shown in Figure 5 and based on the classic game StreetFighter, is an exergame-style game we designed, where at-tributes of the opponent “boss” character correspond to theplayer’s sensed stress levels that day. Encouraging full bodymovement, Stress Fighter continues to capture and incor-porate biofeedback during gameplay as well, since physicalexertion itself can help tackle stress.

Compared to existing games incorporating biometric data,where the dominant game mechanic is instructing playersto try altering their physiological measures in order to makesomething happen in the game (e.g., instructing playersto relax in order to make a sun rise or change the color ofambient lights [7]), Stress Fighter allows us to study moresubconscious approaches to stress reduction through play.

Figure 5: Stress Fighter, ourexergame stress intervention thatmaps daily stress levels ontoopponent attributes and in-gamebiofeedback onto playerperformance

Inspired by games like EyeSpy, where searching for theapproving face in a crowd of frowns is used to help recondi-tion the mindsets of people with low self-esteem [3] or PlayAttention, where neurofeedback is used to control game el-ements and in turn improve ADHD symptoms [8], our nextstep is exploring how games can be used as part of psy-chological or cognitive therapy. In particular, we are inter-ested in the therapeutic role of video games in nonpharma-cological treatments for managing mental illnesses suchas bipolar disorder, for instance to help combat periods ofdepression or act as a safe outlet during manic episodes.

Type of Data Examples

Health Data Food intake, nutrition, exercise,sleep, health conditions

Physiological, Pulse, blood pressure, arousal,Behavioral, skin temperature, brain signals,and Soft eye gaze, motion detection, facialBiometrics recognition, speech, height, weightTechnology Browsing history, app use logs,Usage energy consumptionContext Geo-location, movement, time,

weather, lightInterpersonal Calls, texts, emails,Activity social media

Table 1: A sampling of personalinformation that could be capturedor examined via games or toys

Significance to the PI/QS Workshop

Whether designed for entertainment or more serious pur-poses, games have an extraordinary potential to shapean individual’s mindset, behavior, and overall well-being.

By exploring creative ways to connect games with per-sonal data, our research aims to contribute design patternsthat aid the creation of personally meaningful, tailor-madegames that can guide personal discovery and development.

Having only scratched the surface of opportunities for thesepersonal informatics games in our work to-date, we see avast design space to continue traversing going forward. Forinstance, there is a wide-range of personal data that couldbe captured through or plugged into games. Table 1 pro-vides some promising examples of “playable” data [4], andit would be worthwhile to more deeply contemplate whichamong these and other data would be more or less con-ducive, appropriate, and helpful to incorporate into gamefulexperiences.

Moving beyond gamification, there is also a significant op-portunity to study what gameplay dynamics are particu-larly compelling when playing with one’s own data, howpersonal goals might shape game mechanic choices, andwhat methods are most effective to evaluate the impact ofPI games. Similarly, it is necessary to more fully examinethe perceived value and receptivity of such systems that en-able individuals to engage with personal information in suchnovel (and hence under-studied) ways.

In addition, it is important to consider potentially detrimentalaspects or risks of personal informatics games, such asissues related to privacy, misinterpretation (mappings ofdata to game experience are non-perfect), and where caremust be taken to not cross a line into “exploitationware” [2].

Finally, there is much room to investigate the application ar-eas where novel solutions to PI challenges are most press-ing as well as domains where personal informatics gameswould be particularly opportune, fitting, and well-received.

Acknowledgements

Elizabeth Murnane is supported by the National ScienceFoundation Graduate Research Fellowship under GrantDGE 1144153.

REFERENCES

1. Min Aung, Faisal Alquaddoomi, Cheng-Kang Hsieh,Mashfiqui Rabbi, Longqi Yang, JP Pollak, DeborahEstrin, and Tanzeem Choudhury. 2014. LeveragingMulti-Modal Sensing for Mobile Health: A Case Reviewin Chronic Pain. (2014).

2. Ian Bogost. 2011. Persuasive Games:Exploitationware. Gamasutra. (2011).

3. Stéphane D Dandeneau and Mark W Baldwin. 2004.The inhibition of socially rejecting information amongpeople with high versus low self-esteem: The role ofattentional bias and the effects of bias reductiontraining. Journal of Social and Clinical Psychology 23,4 (2004), 584–603.

4. Nicholas Diakopoulos, Funda Kivran-Swaine, and MorNaaman. 2011. Playable data: characterizing thedesign space of game-y infographics. In Proceedings ofthe SIGCHI Conference on Human Factors inComputing Systems. ACM, 1717–1726.

5. Jason Linder and Wendy Ju. 2012. Playable character:extending digital games into the real world. InProceedings of the SIGCHI Conference on HumanFactors in Computing Systems. ACM, 2069–2078.

6. Elizabeth L Murnane, Saeed Abdullah, Mark Matthews,Matthew Kay, Julie A Kientz, Tanzeem Choudhury, GeriGay, and Dan Cosley. 2016. Mobile Manifestations ofAlertness: Connecting Biological Rhythms withPatterns of Smartphone App Use. In Proceedings ofthe 18th International Conference on Human-ComputerInteraction with Mobile Devices and Services. ACM. (Toappear).

7. Jaime Snyder, Mark Matthews, Jacqueline Chien,Pamara F Chang, Emily Sun, Saeed Abdullah, andGeri Gay. 2015. Moodlight: Exploring personal andsocial implications of ambient display of biosensordata. In Proceedings of the 18th ACM Conference onComputer Supported Cooperative Work & SocialComputing. ACM, 143–153.

8. Naomi J Steiner, Elizabeth C Frenette, Kirsten M Rene,Robert T Brennan, and Ellen C Perrin. 2014. In-schoolneurofeedback training for ADHD: sustainedimprovements from a randomized control trial.Pediatrics 133, 3 (2014), 483–492.