View
1
Download
0
Category
Preview:
Citation preview
1
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Title of PresentationCopyright © 2012 The Nielsen Company. Confidential and proprietary.
Smartphone Applications:The Next (and Most Important?) Evolution in Data Collection
Presenter:
Michael W. LinkSVP/Chief Methodologist
Nielsen Institute for Measurement InnovationThe Nielsen Company
AAPOR 2012 ConferenceOrlando, FL
Michael W. Link, Jennie Lai, & Lorelle Vanno
2
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Entering New Era of Data Collection
Manual Survey Collection Era (1930s-1970s)
Computer-Assisted Survey Collection Era
(mid 1970s-present)
Post-Survey Data Capture Era(2010 - ???)
3
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
New Worry: Respondent Expectations• Being involved in a scientific survey is a relatively rare occurrence
– Little, if any, expectations by respondents
• Respondent experience with new technologies is much different– Respondents are more savvy
• Developing expectations from these experiences– Ease of use/intuitive– Speed– Usefulness/utility or fun/entertainment or both– Ability to share experiences w others– Auto-features: ex. Location awareness
Need to recognize this societal shift and develop our new
measurement approaches accordingly
4
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Critical Nature of User Engagement• “User engagement is the emotional,
cognitive, and behavioral connection that exists, at any point in time and possibly over time, between a user and a resource [i.e., technology]” (Attfield, et al, 2011)
• Encompasses non-utilitarian aspects –emphasis on understanding and designing for the subjective aspects of technology encounters
• It is essential that technology providers do not just design systems but that they design engaging experiences.
Source: Attfield et al, 2011http://www.dcs.gla.ac.uk/~mounia/Papers/engagement.pdf
5
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Developing Measurement Systems for the New Landscape• How do we:
– Break old rules & develop new ones?– Leverage user engagement to improve participation?– Provide scientifically valid and reliable data to end-users/clients?
Form & Function
• Developing data collection tools which are ease to use, logical in flow, & visually appealing
Reciprocity
• Recognize data collection as two-way interaction and provide VALUE to respondent
Gamification
• Utilizing the psychological & sociological components of “game theory” and “game mechanics”
Social Sharing
• Facilitating greater respondent self-expression and potential content sharing with others
Four Potential Areas
6
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Gamification• Concept: Identify sociological & psychological aspects of popular
game & entertainment apps, then apply them to measurement to improve user engagement and participation
– Does NOT mean literally turning measurement into a game
• Game Mechanics: various actions, behaviors, & mechanisms used to create a compelling user experience
• Game Dynamics: Motivations in response to experience that drives continued participation
Game MechanicsPointsLevelsChallengesVirtual goodsLeader boardsGifts & charity
Game DynamicsRewardsStatusAchievementSelf-expressionCompetitionAltruism
Interplay
7
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Social Sharing• Ability to share information with others is ubiquitous (& expected) in
popular apps, measurement systems need to find ways to provide this & quantify measurement bias
• Violates long-held “rule” of measurement – measurement should be singular, not influenced by others
• Expression / respondent voice– To researcher – mechanisms to provide feedback, emotive measures
(beyond “Other: Specify”)– To others – share entries, views, behaviors, attitudes
– Other participants (family, community) or broader non-study groups
• Examples: Likes/dislikes, ratings, opportunities for commentaryPost comments to app-specific message feedPost information to SNSComment on posts of others
8
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Example: Converting Paper Diary to Mobile App
9
Copyright © 2012 The Nielsen Company. Confidential and proprietary.AAPOR 2012
Friends & Family Test
Test Group Recruited RegisteredApp
CompletedFollow-Up
Full Start 100 90 67
Incremental 150 132 97
F&F Study Objective: Get application into as many hands as possible and develop some initial learnings to guide future research
• Sample: 250 Nielsen employees (non-probability, non-projectable)• Platform: iPhone only• Field Period: Jan 17 – Feb 27, 2012• $25 Gift Card Incentive
Research Program Goal: Develop a mobile alternative to Nielsen’s traditional paper TV Audience Measurement Diary
10
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Respondent Engagement: Game Mechanics
• Goal Setting• InstructionBadges
• High Value Activities
• Redemption / Reputation
Points
• Top Performance• Social IncentiveLevels
11
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
BadgesHead Start Badge:You received the head start badge for completing your first full TV viewing log-in
Recall Badge:You received the recall badge for completing your first full past TV viewing log-in
Rebel Badge:You received the rebel badge for completing your first non-traditional (DVR /VCR, OnDemand or Online) TV viewing log-in.
Silver Cornerstone Badge:We could not get the data without you! You received the silver cornerstone badge for completion of 5 days of TV viewing.
Gold Cornerstone Badge:We could not get the data without you! You received the silver cornerstone badge for completion of 10 days of TV viewing.
12
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Points & Statuses
Level
Total points needed to attain new level
0 TV Viewer <300
1 Grip 300
2 Best Boy 600
3 Gaffer 1,100
4 Fixer 1,600
5 Story Assistant 2,300
6 Editor 3,000
7 Assistant Director 3,900
8 Director 4,800
9 Show Runner 5,800
10Producer >7,000
Action for achieving points# points per action
Badge Received 300Survey Taken 200Promotion/Level 100Answered Push 25At least 1 Viewing Login that day 50
SSSSss
13
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Respondent Engagement: Social Mechanics
• Status Updates• Influence
Social Sharing
• App Community• Facebook
CommunitySocial
Network
• TV Viewing• Status updates
“Likes” / Comments
14
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Experimental Conditions Sample Size
Game Mechanics
Social Mechanics
Group #1Full App Week 1-6 100 Enabled Enabled
Group #2Incremental
Week 1-2
150
Disabled Disabled
Week 3-4 Enabled Disabled
Week 5-6 Enabled Enabled
Respondent Engagement Test GroupsTiming of Introduction of Game & Social Mechanics
15
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Average Hours of TV Viewing Recorded per Week
Respondent Characteristics
Nielsen DiaryFeb ‘12
Group #1: Full App Group #2: Incremental App
(n)Weekly
Avg.Week 1
Avg.Week 6
Avg. (n)Weekly
Avg.Week 1
Avg.Week 6
Avg.
Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8
18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1
35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8
50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6
White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1
Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1
Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6
Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7
Note: Includes all respondent who (1) downloaded the app and (2) recorded at least 1 hour of viewing during the 6 week field period.
16
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Average Hours of TV Viewing Recorded per Week
Respondent Characteristics
Nielsen DiaryFeb ‘12
Group #1: Full App Group #2: Incremental App
(n)Weekly
Avg.Week 1
Avg.Week 6
Avg. (n)Weekly
Avg.Week 1
Avg.Week 6
Avg.
Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8
18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1
35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8
50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6
White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1
Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1
Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6
Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7
Note: Includes all respondent who (1) downloaded the app and (2) recorded at least 1 hour of viewing during the 6 week field period.
17
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Average Hours of TV Viewing Recorded per Week
Respondent Characteristics
Nielsen DiaryFeb ‘12
Group #1: Full App Group #2: Incremental App
(n)Weekly
Avg.Week 1
Avg.Week 6
Avg. (n)Weekly
Avg.Week 1
Avg.Week 6
Avg.
Total 26.9 (86) 6.9 10.4 4.7 (117) 7.9 10.7 6.8
18 – 34 yrs 17.5 (35) 6.8 9.5 5.1 (37) 8.2 10.4 8.1
35 – 49 yrs 25.9 (30) 7.2 11.0 4.0 (47) 6.8 10.1 5.8
50+ yrs 39.1 (19) 6.0 11.2 5.6 (31) 9.3 12.3 6.6
White 26.0 (66) 7.2 10.8 5.1 (88) 7.5 10.6 6.1
Black 36.4 (4) 5.9 6.3 3.3 (5) 16.9 20.8 14.1
Asian 20.5 (15) 6.0 10.0 3.8 (18) 7.7 9.1 8.6
Hispanic 25.8 (9) 5.9 7.8 3.8 (12) 7.2 9.1 6.7
Note: Includes all respondent who (1) downloaded the app and (2) recorded at least 1 hour of viewing during the 6 week field period.
18
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Impact of Badges and Point & Levels Avg. Hours per Week per Person (Total Sample)
TV Viewing Hours per Week per Person
Badges, Points, LevelIncremental App
Source: Nielsen, Whatcha’ Watchin’ F&F Test Data, 2012
19
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Impact of Badges and Point & Levels Avg. Number Days w 1+ Entries per Person (Total Sample)
Avg Number Days with 1+ Entry per Week per Person
Badges, Points, LevelIncremental App
Source: Nielsen, Whatcha’ Watchin’ F&F Test Data, 2012
20
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Reasons for Use and Non-Use of Social Feed
0
10
20
30
40
50
60
70
80
90
100
Features UsedPush to FacebookCommentsLikesRead Entries
0
10
20
30
40
50
60
70
80
90
100
Why NOT Use SNSHard to TypeTime ConsumingNot Connected w CommunityNot Want to Share Viewing
N = 50 N = 70
21
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Summary of Learnings To Date• Mobile platform has a ways to go before displacing current
methodology!
• Gamification is more important cross-time Little impact during first 2 weeksWhen introduced incrementally, impact is important but brief
• Social Mechanics had little impactUnclear if it is the way in which it was used or whether social
mechanics are simply not a useful engagement techniqueOrder effect?
• Key questions for all techniques: Tradeoff: enhanced participation vs potential for biasNot driving the attitudes or behaviors you’re trying to measure!
Study represents a first empirical “stake in the ground” – long road ahead to fully understand the best techniques for enhancing user engagement
22
Copyright © 2012 The Nielsen Company. Confidential and proprietary.
Thank you!
Michael LinkMichael.Link@Nielsen.com
Recommended