Mobile Surveys | The 5 th Methodology: New Paradigm or Strategic Enhancement ?

M a r k e t R e s e a r c h i n t h e M o b i l e W o r l d 1 9 - 2 0 J u l y 2 0 1 1 : A t l a n t a

M i c h a e l F r a n c e s c o A l i o t o V i c e P r e s i d e n t , M a r k e t i n g S c i e n c e s

Surveying consumers in

their natural environment

What Role Will Mobile Play in the Future ?

Critical Study Objectives:

• Understand/confirm current mobile survey usability—

integrative hybrid or next survey platform?

• Confirm consumers’ usage of mobile devices in their

daily lives—communication and environment

• Confirm if surveys can be completed within an integrated

platform with similar results

• Compare and contrast smartphone and online surveys—

scale usage, results, incidence rates, survey length, etc.

• Assess potential to analyze data utilizing more advanced

analytical techniques (e.g., MaxDiff) on mobile devices

Path to understanding

research design

A Four-Phase Process

Approaches Toward Analysis:

Usage & Lifestyle

Flash Polls &

Dialogues

n=496

Desk Research

Review literature, conventional thoughts, and

secondary data; establish baseline

Survey Design

Comparison

Smartphone survey vs. online version

n=1,006

(495 smartphone/

511 online)

Survey Results

Comparison

Confirm data comparability and

scope across platforms

Advanced Statistical Analysis

Confirm

comparability of more complex quantitative techniques

PHASE ONE: PHASE TWO: PHASE THREE: PHASE FOUR:

In partnership with

Macro Trends • Two schools of thought: strategic enhancement—

online supplement vs. platform evolution—

5th Methodology

• Tendency toward hybrid platforms and data

integration, but results and strategy not conclusive

• Suggests major limitations for mobile survey

component of hybrid platform—device/technology,

research design elements

• Academic community mostly silent on the issue

• 5th Methodology as mainstream future platform—

will be driven by behavior and mobile device

technological improvements

Current understanding

and benchmarking

Desk Research: Key Findings

Micro Trends • Data comparability between online and mobile

survey platforms debatable— supplement vs.

independent platform

• Critical limitations concerning mobile research

identified: e.g. survey length, exhibits, types of

questions, scales, incentives

• Mobile segments illustrate personal differences:

Millennials, Multicultural

• Online sample providers just beginning to

understand/get comfortable with mobile

surveys (incidences, costing, viability)

PHASE ONE:

The mobile device

l ifestyle integration

Usage & Lifestyle: Key Flashpoll Findings

Smartphone Usage/Integration:

• Placing phone calls, texting, sending e-mails, and

Internet searches are in the top five usage

categories—ideal for communication

• Supports desk research concerning smartphones

as “connectivity to the world”

• Smartphone usage is well integrated into daily

lifestyle (home, work, vehicle)

• Smartphones used extensively while shopping

• Smartphones used to conduct a wide variety of

tasks, many times simultaneously

• Smartphones used throughout the day/within

periods of “constant interruption”

Smartphone Lifestyle Usage

PHASE TWO:

19%

24%

35%

61%

62%

70%

94%

Other

At school

Internet café/Coffee house

While shopping

In the car

At work

At home

Survey subjects:

Internet Sites Visited, Grids

Store Display, Image Testing

Mobile Preferences, MaxDiff

Concept Evaluation

Demographics

Online Survey Assessment

Mobile Survey Assessment

Best Buy Segmentation

1 5

Agree Completely

Disagree Completely

Scales types:

Multiple Mention

Single Response

Three-point, fully-anchored

Five-point, fully-anchored

Five-point, end-anchored

Five-point, semantic differential

Six-point, fully-anchored

PHASE THREE: Sett ing the stage

quantitative survey

Questionnaire and Design: Key Elements

Design elements:

Response Rates

Incentive Structure

Survey Timing

Open-Ended Questions

Respondent Assessment (Worldview)

Technology Limitations

Key Comparisons

quantitative survey

Design Elements: Key Findings

PHASE THREE:

Research Results

No difference between

platforms

Higher incentives not required

for smartphones

Not as restricted as thought

Minimal verbatims possible

Different, but thought to be

converging

Limited by device

Traditional Thoughts

Significantly lower for

smartphones

Higher incentives required

for smartphones

Restricted for smartphones

Difficult for smartphones

Much different for

smartphone users

Limited significantly

RESPONSE RATES

INCENTIVES

TIMING

VERBATIMS

WORLDVIEW

TECHNOLOGY

Image Affinity: 3-Point Scale Respondents were asked how much they liked a

store display using a 3-point scale, where:

1 = Dislike it

2 = Feel neutral about it

3 = Like it

9% 11% 10%

66% 49% 49%

25% 40% 41%

Online: NoSmartphone

(A)

Online: HaveSmartphone

(B)

Smartphone(C )

Like it

Feel neutral about it

Dislike it

B,C

A A

Like vs. Dislike

Comparison of Results

quantitative survey

Questionnaire Design: Key Findings

PHASE THREE:

Significance testing at the 95% confidence level

n=330 n=174 n=487

“Using the scale below, overall, how much do you like this store display?”

Concept Test: 5-Point Fully Anchored

“Which of the following statements best describes how much you like or dislike this product/idea?”

Respondents were asked to rate the

concept on a 5-point anchored scale

Comparison of Results

quantitative survey

Questionnaire Design: Key Findings

PHASE THREE:

8% 3% 2%

7% 2% 7%

47%

31% 24%

28%

40% 43%

11% 24% 24%

Online: NoSmartphone (A)

Online: HaveSmartphone (B)

Smartphone (C )

5 Strongly like it

4 Somewhat like it

3 Neither like it nor dislike it

2 Somewhat dislike it

1 Strongly dislike it

B,C B

B,C

B,C

B,C B,C

B,C A

B,C

A A

C

A

C

C

A

A

A,B

A

B,C

T2B = 64%

A A

T2B = 39%

T2B = 67%

B

A

Significance testing at the 95% confidence level

n=335 n=176 n=495

Display Appearance: 5-Point End Anchored

8% 8% 7%

18% 14% 16%

31% 24% 21%

27% 33% 33%

15% 20% 23%

Online: NoSmartphone

(A)

Online: HaveSmartphone

(B)

Smartphone(C)

Agree Completely

4

3

2

Disagree Completely

C

“The table is inviting and I would want to look at the products displayed.”

Respondents were asked to rate the

display table on a 5-point scale, with:

1 = Disagree Completely

5 = Agree Completely

Comparison of Results

quantitative survey

Questionnaire Design: Key Findings

PHASE THREE:

Significance testing at the 95% confidence level n=300 n=174 n=487

Clearly View Image:

Semantic Differential Mean Summary

Respondents were asked to rate the image on

four 5-point differential semantic scales.

Online: No Smartphone Online: Have Smartphone

Smartphone

Sample Size n=312 n=174 n=487

(1) Organized vs. (5) Messy 2.26 2.18 2.30

(1) Appealing vs.(5) Unappealing 2.65* 2.40 2.53

(1) Stands out vs. (5) Blends in 2.72 2.57 2.68

(1) Clear vs. (5) Confusing 2.31 2.28 2.33

* Significantly Higher than 1 other group

Comparison of Results

quantitative survey

Questionnaire Design: Key Findings

PHASE THREE:

Significance testing at the 95% confidence level

“Below is a list of attributes with differing meanings. For each pair, please select the one attribute that best describes the store display.”

8% 12% 13%

33% 26% 21%

19% 17%

15%

11% 10%

8%

25% 30%

36%

4% 6% 8%

Online, NoSmartphone (A)

Online, HaveSmartphone (B)

Smartphone (C)

Memory

Features

Size

Battery

Price

BrandB,C

A

B,C

B,C

B,C B,C

A

B,C B,C

A,B C

A

C

C

A

A

A

A,B

Respondents completed a MaxDiff exercise where they ranked various mobile phone features, including: brand, price, battery life, size, features (e.g. camera, video) and memory.

Quantitat ive comparisons

statistical analysis

Advanced Analytics: Key Findings

PHASE FOUR:

MaxDiff Analysis: phone

feature rankings

Significance testing at the 95% confidence level

n=335 n=176

n=495

“Of the attributes listed below, which one is most important when choosing a mobile phone?”

Survey Enjoyment Willingness to Participatein Future

Smartphone/Online Survey

Willingness to go toSpecified Location/Store to

Participate

Fair Incentive

68%

90%

73% 69%

76%

99%

66% 66%

Mobile Online

NOTE: Top two boxes on a five-point scale.

Survey comparisons

respondent participation

Survey Satisfaction: Key Findings

PHASE THREE:

Smartphone: n=495 On-Line: n=511

LIKES DISLIKES

Anywhere at anytime

• “I don't have to be at home or in front of the computer. I can do this anywhere.”

• “Ease of being able to complete the survey from anywhere. Not needing to be tethered to my desktop.”

Convenience

• “I always have my phone on and with me. Not so for the computer. So, convenience.”

• “It's convenient, and I can get them done quickly because I always have my phone with me.”

Easy to do, easy to access

• “Easy to complete and does not require more effort than a survey taken on a computer.”

• “First time taking survey on mobile – it was easy to follow and easy to get thru, not confusing in the least.”

Fun, different, new

• “It was fun and easy to answer the questions. Normally I take surveys on my laptop, so using a different method such as a mobile phone was interesting.”

• “It was new, interesting, and fun!”

Difficulty typing/Small keyboard

• “Don't like typing on my phone cause the keyboard is too small.”

• “Excessive typing can be a problem. I'd prefer longer, essay response answers for the computer.”

Pages load slowly/Too long/Drained battery

• “Pages load slowly, takes longer than computer.”

• “Too much time, wastes battery life.”

• “Doing anything on the web on my phone takes longer.”

Screen too small, survey content too much for screen

• “The screen on my phone is too small to see the questions and the answers at the same time.”

• “My blackberry is too small to comfortably take the survey.”

• “The survey was made bigger than my screen and it was hard to select my choices at times.”

Too much scrolling

• “It takes so long, and I have to scroll left and right, up and down to read the question.”

Prefer participating from computer

• “It’s easier to do most things on my computer.”

PHASE THREE: Mobile Survey

respondent participation

Survey Satisfaction: Key Verbatims

Crit ical Findings

quantitative survey

Design Elements – Scale Types

Response rates are similar across platforms. Males trend directionally higher.

Smartphone respondents will not require higher or significantly different incentives.

There are both technology limitations and different “worldviews” for smartphone users, but this should dissipate in the near future.

Smartphone platforms can viably support 5-point fully-anchored scales, but 3-point anchored and 5-point end-anchored scales are recommended.

Scale usage does not differ significantly, supporting the use of a hybrid methodology and the integration of mobile and online platforms.

Statistical techniques have similar results across survey platforms, supporting the smartphone platform for in-store, real-time data collection.

• Difficulty typing/Small keyboard

• “Don't like typing on my phone cause the keyboard is

too small.”

• “Excessive typing can be a problem. I'd prefer longer,

essay response answers for the computer.”

• Pages load slowly/Too long/Drained battery

• “Pages load slowly, takes longer than computer.”

• “Too much time, wastes battery life.”

• “Doing anything on the web on my phone takes longer.”

• Screen too small, smaller text, survey content too much

for screen

• “The screen on my phone is too small to see the

questions and the answers at the same time.”

• “My blackberry is too small to comfortably take the

survey.”

• “The survey was made bigger than my screen and it

was hard to select my choices at times.”

• Too much scrolling

• “It takes so long and I have to scroll left and right, up

and down to read the question.”

• Prefer participating from computer

• “It’s easier and faster to do most things on my

computer.”

Smartphone platform

critical implications

Smartphone surveys are a viable option to supplement studies with hard-to-reach segments of the population

Data and scale usage is comparable across platforms, supporting the use of hybrid designs

Smartphone users today tend to be more tech-savvy with unique shopping, usage and behavior preferences regardless of platform

Smartphone platform may more readily support specific subgroup segments, i.e. Hispanic, Millennials, as well as global markets

Mobile-enabled surveys are ideal for in-store shopping and exercises, real-time assessment, and interrupted surveying

The “Strategic Enhancement” of today could well be

the next paradigm—or the 5th methodology—of

tomorrow.

• How will the marketing research industry prepare for this

evolution?

• Does it reduce barriers to reaching specialized segments

(Millennials, Hispanics, global markets?)

• Will this offer greater authenticity to marketing

research (rational/emotional surveying). Are

smartphone devices more conducive for lifestyle-based

interviewing?

• Does it support the evolution of the marketing research

discipline from the current scientific/data-based structure

to a humanistic/decision-making approach?

Mobile Research

considerations for the future

Michael Francesco Alioto, Ph.D.

Vice President

Marketing Sciences

Gongos Research

Greg Heist

Vice President

Research Innovation

Gongos Research

Mobile Surveys | The 5 th Methodology: A New Paradigm or a Strategic Enhancement ?

q&a | discussion

Presented at:

Market Research in the Mobile World 2nd International Conference | July 19 & 20, 2011 Atlanta

Organized by: Thank you to sponsors:

LinkedIn Group: Mobile MR

Upcoming Merlien Events: Merlien.org GreenBook Directory: GreenBook.org Market Research Blog: GreenBookBlog.org New Qual Blog & Directory: NewQualitative.org

Get Inspired. Stay Informed. Cut Through the Clutter. Sign up for our free email newsletter