Smartphones and Smarter Surveys - Panjaaporpanjaapor.org/images/Buskirk_PANJAAPOR2014_Final.pdfA market research consultant goes to lunch with his clients – one is in the Marketing

. 10/29/14 1Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR

Smartphones and Smarter Surveys:

Trent D. Buskirk, Ph.D.

PANJAAPOR October 29, 2014

Practical Considerations

for Optimizing Mobile

Surveys


Instructor: Trent D. Buskirk, Ph.D.

Current Position: Vice President of Statistics

and Methodology @ Marketing Systems Group

Lives in: Saint Louis, MO

Focus on:

Use of new technologies for data collection

Mobile Phone Sampling

Dual Frame Designs and Weighting

Mobile Analytics and Paradata Collection and Analysis

Addicted to:

Playing Prince for my little Princesses

APPS and iPhones

SAS, SUDAAN and R

Tennis

Boost 101.9 (www.boost1019.com)

http://www.m-s-g.com/Web/Index.aspx

http://www.boost1019.com/

http://www.m-s-g.com/Web/Index.aspx


PANJAAPOR Committee for their invitation!

Colleagues at MSG for their assistance and support

Colleagues at Maritz Research for collaborations on an

advanced mobile survey mode comparison study

3

Acknowledgements and Thanks!


A market research consultant goes to lunch with his

clients –

one is in the Marketing department,

the other is in Operations.

The three of them get in the car together.

The Marketing manager immediately has his foot on the gas,

the Ops guy has his foot on the brake, and the Survey guy is

looking out the back window telling them where to go.

Warm-Up


Agenda & Goals

Framework for Data Collection on Mobile DevicesAn overview of the new mobile survey landscape

Developing Emerging Best Practices: A blend of old and new

Types of Mobile Surveys

Recommendations for Internet/App based survey solutions

Evaluating Mobile Surveys – User Perspective

Goal: Leave with a solid appreciation of:Some practical “best practices” for deploying mobile surveys

Issues to consider for optimizing data collection efforts via mobile devices


Focus on Smartphones

The Picture

Summary!


Remembering a simpler time…And a simpler phone…

Cartoon Sourcehttp://voices.washingtonpost.com/comic-riffs/2009/06/top_picks_what_this_riffster_i.html


And now introducing FUTURE PHONES!

Cartoon Sourcehttp://voices.washingtonpost.com/comic-riffs/2009/06/top_picks_what_this_riffster_i.html


Section 1:

Developing Best

Practices in the

New Landscape…

6:59


Mobile Devices Covering the World

Sources: http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536

http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15

4.55 Billion People Worldwide Using

Mobile Phones

1.75 Billion People Worldwide Using

SmartPhones

400 Million People Worldwide Own

Tablet Devices

788 million mobile-only internet users

by 2015 worldwide (Cisco, 2014)

http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536

http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15


Mobile Devices Covering the USA

240 Million Mobile Users (age 13+) in the

U.S.

91% of All U.S. Adults have a Cell Phone

156 Million Smartphone Owners in the

U.S. (age 13+)

52% of U.S. Adults have a Smartphone

82 Million Tablet Owners in the U.S.

42% of U.S. Adults have a Tablet

Sources: comScore (2014) and Pew Research (2014) http://www.comscore.com/Insights/Presentations_and_Whitepapers/2014/2014_US_Digital_Future_in_Focus?elqCampaignId=252/

http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/


Developing Mobile Surveys: Interface & Flow

User interface: usually an important consideration in data collection,

with new technologies it is essential

New & extended human-computer interaction concepts:

Visual appeal

Intuitive flow (more than simply “logical”)

Ease of use /functions are readily apparent

Few, if any, text instructions

Optimize mode capabilities and reduce user anxiety and frustration

Study of why people participate in mobile surveys and mode choice:

Perceived enjoyment; attitudes toward participation; self-expressiveness, & trust

factors most important (Bosnjak et al 2010)

Previous experience with alternate survey modes (other than computer) increases

preferences for mobile surveys (Buskirk, et al., 2013)

Blend current understanding of CAI systems with expanding literature

on human-computer interactions (use & interaction)

http://www.comscore.com/Insights/Presentations_and_Whitepapers/2014/2014_US_Digital_Future_in_Focus?elqCampaignId=252/

http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/


The new paradigm: MOBILE Surveys

Smartphone Survey

Design Research

Social Science

ResearchBuskirk and Andrus (2012)

Link and Buskirk (2012)

Kinesis White Paper (2010)

Buskirk et al. (2011)

Callegaro (2010)

Callegaro and Macer (2011)

Peytchev and Hill (2010)

Zahariev et al. (2009)

Smith (2011)

Bosnjak, Metzger, Graf (2010)

Lai et al. (2010)

Couper (2010)

Millar and Dillman (2012)

Human Computer

Interaction

Parush and Yuviler-Gavish

(2004)

Jones et al. (1999)

Sweeney and Crestani

(2006)

Fitts (1954)

Goldberg, Fariden and

Alterovitz (2007)

Zichermann and

Cunningham (2011)

Computer Science

and Web

Development

Koch, P-P. (2010)

Campaign Monitor (2012)

Kriss (2013)

Medallia (2013)

EmailVision (2013)

Html5rocks.com (2013)

Pelletier (2013)

Return Path (2013)


Respondent Burden in the Mobile World

Respondent burden in mobile world is similar to respondent

burden in the regular world with a few differences

Two Groups: Personal and Technological

Personal: Safety

Respondents aren’t just responding at home anymore!

Steeh, Buskirk and Callegaro (2007) report respondents completing cell

phone surveys while boarding a helicopter.

Personal: Privacy

New data types like photo/voice may mean information is collected from

more than the consenting respondent

Age of assent on internet is younger than consent via phone (13 vs 18/19)

Personal: Time

To complete survey tasks [ACTIVE]

To load survey pages [PASSIVE]


Respondent Burden in the New Mobile World, Cont.

Technological

Data consumption

Both receiving and sending information (survey questions/content and

submitting responses)

Nielsen (2012, Q3/Q4) Estimates that 96% of mobile phone users have

a data plan and that the average monthly bill is $66 (all mobile) and $93

for smartphone users.

Bandwidth

Slower bandwidth increases time required to send and receive data

Higher data consumption may lead to throttling by the provider –

meaning reduced bandwidth in future

Battery Drain

Persistent GPS capture drains mobile device batteries

Extra HTTP requsts can drain mobile device power


Respondent Expectations

Being involved in a scientific survey is a relatively rare occurrence

Little, if any, expectations by respondents

Respondent experience with new technologies is much different

Respondents are more savvy

Technology changes rapidly – some respondents will find some tasks difficult

or not easy to use.

[Example: Michaud, Buskirk and Saunders, 2014 Voice Data Entry]

Developing expectations from these experiences

Ease of use/intuitive

Speed

Usefulness/utility or fun/entertainment or both

Ability to share experiences w/ others

Location awareness

Auto detection/passive collection


Even my little one knows that a phone isn’t just for

talking!

“Can I WATCH

your Phone?”


Section 2: Types of Mobile

Browser Surveys

16:59


Various Approaches for Survey Data Collection

via a Smartphone

Use the regular phone mode

Use Short Text Messages (SMS)

Flash polls, Invitations, Synchronous Survey Interviews

App-based/administered surveys

Online surveys accessed via Smartphone Browser

Passive and Active versions

Hybrid Approach: App-Like Smartphone Browser

Surveys

Source: Buskirk and Andrus (2012) www.surveypractice.org.


Online Surveys via Mobile Web Browser

Passive: Surveys intended to be completed online are

sometimes completed via mobile phone web browser (passive

mobile web browser approach).

Active: The online web-based survey is optimized for the

mobile environment. (think mobile versions of survey

websites)

Many survey software programs (e.g. SNAP, Qualtrics, etc.)

have mobile publishing options that optimize an otherwise

online survey for the mobile browser.

If you know the type of devices on which those sampled will

complete the survey, you can optimize the survey design for

the particular browser/screen resolution (e.g. Safari / iPhone).

http://www.surveypractice.org/


Passive versus Active Smartphone Surveys

Passive Mobile

Survey from Hotwire

Survey Appearance

after a “double tap”Active Mobile Survey

Both viewed on an iPhone 5


APP-LIKE Mobile Browser Surveys

APP-LIKE Mobile Browser Surveys (A-LMBS) represent a hybrid of

the APP and the Active Mobile Browser Approaches.

A-LMBS are completed via the mobile web, but they rely on active

browser refreshment using a combination of PHP Programming and

Javascript to create “Active” Mobile web pages.

A-LMBS can be developed to run without a persistent internet

connection on multiple mobile browsers (HTML5)

A-LMBS offer native, APP like functionality under the mobile browser

context.

In today’s browser vernacular, app-like surveys might fall under the

class of Web apps which are defined as web pages that function

and appear almost app like but are accessed via browsers

Source: Buskirk and Andrus (2012) www.surveypractice.org.


A-LMBS Compared to Active Mobile Web Browser

A-LMBS

Approach

(iPhone)

Active

Mobile

Browser

Approach (iPhone)

Active

Mobile

Browser

Approach (Android)

A-LMBS

Approach

(Android)


App-based Smartphone surveys administer and collect data via

an (native) app that is installed on the sampled user’s

Smartphone.

Surveys can be “pushed” to the app and then executed by the

end user without the need for perpetual internet connection

and can be designed to take full advantage of the phone’s

capabilities like camera, voice, video, image capture, etc.

The App can also serve as a survey manager and push people

to surveys that are actually optimized and executed via the web

(natively, App-like)

App-based/administered surveys

Data transfer for completed surveys can occur once internet

connection is established.

One popular example of this approach comes from

Confirmit/Techneos’ Survey On-Demand Application (SODA)http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx

http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx


Section 3: Smarter

Smartphone

Survey Data

Collection and

Design Recommendations

S3DR

16:59


S3DR #1: Mobile Survey Approach

Select an implementation

approach that:

(a) optimizes presentation of

survey content

(b) is consistent with the type of

survey data desired

(c) is appropriate for the context.

S3DR #1


How to decide the approach used here?


S3DS #1

Email Invitations and Web Addresses for

Smartphone Surveys should be as simple and

straightforward as possible.

Wherever possible, email invitations should be

designed responsively for easy reading and

navigation on smartphone and tablets.

S3DR Tip

S3DR #2: Recruitment for Mobile Surveys via Email

S3DR = Smarter Smartphone (and Tablet) Survey Design Recommendation

Place the key information (e.g. survey sponsor, incentive…) at the

beginning of the email subject.

Place the survey link as soon as you can in the email invitation, to avoid

scrolling.

Minimize the number of “special characters” in the survey web address.

If at all possible try to avoid “aliased” web addresses (especially for

mobile web version).


Email on Smartphones and Tablets

Email and the Mobile Device Reality:

52% of all cell phone owners use email on their phone (Pew, 2013)

78% of smartphone owners check email on their phone. IDC and

Facebook – “Always Connected” (2013)

10% Of iPad users say Mobile is their preferred device to read

email and 9% for sending it. –Perion “iPad Owners Survey” (2013)

Among mobile email users, If an email does not display

correctly, BlueHornet “Consumer Views of Email Marketing” (2012)

69.7% will delete it immediately;

17.7% will view it on a computer

7.6% read it anyway on their device .

Men open 20% more emails on mobile, but WOMEN click 10%

more often on mobile e-mail TailoredMail – “It’s time to wake up and mobilize!”

(2012)


With the surge in Mobile Internet Use comes

the rise of unintentional mobile respondents

http://www.maritzresearch.com/ Source: Maritz Research


Email Formatting Recommendations

EmailVision provides additional recommendations

pertaining to optimizing email deliverability

For Subject lines and sender options, EmailVision

recommends:

Rather than generic emails such as [email protected] use

relevant email addresses such as:

[email protected],

[email protected]

[email protected]

enable the recipient to understand the intent of the email and

potentially reduces auto spam classifications.

http://www.emailvision.com/sites/default/files/email

vision-deliverability-bestpractices-2011-03.pdf


Responsive Email Designs for Survey Invitations

Campaign Monitor’s E-Publication “Responsive Email

Designs” offers several considerations for developing mobile

responsive emails including:

Single column layouts (500 – 600 pixels)

Links/Buttons minimum target area of 44 X 44 pixels

Minimum font displayed on iPhones is 13 pixels – smaller font than

this will be upscaled and formatting may change

Call to action (i.e. “start survey button” or survey link) should appear

toward the upper portion of the email to avoid scrolling

For CSS programming use display:none option to limit social sharing

buttons and other extraneous details in the mobile environment.

SOURCE: Campaign Monitor Responsive Email E-Book (2013):

http://www.campaignmonitor.com/guides/mobile/

http://www.emailvision.com/sites/default/files/emailvision-deliverability-bestpractices-2011-03.pdf


Why focus on Responsive Mobile Emails?

Email Invitation on

iPhone

Survey that appears

after tapping “Yes”


Mobile Optimized and Non-Optimized Emails

SOURCE: Campaign Monitor Responsive Email E-Book (2013):

http://www.campaignmonitor.com/guides/mobile/design/

Mobile Optimized Not Mobile Optimized

http://www.campaignmonitor.com/guides/mobile/

http://www.campaignmonitor.com/guides/mobile/design/


Example of Responsive Email from Amazon.Com

Email sent to Amazon

Customers Viewed on

Desktop/Laptop (responsive)

Call to

Action

Here

Same email

viewed on

Smartphone

Source: http://marketingland.com/four-responsive-email-layouts-15858

Column Dropping


From Responsive Email to App-Like Mobile Survey…

http://marketingland.com/four-responsive-email-layouts-15858


From Email to Mobile Web…


Internet Use, Time of Day and Device

Source: comScore, 2013 http://bit.ly/1c9vneo :

http://bit.ly/1c9vneo

http://www.comscore.com/Insights/Presentations_and_Whitepapers/2013/2013_Mobile_Future_in_Focus


S3DR #3: Device Paradata

Successful implementation of multimode surveys requires

that the survey software include comprehensive device detection

functionality. Device detection is needed to automatically identify

the type and brand of each respondent’s device, and then display

the survey according to that device’s specifications.

-Cazes, et al. Kinesis Mobile Landscape Report, 2010http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

Plan to collect paradata from mobile sample

including device type, size and browser size.

Browsers and devices matter and can be key to

an overall survey’s functionality on a

respondent’s device.

S3DR #3


Here’s a look at why browsers and device types matter

Users from Android devices have some systematic differences

from those of iOS devices – can be used as a covariate in

analyses.

Ultimately, device detection via user agent strings helps to

deliver overall content to the correct type of device (phone

versus tablet versus computer) and perhaps the operating

system.

However, the way the device translates the survey

programming (HTML markup etc) is determined by:

The browser in use

The features of the browser

The features of the device (e.g. hard keyboard versus virtual, front facing camera or not)

http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf


Android Owners vs iPhone Owners

Source: Estimates Recomputed from Tables Included in Pew’s

Smartphone Ownership – 2013 Update (Smith, 2013): http://bit.ly/Qnt4Rr

Demographic VariableDistribution among

iPhone Users(U.S. Adults)

Distribution amongAndroid Users (U.S. Adults)

Men 45% 51%

18-34 36% 43%

55+ 27% 20%

High School or Less 20% 32%

College Degree (or More) 54% 39%

Non-Hispanic Black 8% 19%

Non-Hispanic White 80% 70%

<30K annual HH income 15% 28%

>75K annual HH income 51% 35%


Android and iDevices and Geography!

Geographic differences may also exist among the iPhone

and Android User base within the U.S.

Study conducted by Mobify in 2013

Using 20,000 partner websites with over 200 million visits through 2012

93% of mobile traffic came from either an iOS or Android mobile devices

(tablets and smartphones)

Source Mobify (2013): http://www.mobify.com/blog/ios-vs-android-in-2012/

More blue=higher percentage

of views from iOS

More Red = Higher Percentage

of views from Android

http://www.mobify.com/blog/ios-vs-android-in-2012/


Specific Example: Same Phone…Different Browser

http://user-agent-string.info/


Why browsers matter for Mobile Surveys!

Android Browser

(default/native)

Galaxy Note 3

Android (KitKat

4.4.2) SmartphoneChrome Mobile

Browser (activated)

Only first three choices

visible without scrolling

(2 taps)

ALL Choices

Visible

(One Tap)


Smartphone surveys should be designed to

be as short as possible, but not necessarily

abbreviated versions of longer online

surveys.S3DR #4

S3DR #4: Survey Length

Dead spots still exist for Smartphones

In both voice and data (i.e. edge, roaming)

Internet speeds vary also by area

Sprint has both 4G and 3G speeds in St Louis, for example.

Smartphone users multitask

Three good reasons to allow for users to return to the survey site

Start over?

Begin where they left off (approximately)


S3DS #1

The overall survey layout should minimize the need

for scrolling (either horizontally or vertically) to the

extent possible.

Survey layout should also minimize the need for

pinching/zooming.

The number of questions per screen should

generally be determined in order to minimize

scrolling (implied scrolling)

Be mindful for the device and place key actions in

“good” zones

S3DR #5

S3DR #5: Survey Layout

Some devices don’t support zooming

Answer choice layout may create need for scrolling on some devices.


Putting Responses within Reach, for mobile devices!

Source: http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/

Smartphone

in Portrait

Tablet in Portrait

Tablet in Landscape


Horizontal Scrolling Experiment (Peytchev & Hill, 2010)

No Significant Differences Noted

More

Responses

M

o

r

e

Q

u

e

s

t

i

o

n

s

http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/


Example of Survey from Frontier Airlines

with Horizontal Scrolling Prompts


Consider reducing the number of response options

for grid questions or reorganizing grid questions as

single questions per screen.

Consider organizing response options vertically or

horizontally with scale labels provided per

occurrence (see Peterson, 2013).

If Multiple choice/select all that apply require a long

list of alternatives, consider converting question

into free response with as many single-text fields

as answers expected (i.e. what are three of your

current research interests – list up to three).

S3DR #6

S3DR #6: Question Formats


Some Grid Examples on Mobile Devices

Example of Grids – reworked for mobile surveys

Source: Peterson et al. (2013) http://bit.ly/1i5WYo0

HARD!! Easier! Easier!


Got Health Apps Study Screenshots Illustrating

Question Layout and Response Choice Feedback

Source: Buskirk and Andrus (2014)

http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref

http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref


Paging Versus Scrolling Question Presentation

Scrolling question presentation in mobile surveys

presents a series of survey items on a single page

Requires respondents to scroll (vertically) in order to see (and

answer) all survey questions on the page

Paging question presentation in mobile surveys

limits the number of questions per page

Generally one question per page and the respondent

navigates multiple pages in order to complete the survey

Buskirk and Andrus (2014) present a hybrid with two

questions per page to limit (a) the vertical scrolling required

on mobile; (b) the overall number of pages needed for survey

completion and (c) increase comparability for desktop version


S3DR #7: Scrolling Versus Paging

(a) Consider an optimal combination of these

methods for presenting questions.

(b) If you have a series of questions with similar

response options and type then scrolling may

help move a respondent along more quickly

and simply.

(c) If you have multiple skip pattern logic in your

survey, paging may be preferred (at least for

that section).

(d) Be careful that scrolling is suggested or implied

so respondents won’t skip (especially if

Next/Back are at the top)

S3DR #7


An example of Scrolling in practice

Requires

tapping here;

then entering

value; then

tapping out

then scrolling

and repeating.


Example of a Survey w/ a Heavy Scrolling Section

IMPLIED Scrolling

here

Questions have

similar answer

choices and

method of entry


Sliderbars, sum tallies and drag and drop sorters

may not work on all devices.

These question types should be tested extensively

using emulators prior to their adoption in your

mobile survey.

Text entry fields should be wide enough so that

users can see their entire entries without scrolling

horizontally. Use this type of entry judiciously.

Decisions about types of input/forms or input should

include consideration of number of taps or clicks

required (NTR or NCR) to enter (and register) an

answer.

S3DR #8

S3DR #8: Question Types


A quick note about sliders

Be wary of the default starting positions – could

grossly impact final results

Michaud, Buskirk and Saunders, 2014

Peterson et al., 2013

Can’t distinguish a “3” from missing if default start is midpoint

Source: Peterson et al. (2013) http://bit.ly/1i5WYo0


Using the NTR metrics to help weigh design

choices, in practice.

Michaud, Buskirk and Saunders (2014) included an

experiment to compare radio button and list inputs for

various substantive questions.

Initially respondents that were randomized to lists received

this format for all survey questions including demographic

questions about gender, education, race, etc..

Pilot testing and the NTR metrics changed our plans for the

demographic questions…


Using the NTR metrics to help weigh design

choices, in practice.

Radio Button

Version (all devices)

List/Dropdown iOS

and Stock Android

Selector Android

(some browsers)

NTR=1

NTR=3/4

NTR=2


Beware of the Pagination/Scrolling Mashup!

Just because it’s optimized for smaller screens doesn’t mean it’s

optimized for respondents!

Presentation of questions only without their response options will save space

on a screen, but makes data collection/completion a mine field!

Example from Major League Baseball!

https://www.fansatbat.mlb.com/PORTAL/default.aspx

The survey question (depicted on an

iPhone 5):

“Are any of these networks included

in your current cable lineup?”

Tap Here to make selection

NTR=2 per item


App-Like Mobile Browser Surveys are easier to

deploy using HTML 5 functionality

An App-based approach may not be the only way to

collect modern types of mobile data – the mobile

browsers may now access many “on-board”

components of the mobile phone via HTML 5.

Familiarize yourself with the functions that are

available for the majority of operating

systems/devices for your users

Make sure you have a “back-up” plan for when the

HTML 5 functionality is not available

S3DR #9

S3DR #9: Get Familiar with HTML 5 and Use its

functionality when possible:

https://www.fansatbat.mlb.com/PORTAL/default.aspx


HTML5 and the rise of app-like functionality

The advent of HTML5 has given programmers and

researchers a lot of flexibility and versatility in creating

mobile versions of websites with many supported input

types for mobile forms and capabilities:

Geolocation collection

Media capture (voice, photo and video)

Improved and expanded input types

Web storage

For survey researchers, this new development means that

our active mobile browser surveys can be made more and

more app like with standard, predefined enhancements that

have been include in HTML5.

But this is not yet a silver bullet!! Care must be taken!

http://html5test.com/index.html


Case Study Example of HTML 5 in Action

Walmart in partnership with Mattel Hot Wheels launched a loyalty

program for shoppers.

Shoppers who purchased Hot Wheels products could receive

points redeemable for various rewards by scanning their receipts

showing the Hot Wheels item purchased.

The loyalty program required NO app, yet enabled mobile photo

capture!

Scanning/photo functionality and uploads to the Walmart program were

managed via an online website optimized for mobile visitors with (modern)

browsers


Hot Wheels Redemption Program on iPhone

SOURCE: Hot Wheels (2013) https://hotwheelsrewards.com/


Practical Example from New Study!

Michaud, Buskirk and Saunders

(2014) designed a study in

partnership with Decipher and

Research Now to compare various

touch and data collection

possibilities across three modes –

computers, tablets and smartphones.

A photo capture question was

deployed to all devices detected as

either “Tablet” or “Smartphone”

using user agent string classification.

Kindle Fire Tablets Respondents

running the native Silk browser had

NO photo uploads as expected from

browser limitations

Those with devices without this

functionality had to press continue

twice to continue past this screen.

https://hotwheelsrewards.com/desktop/


Taking Advantage of New Input Types from HTML5

With the release of HTML 5 there are several new input

types available that might be of interest to survey

researchers and questionnaire designers including:

Number

Tel

Email

Range (a.k.a. sliders)

Time

These new input types when used wisely will likely reduce:

The overall amount of additional scripting that is required to format

the survey questions and response options

Reduce survey page load times

The NTR metric for a given question

Source: MobiForge, 2013 http://bit.ly/1aH3vnM


Numeric, open ended questions using new HTML

input options Number and Tel

How many miles per year do you drive the car that you personally drive

most often. (Enter a number between 0 and 50000)

Question PageDefault Input Using HTML5

“number”

input type

By using a text default,

the NTR increases by one

because respondents

have to select the number

window

http://bit.ly/1aH3vnM


Making Numeric Input Similar across Devices

By using the HTML5 “tel” input type, a numeric

keypad similar to what is displayed when making

calls is displayed on all devices.

Formats consistent

across devices in

terms of number entry.

Note however, that if

decimals are required,

then iOS keypad does

not offer this capability

directly.


S3DS #1

Limited screen landscape requires compromise

and judgment when using logos, progress bars,

disclaimers and help links in mobile surveys.

Consider using abbreviated progress bars

Placement of next and back buttons should be

oriented toward the top of the screen or at the

bottom of the screen as long as they are

persistent and visible without scrolling

Disclaimers should be placed on the welcome

page.

S3DR #10

S3DR #10: Navigation/Progress


Illustrating Back Button and Progress Bar Placement

Next/Back Buttons

Progress Bars


Another Take on Next/Back and Progress Bars

Progress Bar

(Persistent)

Next/Back

Buttons

Persistent


S3DS #1

Response selections, action buttons and navigation

tabs in surveys should be made as large as possible

on mobile surveys.

Touch input differs from Mouse input.

Fattest Fingers – Average width of index finger is 11

mm with a range of under 7 for babies to over 19 for

tall athletes.

Make sure to include padding around important

navigation tabs (next/continue/enter) to avoid

accidental taps .

S3DR #11

S3DR #11: Buttons, Tabs and More Buttons


Importance of Button Sizes

Make butons/tabs large enough to not be missed/avoid mistaps

40 pixels (7 mm by 7mm) GOOD

50 pixels (9mm by 9 mm ) BETTER

30 pixels (5 mm by 5 mm) OK

Give enough buffer around/between tabs/buttons

Padding of at least 2mm (10 pixels) between targets

Source: Microsoft Touch Guidance


One accidental tap could be missing data!

From Michaud, Buskirk and Saunders (2014)

We included slider bars as input options for scale questions. The slider bar

button was made large enough to drag across phone and tablet devices.

On smartphones such as iPhone, the “continue” tab was too close to the

upper answer choices, so people who dragged to either 7 or 8 could have

tapped the continue button by accident on the way to a rating of 9 or 10.

In the radio button version, people attempting to tap an 8 9 or 10 could have

tapped continue on the way there and recorded no data.

Before

adding

padding

of 4

pixels

After

Padding

Added


S3DR #12: Recommendation for Mobile Surveys that have

a Multilingual Audience/Various Language Versions

Remember the higher incidence of mobile users

among Hispanics and Asians in the U.S.

When deciding on spacing of answer choices,

resulting font size implications and layout for a

given set of questions, make choices in light of

the languages that will be presented.

For example, if the Spanish version of the

answer choices are actually longer, then

optimize for these choices first rather than

always optimize for English versions.

S3DR #12


Optimizing for English Versions may not be sufficient!

Questions or

their answer

choices (in

English) that

appear to be well

optimized and

presented using

mobile browsers

may not be

optimally

presented in

another language

ENGLISH VERSION SPANISH VERSION


Company website development method and approach

may not be the most optimal for survey data collection

Distinguish data/content consumption from data

collection using web

Carefully consider other design alternatives such as

mobile specific survey sites or more adaptive designs

S3DR#13: Choose The Survey Website Development Method Carefully:

The default website method may not work for your survey needs!

S3DR #13

Three main approaches for developing an active mobile browser survey:

Responsive Web Design (RWD) for Survey Pages

Mobile-First RWD

Adaptive Designs

Responsive Design with Server Side Components (RESS)

Separate Mobile Web site developed for Surveys


RWD and Mobile First RWDs for Surveys

Source: http://web3canvas.com/what-is-mobile-first-responsive-design/

RWD for Surveys

would be a strategy

that takes an online

(computer survey)

and thinks about how

to render it down to

tablet and then down

to smartphones.

Mobile First RWD

would take a mobile

survey and

progressively push it

to larger devices with

potentially different

functionality.


It’s Time for Vacation– Let’s Go to Disney!

Responsive Web Design Example: www.disney.com

mini

http://www.disney.com/


The other side of RWD…LAPTOP Pinch and Zoom!

Responsive web design can be great at adapting content to smaller

viewports (or devices) but one of the drawbacks for surveys is the

bringing back the pinch and zoom issue to lap/desktops as

illustrated here – a laptop and iphone view of a question from a

sample survey from CheckMarket. https://www.checkmarket.com/mobile-surveys/

Laptop

View

iPhone

View


Adaptive Designs

There are various versions of Adaptive web designs but the

approach that has a lot of traction recently is Responsive Web

Design with Server Side Components (RESS)

RESS is sometimes referred to as a hybrid approach between Server-

side implementation (rendering unto Caesar what is Caesar’s) and

RWD.

RESS uses device detection on the server side to send device specific

survey web page content/scripting

The device specific code can include components that allow it to

progressively enhance the survey page depending on browser

features it detects on the respondent’s device.

References:

S. Rieger (2011) http://slidesha.re/1k1HLUH Pratap (2013) http://bit.ly/IK6Q8A

Ronan (2011) http://bit.ly/1tuHNcJ B. Rieger (2011) http://slidesha.re/1pnzBeA

https://www.checkmarket.com/mobile-surveys/

http://slidesha.re/1k1HLUH

http://bit.ly/IK6Q8A

http://bit.ly/1tuHNcJ

http://slidesha.re/1pnzBeA


Survey Questions

Images used in Survey

Computer Scripting and

HTML Markup (all devices)

www.takesurvey.com

RESPONSIVE WEB

DEISGN – 10000

Feet Overview


Survey Questions

Images for

PHONESComputer

Scripting and HTML Markup

(PHONES)

www.takesurvey.com

RESPONSIVE WEB DEISGN

with Server Side Components

10000 Feet Overview

What Kind of

Device is it? User

Agent String

Says:

iPhone

http://www.takesurvey.com/

http://www.takesurvey.com/


Giving your Online Survey Site a Mobile Sibling

Separate mobile version of the site is the ultimate adaptive

strategy

Special scripts are created to program survey functionality specifically

for the mobile devices

The survey website will generally be a stand-alone website and could

potentially require a different web address.

If you know a priori that you are recruiting mobile respondents, then a

unique mobile link and QR code could be provided.

If you are interested in field work or in-person data collection using

mobile devices and you don’t want to use an app, then a separate

mobile web survey optimized for mobile devices could be a cost

effective and viable solution.

Frost (2012) offers a great comparison or responsive and separate

versions of presidential websites from the 2012 election.



Focus on a survey design that is technically sound

On mobile websites there is a smaller margin of error for:

Image rendering, browser requests, survey page size, waiting times.

77% of cell internet users say they experience slow download speeds

that prevent things from loading as quickly as they would like. Of those

cell internet users, 46% face slow download speeds weekly or more

frequently. (Pew, 2012)

97% of time on a single web page is spent on the front end – meaning

the loading time of the page (Everts, 2011)

Why do we need to focus on techniques that can optimize

survey websites?

Overall Respondent burden

Lag/Latency Waiting time, Time on Tasks, data consumption/exchange, effort

Break-off, partial completion, full completion

Keep in mind that a web survey is a series of consecutive and linked

web pages (each serving up survey tasks to complete).


There is a great deal of functionality that is consistent

across mobile smartphones- but there is enough

inconsistency to require extensive pre-testing of your

mobile survey across multiple platforms.

Consider using smartphone emulators to pretest the

mobile version of your survey or alternatively a virtual

device laboratory to test mobile versions of your survey.

Test availability of flash content using a resource such

as: http://www.adobe.com/flashplatform/supported_devices/smartphon

es.html

(or on your mobile phone/emulator) http://www.adobe.com/software/flash/about/

S3DR#14: Pre-Testing Mobile Surveys

S3DR #14


Mobile Emulator Software Example:Keynote’s DeviceAnywhere

http://www.keynotedeviceanywhere.com/da-free-product-overview.php

Many types of

devices centrally

located in one

source (host of

tablets and

smartphones)


Section 4: Resources

for Mobile

Surveys

16:59

http://www.keynotedeviceanywhere.com/da-free-product-overview.php


Resources for Mobile Surveys

An overview of the survey research to date on mobile devices,

capabilities, privacy and legal issues and some discussion of

what’s on the horizon can be found in Chapter 1 of the: AAPOR

Emerging Technologies Task Force Report

Mobile Web Best Practice Guidelines produced by the World Wide

Web Consortium (W3C)

Gives very detailed suggestions on how to create web pages for mobile

devices, in general. Some of the information on touch interfaces, font sizes

and mobile versioning of online websites may directly pertain to survey

development and practice. http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap

Survey Practice eJournal

Generally has a short publishing time from submission to online posting and

contains many shorter articles highlighting experiments conducted with

mobile devices.


Resources for Mobile Surveys, Take 2

Mobile Marketing Research Association

Has a Knowledge Center where content on mobile market research is

searchable by keywords. http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects

Mobile Glossaries

uSamp Mobile Glossary Contains detailed definitions for many things

mobile including API, GeoFencing and many other terms relevant to

mobile devices and use.http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&

The European Society for Opinion and Market Research

(ESOMAR)

Guidance for Mobile Market Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-

guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf

Guidance for Mobile Survey Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-

and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf

http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap

http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects

http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&

http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf

http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf


Resources for Mobile Surveys, Take 3

Our Mobile Planet Google Site

Contains data from several countries on mobile behaviors by

basic demographics of the user by year (from 2011 onward)

for a host of countries around the world.

http://think.withgoogle.com/mobileplanet/en/

Mobi-Thinking – Website Offered by dotMobi to

educate practitioners about the mobile arena

Special Section dedicated to Global Mobile Statistics

http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats

Flurry Analytics (Service and Blog)

Track user activity in the app with events that are defined by the app

owner. May be useful to collecting paradata for app related surveys.

Also have a blog that provides regular estimates of app usage across a

variety of platforms.

http://blog.flurry.com/ and http://www.flurry.com


Final Comment!

I leave you with this final word from Sir Richard

Branson to encourage all of you to continue to:

invest in experimentation in the mobile survey arena and

share your results (and allow us to LIKE them )

http://think.withgoogle.com/mobileplanet/en/

http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats

http://blog.flurry.com/

http://www.flurry.com/


Thank you!

[email protected]

314-695-1378

www.m-s-g.com

@trentbuskirk


References

AAPOR Cell Phone Task Force Report, 2010 available at: http://www.aapor.org/Cell_Phone_Task_Force_Report.htm , accessed on October 28, 2011

Attfield, Simon, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. “Towards a Science of User Engagement (Position Paper).” Presented at the WSDM Workshop on User Modelling for Web Applications, Hong Kong, China.

Bailey, Justin and Lorelle Vanno. 2011. “Market Research Association Current Mobile Research Trends.” Webinar for the Florida Chapter of Marketing Research Association.

Bosnjak, Michael, Gottfried Metzger, and Lorenz Gräf. 2010. “Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, Self-Expressive, and Trust-Related Factors,” Social Science Computer Review 28: 350-370.

Brick, J. M., P. D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71 (1): 23–39.

Buskirk, T.D., Callegaro, M. and Rao, K. (2010) ‘N the Network'? Using Internet Resources for Predicting Cell Phone Number Status. Social Science Computer Review: Special Issue on Mobile Surveys, Vol. 28, No. 3, 271-286.

Buskirk, T.D. and C. Andrus. (2012a). Smart surveys for smartphone: exploring various approaches for conducting online mobile surveys via smartphones. Survey Practice. Available at: http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ .

Buskirk, T. D. and Callegaro, M. “Cell Phone Sampling,” in Encyclopedia of Survey Research Methods, Lavrakas, P.J. (editor), Sage, Los Angeles, © 2008, pp. 84-89.

Buskirk, T.D., Gaynor, M., Andrus, C. and Gorrell, C. (2011) “An App a Day Could Keep The Doctor Away: Comparing Mode Effects for a iPhone Survey related to Health App Use.” American Association of Public Opinion Research, Phoenix, AZ.

mailto:[email protected]

http://www.m-s-g.com/

http://www.aapor.org/Cell_Phone_Task_Force_Report.htm

http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/


References (con’t)

Buskirk, T.D. and Andrus, C. (2014) “Making Mobile Surveys Smarter: Results from a randomized

experiment comparing online smartphone and desktop/laptop computers,” Field Methods,

forthcoming. See http://fmx.sagepub.com/content/early/recent

Callegaro, Mario. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online

Survey?” Survey Practice, December: www.surveypractice.org.

Callegaro, M. and Macer, T. (2011) “Designing Surveys for Mobile Devices: Pocket-sized Surveys that

Yield Powerful Results,“ Shortcourse Presented at the 66th Annual Meeting of the Americaon

Association of Public Opinion Research, Phoenix, AZ.

Cazes, J., Townsend, L., Rios, H., & Ehler-James, J. (2010). The mobile survey landscape - Today and

Tomorrow. Impacts of mobile devices usage on current and future market research practices.

Retrieved from http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

comScore (2012) 2012 Mobile Future in Focus. Retrieved from

http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Fo

cus

Couper, M. P. (2010). Visual design in online surveys: Learnings for the mobile world. Presented at

the Mobile Research Conference 2010, London. Retrieved from

http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf

Cunningham, JA et al. (2013) “Use of mobile devices to answer online surveys: implications for

research,” BMC Research Notes. Vol 6:258 available online: http://www.biomedcentral.com/1756-

0500/6/258

Duan, Shu, Justin Bailey, and Michael Link. 2011. “Data Collection Method Innovation: A

Crowdsourcing Application to Collect Factual Information in Africa”. Paper presented at the Midwest

Association for Public Opinion Research Conference, Chicago, IL.


Fuchs, Mark, 2008. “Mobile Web Surveys: A Preliminary Discussion of Methodological Implications”. In F.

Conrad & M. Schober (Eds.), Envisioning the Survey Interview of the Future (pp. 77-94). Hoboken, NJ: Wiley.

Graham, P, and C. Cobb (2013). “Comparison of Instantaneous Mobile Time Use Data Collection Methods to

Traditional Time Diary Methods.” Paper presented at the Annual Conference of the American Association for

Public Opinion Research, Boston, MA.

Gregoski, Mathew, Martina Mueller, Alexey Vertegel, et al. (2012). “Development and Validation of a

Smartphone Heart Rate Acquisition Application for Health Promotion and Wellness Telehealth Applications.”

International Journal of Telemedicine and Applications, doi:10.1155/2012/696324.

http://www.hindawi.com/journals/ijta/2012/696324/

Kennedy, C. (2007), Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys,

Public Opinion Quarterly, Vol 71 / Num 5: 750-771.

Koch, P-P. (2010) Smartphone Browser Landscape, A List Apart Article No. 320, available at

http://www.alistapart.com/articles/smartphone-browser-landscape/ (accessed on 10/30/2011)

Kumar, A. (2012) “Pilot study tests wearable sensors and smartphone-based data collection for diabetes,”

iMedicalApps, accessed at http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone-

diabetes/

Kuntsche, Emmanuel and Benjamin Robert. 2009. “Short Message Service (SMS) Technology in Alcohol

Research – A Feasability Study.” Alcohol &Alcoholism 44: 423-428.

Lai, Jennie W., Lorelle Vanno, Michael Link, Jennie Pearson, Hala Makowska, Karen Benezra, Mark Green.

2010. “Life360: Usability of Mobile Devices for Time Use Surveys” Survey Practice, February:

www.surveypractice.org

9

8


http://fmx.sagepub.com/content/early/recent


http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Focus

http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf

http://www.biomedcentral.com/1756-0500/6/258

http://www.hindawi.com/journals/ijta/2012/696324/

http://www.alistapart.com/articles/smartphone-browser-landscape/

http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone-diabetes/




Lai, Jennie, Michael Link, and Lorelle Vanno. 2011. “Like? The Effect of Social Networking in the Digital Age

of Mobile Application Research.” Paper presented at the Midwest Association for Public Opinion Research

Conference, Chicago, IL.

Maxl, E., Döring, N., & Wallisch, A. (Eds.). (2009). Mobile market research. Köln: Herbert Von Halem Verlag.

Nitsche, P, et al. (2012) “A strategy on how to utilize smartphones for automatically reconstructing trips in

travel surveys,” Procedia-Social and Behavioral Sciences, Vol. 48, pp. 1033-1046.

Olson, K, and J. Wagner (2013). “A Field Experiment Using GPS devices to measure Interviewer Travel

Behavior.” Paper presented at the annual conference of the American Association for Public Opinion

Research, Boston (May).

Paudela, D., M. Ahmedb, A. Pradhanc, R. Lal Dangold (2013) “Successful use of tablet personal computers

and wireless technologies for the 2011 Nepal Demographic and Health Survey,” Glob Health Sci Pract August

12, 2013 vol. 1 no. 2 p. 277-284; http://www.ghspjournal.org/content/1/2/277.full.pdf+html

Perkins,J, R. Granger, and E. Saleska. 2009. “Data Security Considerations When Using Social Networking

Wesites for Locating and Contacting Sample Members.” Presented at International Field Directors &

Technology Conference.

Peytchev, Andy and Craig A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other

Modes and Unique Considerations,” Social Science Computer Review 28: 319-335.

Pew Research Center. 2011. “How People Use Tablets and What It Means for the Future of News.” Report.

Available at: www.pewresearch.org/pubs/2119/tablet-news

Scagnelli, Jeff, Justin Bailey, Michael Link, Hala Moakowska, and Karen Benezra (2012). “On the Run: In the

Moment Smartphone Data Collection.” Paper presented at the annual conference of the American Association

for Public Opinion Research (Orlando, FL).



Schober, Michael, Frederick Conrad, Christopher Antoun, Alison Bowers, Andrew Hupp, Huiying Yan (2013).

“Conversational Interaction and Survey Data Quality in SMS Text Interviews.” Paper presented at the Annual

Conference of the American Association for Public Opinion Research, Boston, MA.

Smith, Aaron. 2011. “Smartphone Adoption and Usage.” Report available at:

www.pewinternet.org/reports/2011/Samartphones.aspx

Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field

Methods, 19, 59–75.

van Heerden, A. Shane A. Norris, Stephen M. Tollman, and Linda M. Richter, “Collecting Health Research

Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-pen Data Collection,” in press,

Field Methods, http://fmx.sagepub.com/content/early/recent

Vanno, Lorelle, Jennie Lai, and Michael Link. 2011. “Want to Collect Behavioral Data in an Engaging Way?

There’s an App for That!” Paper presented at the Southern Association for Public Opinion Research

Conference, Raleigh, NC.

Zahariev, M., Ferneyhough, C., Ryan, C., & Bishop, S. (2009). Best practices in mobile research. Presented at

the ESOMAR Online Research, Chicago, IL: ESOMAR.

Zickuhr, K. (2011). Generations and their gadgets. Pew Internet & the American Life Project. Retrieved from

http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx

Zuwallack, R. (2009) Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned,

Field Methods, Vol. 21, No. 4, pp. 388–406

Zichermann and Cunningham (2011) Gamification by Design: Implementing Game Mechanics in Web and

Mobile Apps available at: http://www.amazon.com/Gamification-Design-Implementing-Mechanics-

Mobile/dp/1449397670

Andbergen, P. A. (2012). Comparison of WiFi positioning on two mobile devices. Journal Of Location Based

Services, 6(1), 35-50. doi:10.1080/17489725.2011.630038

http://www.ghspjournal.org/content/1/2/277.full.pdf+html

http://www.pewresearch.org/pubs/2119/tablet-news

http://www.pewinternet.org/reports/2011/Samartphones.aspx

http://fmx.sagepub.com/content/early/recent

http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx

http://www.amazon.com/Gamification-Design-Implementing-Mechanics-Mobile/dp/1449397670

Documents

Smartphones and Smarter Surveys - Panjaaporpanjaapor.org/images/Buskirk_PANJAAPOR2014_Final.pdfA market research consultant goes to lunch with his clients – one is in the Marketing