Jeffrey henning behavioural economics - 2012

A Presenta*on from The NewMR Behaviour Economics Event

19 April 2012

Be#er Surveys through Behavioral Economics Jeffrey Henning, Affinnova

Event sponsored by Greenbook All copyright owned by The Future Place and the presenters of the material

For more informa>on about Greenbook visit www.greenbookblog.org For more informa>on about NewMR events visit newmr.org

Speaker: Jeffrey Henning, Affinnova, US NewMR Behavioural Economics Event, 19 April 2012, Session 3

Better Surveys through Behavioral Economics

Jeffrey Henning Affinnova


Agenda

Surveys & BE

Framing Issues

Research on Research

Some Sugges>ons


Classical Surveys


Significant Overlap

Good Survey

Methodology

Good Behavioral Economics


Surveys & BE are Hard

•  Even Dan Ariely gets it wrong

•  Dan Ariely’s research uses a 101-point scale

•  For unipolar scales, 5-point, fully labeled scales are the most reliable (Krosnick & Tahk)


3 Key BE Themes

Framing Heuristics Market Inefficiencies


Framing

Q: Name all of your favorite foods.

Q: You have an opportunity to go to the supermarket with an unlimited budget and buy all of your favorite foods. The catch is, you only have two minutes.

Source: Jon Puleston, GMI


They’ll Answer Anything


Why That Wine?


Trust Respondents’ Choices, Not Reasons


Predicting Share for Frozen Dinners

Top 4 selling SKUs

Top selling varieties (based on distribution

and facings)

Price (EDP, promotional price and activity)

Logo


Monadic vs. Discrete Choice

Predicted Market Share

Likert Scale Scores Discrete Choice Data


Frozen Dinners (73% category coverage)


Correlation Analysis

Monadic Correlation .76 DC Correlation .94

Boston Market

17%; 8.08

19%; 29.71

43%; 113.96

Amy's

HC Café Steamers HC Complete Selections

Hungry Man

Marie Callender's

Stouffers

LC Café Cuisine

LC Comfort Cuisine

LC Simple Favorites

LC Spa Cuisine

Smart Ones

Smart Ones Bistro

29%; 68.06

33%; 42.1134%; 47.32

31%; 31.6128%; 30.53

30%; 23.7133%; 22.99

26%; 16.09

36%; 65.46

33%; 54.46

0

20

40

60

80

100

120

10% 15% 20% 25% 30% 35% 40% 45% 50%

46; 8.08

203; 113.96

Amy's

Boston Market

HC Café Steamers

HC Complete Selections

Hungry Man

Marie Callender's

Stouffers

LC Café Cuisine

LC Comfort Cuisine

LC Simple Favorites

LC Spa Cuisine

Smart Ones

Smart Ones Bistro93; 16.09

121; 47.32

147; 54.46

165; 65.46135; 68.06

108; 22.9988; 23.71

92; 29.71 109; 31.61

111; 42.11

107; 30.53

0

20

40

60

80

100

120

40 60 80 100 120 140 160 180 200 220

Discrete Choice Index of Linear Utilities

Volum

e Sale

s (in

millio

ns)

Monadic Weighted T2Box Purchase Intent

Volum

e Sale

s (in

millio

ns)


Marie Callender's

46; 17%

165; 36%

203; 43%

Amy's

Boston Market

HC Café Steamers

HC Complete

Selections

Hungry Man

Stouffers

LC Café CuisineLC Comfort Cuisine

LC Simple FavoritesLC Spa Cuisine

Smart OnesSmart Ones Bistro

147; 33%

111; 33%108; 33%

109; 31%

121; 34%

107; 28% 135; 29%

88; 30%

93; 26%

92; 19%

10%

15%

20%

25%

30%

35%

40%

45%

50%

40 60 80 100 120 140 160 180 200 220

Pairwise Prediction

Correct Predictions

Pairings Monadic DC

Methods Agree 67 61 61 86% 91% 91%

Methods Disagree 11 1 10 14% 9% 91%

Total 78 62 71

79% 91%

Discrete Choice Index of Linear Utilities

Mona

dic W

eighte

d T2B

ox P

urch

ase I

ntent


Signal-to-Noise Analysis

Monadic DC WPI Raw Utility

Signal* StdDev across Pts. 5.6 46.3 Highest point 41 63

Lowest point 21 -59

Noise Standard Error 2.4 4.4 as % of signal range 12% 4%

Ratio Signal-to-Noise 2.3 10.5

0.19; 26%

0.05; 28%

0.05; 30%0.06; 32%

0.46; 35%

0.13; 32%0.02; 32% 0.32; 32%-0.15; 30%

0.25; 28%-0.07; 26%

0.63; 41%

-0.59; 21%

10%

15%

20%

25%

30%

35%

40%

45%

50%

-1.00 -0.80 -0.60 -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00

Discrete Choice Raw Linear Utilities

Mona

dic W

eighte

d T2B

ox P

urch

ase I

ntent

* For these purposes signal is defined as the dispersion across the concept scores which does not net out point estimation error


Monadic vs. Discrete Choice

.76 Correlation .94 79% Pairwise Prediction 91% 2.3 Signal-to-Noise 10.5

Predicted Market Share

Likert Scale Scores Discrete Choice Data


“None of the Above”: A Framing Problem

•  Overstatement is a recognized problem in behavioral economics, where “None” is interpreted as the “external good”

•  This can be improved by having consumers compare a thing to specific goods – they are usually less likely to buy the thing

•  If we offer multiple “None” options, each describing a different class of external goods from a related category, would this help?


More Research to Come

•  What – Expanding the number of categories reviewed – Modeling “none of the above”

•  Where – The Market Research Technology Event – ESOMAR Annual Congress


Let’s Not Overreact to BE

•  Behavioral Economics doesn’t invalidate the need for direct questioning…

•  …just points out the need for better questioning

•  Want proof? •  The firm with more behavioral data than

any other just added direct questioning



Some Guidelines for BE Surveys

Do • Frame questions to reflect

the market being studied • Let respondents choose • Analyze choice drivers • Derive importance

Don’t • Ask for reasons • Ask for importance • Ask for ratings if you can

ask for choices


Q & A

Jeffrey Henning Affinnova

Lenny Murphy Greenbook


Thank you

Jeffrey Henning http://www.affinnova.com

@jhenning [email protected]

+1 781 464 4750

A Presenta*on from The NewMR Behaviour Economics Event

19 April 2012

Be#er Surveys through Behavioral Economics Jeffrey Henning, Affinnova

Event sponsored by Greenbook All copyright owned by The Future Place and the presenters of the material

For more informa>on about Greenbook visit www.greenbookblog.org For more informa>on about NewMR events visit newmr.org

Marketing

Jeffrey henning behavioural economics - 2012