Sudhir-Demand Estimation-Aggregate Data Workshop-Updated 2013

5/28/2018 Sudhir-Demand Estimation-Aggregate Data Workshop-Updated 2013

1/72

Demand Estimation Using Aggre

Static Discrete Choice Mo

K. SudhirYale School of Management

Quantitative Marketing and Structural Econometrics WDuke University

July 30 - August 1 2013


2/72


3/72

Why has BLP demand estibecome popular in marke


4/72

Motivation: Demand estimation using agg

Demand estimation is critical element of

analysisValue of demand estimation using aggreg

Marketers often only have access to aggregaEven if HH data available, these are not full

Two main challenges in using aggregate Heterogeneity: Marketers seek to differentia

appeal differentially to different segmentstocompetition and increase margins; need to a

Endogeneity: Researchers typically do not kn

all factors that firms offer and consumers vaa given time or market, firms account for thmarketing mixthis creates a potential end


5/72


6/72

What data do we need estimation?


7/72

Canonical aggregate market level data

Aggregate Market Data

Longitudinal: one market/store across time 1995/Sudhir Mkt Sci 2001/Chintagunta Mk

Cross-sections: multiple markets/stores (e.gSudhir 2011)

Panel: multiple markets/stores across time (Chintagunta, Singh and Dube QME 2003)

Typical variables used in estimation

Aggregate quantity

Prices/attributes/instruments

Definition of market size

Distribution of demographics (sometimes)


8/72

Typical Data Structure

Product Quantity Price Attributes Instr

Market/Time 1

Market/Time 2Product Quantity Price Attributes Instr

Market size (M) assumption needed to gej

jtt

qs

0

1

1

J

t j

j

s s


9/72

Notation

Markets or Periods:

ProductConsumer makes choice in

Indirect Utility Function:

observed product characteristics

unobserved (to researcher) product

price

observable consumer demograph

unobserved consumer attributes Indirect Utility Function:

1, ,t T

= =0 1 0 with the ou, , , ,j J j 0 1{ , , , }j Ji

x n( , , , , jt jt jt it

U x p D

jtx

xt

jtp

nit

itD

ab = + i iijt jt

x pu


10/72

Notation


where

Convenient to split the indirect utility int

n

n

a a

b b

qq

= + P + S = P S

2

1

( , )

HeterogeneityMean

i

i

i

i

i

D

d x q m = + 1

( , , ; ) ( , , ,

Mean HH Deviations f

ijt jt jt jt jt jt iu x p x p D

d b a x m= + + = , and ( , jt jt jt jt ijt jt

x p p x

a eb x= ++ +i iijt jt jt jt ijtx pu


11/72

Key Challenges for Estimation with Mark

Heterogeneity:

Recovering heterogeneity parameters witdata

Not an issue with consumer level data, bheterogeneity in choices across consumer

Endogeneity: is potentially correlated with price (an

marketing mix variables)

Contrary to earlier conventional wisdom

issue with even consumer level data

b a( , )i i

x( , )jt jtCorr px

t


12/72

Motivation for Addressing Endogeneity

Trajtenberg (JPE 1989)

famous analysis ofBody CT scannersusing nested logitmodel

Positive coefficient onpriceupward slopingdemand curve

Attributed to omitted

quality variables


13/72

Special Cases: Homogeneouand Nested Logit


14/72

Homogeneous Logit Notation


where

Convenient to split the indirect utility int

b a x e= + + +ijt jt i i jt jt ijt u x p

n

n

a a

b b

qq

= + P + S = P S

2

1

( , )

HeterogeneityAverage

i

i

i

i

i

D

d x q m = + 1

( , , ; ) ( , , ,




x p p x


15/72

Homogenous Logit with Aggregate Data

If we have market share data,

Normali

With homogeneous logit, we can invert smean utility

ts

d

d=

=

0

exp{ }

exp{ }

jt

jt J

ktk

s

d b a x - = = + +0

ln( ) ln( )

: Datat t jt jt jt jt

jt

s s x p

y

d=

=

0

0

1

exp{ }t J

ktk

s

d=0 exp{ }jt

jtt

s

s

d( )jt


16/72

Homogenous Logit with Aggregate Data:

If no endogeneity, we can use OLS

Given endogeneity of price, one can instrand use 2SLS

d b a x - = = + +0ln( ) ln( ): Data

jt t jt jt jt jt

jt

s s x py


17/72

Homogeneous Logit with aggregate data:

Alternatively, Berry (1994) suggests a GM

with a set of instruments ZStep 1: Compute

Let where

Step 2: GMM with moment conditions:

where

We have a nice analytical solution

where

Start with or to get in

Re-compute for new estim

d = -0

ln( ) ln( )t jt t

s s

Min ( )' ' ( )q

x q x qZWZ

x q d b a= - -( )t jt jt jt

x p q =

( ' ' ) ( ' ' )q d-= 1X ZWZ X X ZWZ

=[ ]X x p

=W I -= ' 1( )W Z Z

xx -= ' ' 1( ( )}W E Z Z

xx= ' ' ( ( W E Z Z

E


18/72

Why homogeneous logit not great as a de

Own Elasticity:

Cross Elasticity:

Bad Properties:

Own elasticities proportional to price, so share more expensive products tend to belastic!!

BMW328 will be more price elastic than For

Cross-elasticity of productj, w.r.t. price

dependent only on product ks price and BMW328 and Toyota Corolla will have same

elasticity with respect to Honda Civic!!

h a = = -

(1 )j jj j j

j j

s sp s

p p

h a = =

j j

jk k k

k k

s sp s

p p


19/72

Nested Logit with Aggregate Data: Apply

Nested logit provides more flexible elastic

Where proxies for intra-grouppreferences

Even if no price endogeneity,

we cannot avoid instrumentsAdditional moments are

needed to estimate

The GMM approach will still work, as lon

two or more instruments to create enougrestrictionsone each for

d b a r- = = + + 0 ln(ln( ) ln( )jt t jt jt j t s s x sp

anda r


20/72

The Canonical BLP RanCoefficients model


21/72

The Canonical BLP Random Coefficients


where

Split the indirect utility into two parts

Analytical inversion of no longer feasib


n

n

a a

b b

qq

= + P + S = P S

2

1

( , )


i

i

i

i

i

D

d x q m = + 1

( , , ; ) ( , , ,




x p p x

djt


22/72

Elasticities with heterogeneity better de

Own Elasticity:

Cross Elasticity:

Good Properties

Higher priced products more likely purch

customerscan have lower elasticitiesCross elasticities vary across productsp

Honda Civic induces more switching fromthan from BMW 328

h a = = -

(1 jj jj i ij ijj j j

ps ss s

p p s

h a = = - j j kjk i ij ik

k k j

s s ps s

p p s

n

d m n qd q

d m n q=

+=

+

2

2

,2

0

exp{ ( , ; )}( , )

exp{ ( , ; )}i i

jt ijt i i

jt t J

Dkt ikt i i

k

Ds

D


23/72

Logit vs RC logit: the value of heterogene

With logit, outside

good captures alleffects of priceincrease due to IIA

With RC logit, IIA

problem reducedExpensive cars

have lesssubstitution to

outside good


24/72

Estimation using market level data: BLP

1. Draw (and if needed) for a set of (say

consumers. Compute initial based on homoge2. Predicted shares

For given compute the HH deviations from me

For given mean utility ( ) & ,compute predicte

3. Contraction Mapping : Given nonlinear paramsearch for such that

4. From , estimate the linear parameters usformula. Then form the GMM obj function

5. Minimize over with Steps 2-4 nested

q2s q

t, x , p 2( ; )j t td

q2

s q= t

, x , p 2( ; )jt j t ts d

dt

in iD

m n q2

( , , , ; )jt jt i i

x p D

dt

dt

q1

( qQ

q2( )q2Q

jtd


25/72

An illustrative problem

Code and Data

Data provided in data.csvMatlab code: Agglogit.m (main program

AgglogitGMM.m (the function to be min

Problem Definition

J=2 (brands), T=76 (periods)

Variables: price, advertising, 3 quarterly d

Cost instruments: 3 for each brand

Heterogeneity only on the 2 brand interc

Covariance: s s

s s

s

S =

11 12

21 22

33

0

0

0 0

= 0

b

L b


26/72

Step 1: Simulating HH draws

%Fix these draws outside

the estimationw1=randn(NObs1,NCons);

w2=randn(NObs1,NCons);

wp=randn(NObs1,NCons);

%Convert to multivariate

draws within nonlinear

parameter estimation

aw1=b(1)*w1+b(2)*w2;

aw2=b(3)*w1;

aw2=b(3)*w1;

Cholesky

S

=L


27/72

Step 2: Computing market shares based o

For logit and nested logit, can use analytic

For random coefficients logit, integrate overheterogeneity by simulation

Where and , 1, , are draws fromthat are drawn and fixed over optimization

Simulation variance reduction (see Train Ch

Importance sampling: BLP oversamples on draw

purchases, relative to no purchase

Halton draws (100 Halton draws found better tdraws for mixed logit estimation; Train 2002)

d ns

d n=

=

+ P +=

+ P +

10

exp{ ( )( )}1

exp{ ( )( )}

NS

jt jt jt i i

jt Ji

kt kt kt i i

k

p x D

NSp x D


28/72

Step 3: Contraction Mapping to get mean

For logit and nested logit, you can get m

analytically

For random coefficients, logit, you need mapping, where you iterate till convergen(i.e.,

< Tolerance)

d d s d q+

= + - x , p1

2ln( ) ln( ( , ; h h ht t t j t t t NS s F,

d d b a - = - = + +0 0

ln( ) ln( )jt t jt t jt jt

s s x p

d d b a r- = - = + +0 0

ln( ) ln ( ) jt t jt t j t jt

s s x p


29/72

Steps 2&3:Market Shares and Contractiowhile (Err >= Tol)

de=de1;

sh=zeros(NObs1,1);psh=zeros(NObs1,1);

%Integrating over consumer heterogeneity

for i=1:1:NCons;

psh=exp(aw(:,i)+awp(:,i)+de); psh=reshape(psh',2

spsh=sum(psh')';

psh(:,1)=psh(:,1)./(1+spsh); psh(:,2)=psh(:,2).sh=sh+reshape(psh',NObs1,1);

end;

%Predicted Share

sh=sh/NCons;

%Adjust delta_jt

de1=de+log(s)-log(sh);

Err=max(abs(de1-de));

end;

delta=de1;


30/72


31/72

Step 4: Linear parameters and GMM obje

% Analytically estimating linear p

blin=inv(xlin'*z*W*z'*xlin)*(xlin'*z*W*

% GMM Objective function over nonlinear

er=delta-xlin*blin;

f=er'*z*W*z'*er;


32/72

Step 5: Optimizing over

Use a nonlinear optimizer to minimize th

objective function in Step 4The main file with data setup, homogene

homogeneous logit IV, and calling the nooptimizer are in file AggLogit.m

The GMM objective function nesting stefile AggLogitGMM.m

% Calling the optimizer with appropriate options

[b, fval,exitflag,output,grad,hessian] = fminunc('Agglog

Standard errors should be computed by sformula (Hansen 1982)


33/72

Summary

Why is BLP popular?

Handles aggregate data, heterogeneity an

Reviewed estimation algorithms

Homogenous and nested logit reduces toand can be estimated using an analytical

Random coefficients logit requires a nest

Reviewed an illustrative coding example


34/72

Summary: Key elements in programming

BLP illustrative example code:

Simulation to integrate over random coedistribution

Drawing from a multivariate distribution

Contraction mapping

Linearization of the mean utility to facilitateGeneralized Method of Moments

We numerically optimize only over the nonliwhile estimating the linear parameters affectthrough an analytical formula (as with homo


35/72

Session 2


36/72

Session 2: Agenda

Contrasting the contraction mapping alg

the MPEC approach Instruments

Identification

Improving identification and precision

Adding micro moments

Adding supply moments


37/72

Recall: The BLP Random Coefficients Lo

Indirect Utility Function: (Ignore income

where

Split the indirect utility into two parts



n

n

a a

b b

qq

= + P + S = P S

2

1

( , )i

i

i

i

i

D

Mean HH Deviations from

d x q m = + 1

( , , ; ) ( , , , ijt jt jt jt jt jt i

u x p x p D


x p p x


38/72


39/72

Problem with BLP Contraction Mapping

BLP: Nesting a Contraction mapping for

Problems:

Can be slow: For each trial of , we havecontraction mapping to obtain . This cif we have poor trial values of

Unstable if the tolerance levels used in thhigh (suggested 10-12)

An alternative approach is to use the MPthat avoids the contraction mapping

Min ' '( ) ( )q

x q x qZWZ

dt


40/72

MPEC Approach (Dube, Fox and Su, Ect

MPEC: Mathematical Programming with

ConstraintsReduce to a constrained nonlinear program

You have to search over both

With Jbrands and Tperiods(markets),JT

But no contraction mapping for each trial of

Nonlinear optimizers can do this effectively foconvergence can be tricky as JTbecomes larg

Min

subject to:

' '

,

( , , )

q xx x

s x q =, x , pj ns

ZWZ

F S


41/72

Contrast MPEC with BLP Contraction M

MPEC (Dube, Fox and Su 2011)

BLP: Nesting a contraction mapping for Min ' '( ) ( )

qx q x qZWZ

Min

subject to:

' '

,

( , )

q xx x

s x =, x , pj ns j

ZWZ

F S


42/72

Choosing Instruments


43/72

Choosing instruments

The BLP (and MPEC) estimation proced

on instruments Zthat satisfy the momen

IVs are needed for:

Moment conditions to identify (hetero

Recall nested logit needed instruments even endogenous

Correcting for price (and other marketingendogeneity

IV should be correlated with price but not w

( | )x =0jtE Z

q2


44/72


45/72

Common Instruments (2): Cost Shifters

Characteristics entering cost, but not dem

Generally hard to find

BLP use scale economies argument to usproduction as a cost instrument

Input factor prices

Affects costs and thus price, but not dire

Often used to explain price differentials aoften does not vary across brands (e.g., m

If we know production is in different states o

can get brand specific variation in factor cosChintagunta and Kadiyali (Mkt Sci 2005) usJapanese factor costs for Kodak and Fuji res


46/72

Common Instruments (3): Prices in other

Prices of products in other markets (Nevo 2001

1996) If there are common cost shocks across markets

other markets can be a valid instrument

But how to justify no common demand shocks?advertising; seasonality)


47/72


48/72

Importance of IV Correction: BLP


49/72

Identification


50/72

Step back: suppose we have consumer ch

Step 1: Estimate by Simulated ML

Assuming iid double exponential for

Note, , i.e., heterogeneity is identifi

household choices for same - not availab Step 2: Estimate

identified based on cross market/time variat

orrection for endogeneity needed even with

d q2

( , )

q1

d b a x = + +

jt jt jt jtx p

n

d n

d n=

+ P +=

+ P +

0

exp{ ( ) }(

exp{ ( ) }i

jt jt jt i i

ijt J

kt kt kt i i k

p x DP dF

p x D

eijt

q1

nq = P S

2 ( , )

i

( )jt jtp x


51/72

Variation in aggregate data that allows id

Some of the identification is due to funct

But we also need enough variation in procharacteristics and prices and see shares response across different demographics fo

Across markets, variation in

demographics , choice sets (possibly)

Across time, variation in

choice sets (possibly), demographics (po

To help further in identification

add micro data

add supply model


52/72

Variation in some familiar papers

BLP (1995)

National market over time (10 years)

Demographics hardly changes, but choice(characteristics and prices) change

Identification due to changes in shares d

Nevo (2001)

Many local markets, over many weeks

Demographics different across markets, pcharacteristics virtually identical, except

Identification comes from changes in shachoice sets and across demographics


53/72

Caveats about cross-sectional variation ac

Selection Problem: Is affected by mark

characteristics? E.g., less fresh vegetables sold at lower prices i

neighborhoods

May need to model selection

Can distribution of be systematically differen

If richer cities have better cycling paths for bikeunobservable), distribution of random coefficiencharacteristics may differ across markets (by inc

allowing for heterogeneity in distributions acrossnecessary.

Caution: identification of heterogeneity is toaggregate data, so we should not get carriedemanding more complexity in modeling

ni


54/72

Two other ways to improve identification

Add micro data based moments

Add supply moments


55/72

Identification:Adding Micro Momen


56/72

Aggregate data may be augmented with

Consumer Level Data

panel of consumer choice data (Chintagunta2005)

cross-sections of consumer choicesecond choice data on cars (BLP, JPE 2004)

Survey of consideration sets (theater, dvd) (Lua2006)

Segment Summaries

Quantity/share by demographic groups (Pet

Average demographics of purchasers of goo2002)

Consideration set size distributions (AlbuqueSci 2009)


57/72

Augmenting market data with micro data

Examples of micro-moments added by Pe

Set 1: Price Sensitivity

Set 2: Choice and demographics (Family

1

1

2

purchases new vehicle|{



[{

[{

[{

i

i

i

E i y y

E i y y

E i y y

| purchases a minivan

| purchases a station wagon

| purchases a SUV| purchases a fullsize van

[{ }]

[{ }

[{ }][{ }]

i

i

i

i

E fs i

E fs i

E fs i E fs i


58/72

Importance of micro data (Petrin 2002)


59/72


60/72

Identification:Adding Supply Momen


61/72


62/72

Adding a Supply Equation (contd)

Construct the supply errors as

One can create and stack supply momensupply errors with cost based instrument1995; Sudhir 2001)

...

... ... ... ... ... . * ...

...

t t

t Jt

Jt Jt

t Jt

ds ds

dp dpt t t

t

ds ds

Jt Jt Jtdp dp

c p s

O

c p s

- = -

1 1

1

1

1

1 1 1

Margin ( )mjt

jt jt jt jt

jt

p m wW

c

w- = +

( )jt jt jt jt

p m Ww q w= - -


63/72

Adding a supply equation (contd)

Supply errors:

Supply moments based on cost side instr

Stack the supply moments over the dem

Since there is a pricing equation for each

in effect, we double the number of obser

course correlation between equations),at the expense of estimating few more co

This helps improve the precision of estim

( )jt jt jt jt

p m Ww q w= - -

( | )w =0jt cE Z

( ) |( , ) |

x qw q w

=

0jt

jt c

ZEZ


64/72

Where to modify earlier code for supply ewhile (Err >= Tol)

de=de1;

sh=zeros(NObs1,1);

psh=zeros(NObs1,1);

%Integrating over consumer heterogeneity

for i=1:1:NCons;

psh=exp(aw(:,i)+awp(:,i)+de); psh=reshape(psh',2,NObs)

spsh=sum(psh')';

psh(:,1)=psh(:,1)./(1+spsh); psh(:,2)=psh(:,2)./(1+spssh=sh+reshape(psh',NObs1,1);

end;

%Predicted Share

sh=sh/NCons;

%Adjust delta_jt

de1=de+log(s)-log(sh);

Err=max(abs(de1-de));end;

delta=de1;

1. Compute owelasticity (sformula) foralong with s

2. Use these tomargins alo


65/72

Recall: Elasticities with heterogeneity

Own Elasticity:

Cross Elasticity:

h a = = -

(1 jj jj i ij j j j

ps ss s

p p s

h a = = -

j j kjk i ij ikk k j

s s ps s

p p s


66/72

Where to modify earlier code for supply e

% Analytically estimating linear p

blin=inv(xlin'*z*W*z'*xlin)*(xlin'*z*W*

% GMM Objective function over nonlinear

er=delta-xlin*blin;

f=er'*z*W*z'*er;

3. Estimate thparameters4. Construct t5. Stack the s

with appropmatrix in coGMM objec


67/72

Exercises

Estimate the model with supply side mom

Compute the standard errors for the estiside parameters

See Appendix to these slides on computierrors.


68/72

Summary

Session 1:

Why is BLP so popular in marketing?Handles heterogeneity and endogeneity in estimating dema

available aggregate data

The BLP algorithm and illustrative code

Simulation based integration, contraction mapping for meaformula for linear parameters, numerical optimization for th

Session 2 MPEC versus BLP Contraction mapping (Nested Fixed P

Instruments

Identification

Improving precision through

Micro data based moments Supply moments


69/72


70/72


71/72

8/ 2/ 13 7: 15 AM C: \ User s\ sk389\ Dr opbox\ Duke\ Aggl ogi t GMM. m 1 of

% Sampl e code to i l l ust r at e est i mat i on of BLP r andom coef f i ent s model

% Wr i t t en by K. Sudhi r , Yal e SOM

% For Quant i t at i ve Market i ng & St r uctur al Economet r i cs Workshop @Duke- 2013

% Thi s i s code f or t he GMM obj ect i ve f unct i on t hat needs t o mi ni mi zed

f uncti on f =Aggl ogi t GMM( b)

gl obal w1 w2 wp x y z NCons NObs NObs1 xl i n bl i n W z s bl i n l og_s_s0;

%St ep 1: Mul t i pl yi ng f i xed St andard Normal dr aws by l ower t r i angul ar Chol esky mat r i x

% parameters t o get t he mul t i var i ate heterogenei t y dr aws on i nt ercept s and

% pr i ces

aw=w1;

awp=wp;

aw1=b( 1) *w1+b( 2) *w2;

aw2=b( 3) *w1;

%St ep 2: Const r uct i ng the nonl i near part of t he share equat i on ( mu)

% usi ng t he heterogenei t y dr aws on i nt ercept s and pr i ce coef f

f or i =1: 1: si ze( w1, 2) ;

aw( : , i ) =aw1( : , i ) . *x( : , 1) +aw2( : , i ) . *x( : , 2) ;

awp( : , i ) =b( 4) *x( : , 3) . *wp( : , i ) ;

end;

del t a=l og_s_ s0;

Er r =100;

Tol =1e- 12;

de1=del t a;

% St ep 3: Cont r act i on Mappi ng t o get t he del t a_j t unt i l Tol er ance l evel met

whi l e ( Er r >= Tol )

de=de1;

sh=zer os( NObs1, 1) ;

psh=zer os( NObs1, 1) ;

%Obtai ni ng t he predi ct ed shar es based on model

f or i =1: 1: NCons;

psh=exp( aw( : , i ) +awp( : , i ) +de) ;

psh=r eshape( psh' , 2, NObs) ' ;

spsh=sum( psh' ) ' ;

psh( : , 1) =psh( : , 1) . / ( 1+spsh) ;

psh( : , 2) =psh( : , 2) . / ( 1+spsh) ;

sh=sh+r eshape( psh' , NObs1, 1) ;

end;

sh=sh/ NCons;

%Updat i ng del t a_j t based on di f f erence between actual shar e and

%predi ct ed shares

de1=de+l og( s) - l og( sh) ;

Err =max(abs( de1- de) ) ;

end;

del t a=de1;

% St ep 4: Get t i ng t he l i near par amet er s and set t i ng up t he obj ect i ve f n

bl i n=i nv( xl i n' *z*W*z' *xl i n) *( xl i n' *z*W*z' *del t a) ;

ksi =del t a- xl i n*bl i n;

% The GMM obj ect i ve f unct i on t hat wi l l be opt i mi zed over t o get

% nonl i near parameters

f =ksi ' *z*W*z' *ksi ;


72/72

Documents

Sudhir-Demand Estimation-Aggregate Data Workshop-Updated 2013