Recovering SKU-level Preferences and Price

Sensitivities from Market Share Models Estimated on Item Aggregates

David R. Bell, Andre Bonfrer and Pradeep K. Chintagunta1

August 7, 2003

1The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia PA 19104 (email:

davidb@wharton.upenn.edu); Singapore Management University, 469 Bukit Timah Road, Singapore 259756

(email: andrebonfrer@smu.edu.sg); and Graduate School of Business, University of Chicago, 1101 E 58th

Street, Chicago IL 60637 (email: pradeep.chintagunta@gsb.uchicago.edu), respectively. The authors would

like to thank the Wharton-SMU Research Center for generous support, and seminar participants at the

Marketing Science Conference for helpful comments and suggestions.

Recovering SKU-level Preferences and Price

Sensitivities from Market Share Models Estimated on Item Aggregates

Abstract

Market share models such as the logit and MCI have been used extensively to studythe effects of marketing activities on aggregate shares of brands. Many product cate-gories contain a large number of stock keeping units (SKUs) and consist of assortmentsthat change over time (due to the prevalence of introductions, deletions and stock-outs).To aid in decisions such as product deletions (for example), marketers are interested inmeasuring preferences and sensitivities to marketing activities at the SKU level. Directestimation of these parameters may be hampered by the large number of coefficientsto be estimated and by the possibility that few observations are available for certainSKUs. Fortunately, even though most product categories contain a large and variablenumber of individual alternatives, they can also be defined by a much smaller stableset of attributes (e.g., brand, size, flavor, function, etc.). Instead of estimating thepreference for each SKU, researchers have estimated the preferences of attributes thatcomprise the SKU and then computed the implied preference for the SKU.In this paper, we propose an alternative approach to obtaining SKU-level prefer-

ences. We distinguish an attribute-level model in which the unit of analysis is themarket share for an alternative created by aggregation (e.g., “Colgate toothpaste”)from a true SKU-level model and develop an analytical relationship between para-meters obtained from these two models. We show that the researcher can recoverSKU-level parameters via calculation from estimated attribute-level parameters, cir-cumventing the need for direct estimation of the more complex true SKU-level model.So in a category with 168 SKUs involving 10 distinct brands, we show that instead ofestimating 168 preference parameters (when we have an “outside” alternative in addi-tion to the 168 “inside” ones), we need to estimate only 10 preference parameters fromwhich we can compute the 168 parameters as long as share and marketing mix data areavailable at the SKU level. Our market share models are calibrated using ninety-eightweeks of data for 168 SKUs in the toothpaste category. Estimation on holdout samplesdemonstrates superior predictive performance compared with other available methods.Implications for the derivation of SKU-level elasticities and forecasts of market shareand price sensitivity for new items introduced to the category are discussed.

Key Words: Discrete Choice, Market Share Models, SKU

1

1 Introduction

A typical retail assortment for a given product category consists of several hundred items

(often referred to as Stock Keeping Units (SKUs) or Universal Product Codes (UPCs)). The

presence of so many items poses considerable challenges to modelers of market response and

to retail managers trying to optimize the product offerings. While it may be desirable or

important to study behaviors across the full set of alternatives, this would likely result in a

model that is non-parsimonious and computationally intensive. If the goal of the researcher is

to uncover true SKU-level parameters, then two options are available. The first is to estimate

a model with a complete set of fixed effects, one for each item. This option is difficult

to implement with aggregated data of the type usually seen in marketing studies, since

the modeler quickly loses degrees of freedom with this approach. Furthermore, additional

problems may be encountered when there exists considerable volatility in the underlying

choice set due to entry and exit of SKUs in the category (Bucklin and Gupta, 1999). A

less serious problem is that the computational requirements for such a non-parsimonious

model can be quite significant, especially if one is constructing multiple models over many

categories, or running simulations that require multiple estimation of these models.

The second option to estimating SKU-level market share response models is to use a

pure characteristics based approach of the type introduced to the panel data literature by

Fader and Hardie (1996) (hereafter FH). This method avoids the ad hoc aggregation of SKUs

into choice composites – an approach still prevalent in individual-level choice models that

populate the marketing literature. FH propose consumer utility for a particular item should

be represented as an additive combination of utilities for the attribute levels that define the

alternative, rather than as a single fixed effect. An important property of this model is that

the number of parameters required does not increase with the number of alternatives, but

rather only with the number of additional levels of underlying attributes. This procedure not

only produces superior model fits in sample, but also allows researchers to create forecasts

for new items (provided the individual elements that make up the new alternative are present

in the market).1

While this second approach has been implemented on individual panel data, Bucklin and

1Imagine that Tide currently has 16oz and 32oz sizes, but no 64oz offering. Provided some competitor

offers a 64oz size, it is possible to develop a forecast for the new product “Tide, 64oz”.

2

Gupta (1999) note that managers have a preference for market share (or “aggregate”) data

which is more readily available and thought to be more reliable (less subject to sampling

biases). Moreover, academic researchers in marketing are showing increasing levels of interest

in market share models of the type advanced in the economics literature (e.g., Nevo 2001).

The model by FH represents a pioneering effort at modeling an entire product assortment

based on identifiable (to the researcher) product characteristics. Ho and Chong (2003) extend

this idea and develop a model where consumers “reinforce” chosen and non-chosen options in

their patterns of item selection in a manner consistent with an experience-weighted attraction

theory of consumer learning (Camerer and Ho 1999). Both models have a good deal to offer

market share-based approaches, yet these ideas have not diffused into that literature. In

this paper, we seek to bridge that gap and develop a model that is grounded in theory

and in the data actually used by retailers. We go beyond these previous empirical studies

and provide formal analytic relationships between the parameters estimated from simple

“composite” demand models and the true SKU-level fixed effects which can be derived from

them, circumventing the need for direct estimation.

To gain an appreciation of the substantive nature of the task at hand, consider the

following data from the toothpaste category (used in our empirical application). Table 1

(Panel A) shows that there are 168 unique items in the category. Furthermeore, over a period

of almost two years of observation many items are added, dropped, or stocked out. Table 1

(Panel B) indicates that each unique item can be further described as a combination of levels

over the following five attributes: brand, flavor, form, function and size. A “full assortment”

reflecting SKUs across all possible combinations would yield 10 × 14 × 3 × 7 × 3 = 11, 760items.

––––––––––––––

[ Table 1 about here ]

––––––––––––––

How then, is the market structured according to the product characteristics? Table 1

generates several interesting empirical observations, some of which (to our knowledge) appear

to have been overlooked in the estimation of market share models, yet seem highly germane

to model building2

2While we focus on toothpaste for our empirical analysis, the general empirical patterns shown in Table

1 can also be seen in many other categories (e.g., cereal, soft drinks, yogurt, etc.).

3

• The choice set faced by consumers changes dramatically over time (Panel A).• There are many more possible product locations (11,760) than locations actually oc-cupied by products (168) (Panel B).

• There is heterogeneity across attributes in the extent to which the owners of brands“fill in” the product line. Sixty-five percent of brand-size combinations are occupiedsuggesting that manufacturers decide on average to offer most sizes, whereas onlytwenty-one percent of brand-flavor combinations are occupied, implying that SKUsare more likely to be “unique” on the dimension of flavor (Panel B).

The implications are immediate. A standard market share model estimated on the true

alternatives dictates a very large number of fixed effects and estimation on the correct unit

of analysis (the SKU) could be complicated by time-dependent variation in the composition

of the choice set. Moreover, the location of items in the attribute space is clearly non-

random, which implies there is something to be learned from studying consumer sensitivity

to changes in attribute levels.

––––––––––––––

[ Table 2 about here ]

––––––––––––––

Table 2 reports for each attribute a Herfindahl index obtained from attribute-level shares

and compares it to a “competitive” value of 1/n where n is the number of levels. Consistent

with the data in Table 1, flavor and function are attributes with greater degrees of concentra-

tion (suggesting these attributes offer more unique value for the brands) whereas form and

size have index values close to that for the benchmark competitive level (suggesting these

are parity attributes).

Overview and Contribution. We derive a market share model that takes the fundamen-

tal properties of the data shown in Tables 1 and 2 into account, and provide the following

contributions to the extant literature on development and estimation of market share mod-

els. First, we show how one can recover underlying SKU-level fixed effects from a much

more parsimonious share regression on attribute-level shares.3 For example, assume that

the researcher is provided with the fixed effects arising from a market share model specified

3The attribute-level model is generated by aggregation in which the unit of analysis is the market share

of a level of a particular attribute (e.g., brand, size, etc.). In a brand-level model, for example, one defines

4

on brand-level data. These fixed effects represent the mean utilities of, for example, the

Colgate, Crest, Arm & Hammer brands. We show how these parameter estimates can be

used to generate fixed effect coefficients for each of the individual items under the respective

brands, such as mint-flavored, gel-form Colgate 6OZ toothpaste with tartar control. This is

possible for any number of items (SKUs) and works for any attribute.

Second, marketing mix coefficients for an underlying item-level model can be recovered

from a second-stage regression of transformed shares on appropriately mean-centered prices.

Third, the attribute-level coefficients can be used to relate item-level response measures

(e.g., elasticities) to higher order attribute-level response measures.4 An important practical

benefit is that the retailer gains insight into which attributes show the greatest sensitivity

and therefore suggests ways in which the assortment could be changed. We are also able to

provide forecasts for new items that may be introduced to the assortment and outperform

current approaches on predictive performance. More significantly, we can also forecast the

price sensitivity of these items, based on their product characteristics. Thus retailers and

manufacturers can adopt this procedure to improve decisions in category assortment planning

and new product design.

The remainder of the paper is organized as follows. The next section briefly reviews

the relevant market share modeling literature and develops the specification and demand

equations for our approach. In particular, we explicate the analytical relationship between

an SKU-level model and an attribute-level model. Next, we provide a description of the data

used for the empirical application and further motivate our study. The empirical results are

presented in a separate section and the paper concludes with the discussion of the empirical

findings which includes a comparison with an alternative “pure characteristics” modeling

procedure.

market share for the Colgate brand by adding up the shares of all items that have this brand name (and

similarly for Crest, etc.). In a size-level model one computes share for all items under the 4OZ size, 6OZ

size, etc.4This provides some determination of the extent to which combinations of attributes produce interactive

effects. For example, the “baking soda” flavor applied to the brand “Arm and Hammer” may deliver market

share beyond that predicted by an additive model.

5

2 Background and Model

Increasingly, researchers in marketing are turning their attention to the analysis of market

share data obtained from retail settings. First — as noted earlier — retail managers are more

comfortable and familiar with market share data than with the panel level data often favored

by marketing academics (Bucklin and Gupta 1999). Store level data is also less subject

to sampling biases and especially useful for understanding “micro-marketing” and retail

tailoring (e.g., Montgomery 1997). Moreover, market share modeling has a long tradition

dating back to early work by Cooper and Nakanishi (1974; 1982) on the MCI model and

forecasting models of Brodie and deKluyver (1982).5

In many instances it may be reasonable to aggregate data to the level of brand (man-

ufacturer) and hold the set of brands constant over other dimensions of the dataset (e.g.,

time and or/markets), however if the goal is to better understand interrelationships among

items (e.g., for a product assortment decision) it is necessary to take a more disaggregate

approach. It is important not to be constrained methodologically by techniques that require

ex ante aggregation or constancy of this (aggregated) choice set over other dimensions. The

question of how items can be derived from attributes has been the subject of successful study

in the panel data literature (see FH 1996; Ho and Chong 2003) and this work provides a

starting point for our methods. FH stress that consumers choose items, not ad hoc brand-

level composites defined by researchers, and that retailers and manufacturers also require

understanding of item-level behavior. This is especially true in complex categories where

items are markedly different in their observed characteristics (e.g., flavor, form, function,

size, etc.). In the toothpaste category on which our model is calibrated, we observe consid-

erable variation in items on the shelf at any point in time, but no variation over time in the

number of brands which remains constant at ten.6 Figure 1 shows the time variation in the

5For a more detailed review see Leeflang, Wittink, Wedel and Naert (2000, p. 171-178).6Although variation in the number of levels may occur, it is relatively rare compared with the variations

in the items available in a choice set. One instance where this occurs is with the introduction of very new

products, involving a new level of a given attribute. For example the recent introduction of “Vanilla Coke”,

where the vanilla flavor does not exist in any other soda beverages.

6

number of items (previously summarized in Table 1).

––––––––––––––

[ Figure 1 about here ]

––––––––––––––

2.1 Model

We start with a basic market share model commonly found in the literature and describe the

relationship among fixed effect parameters estimated at different levels of aggregation. In

particular, we show how fixed effect parameter estimates for SKU j – the most disaggregate

unit of analysis – obtained from market share equations for all j = 1, . . . J items are related

to fixed effect estimates for attribute level l of attribute a estimated from market share

equations for the l = 1, . . . La levels of that attribute (where La is much smaller than J –

see Table 1). For ease of exposition we focus on the “brand” attribute as the higher level

composite.7 To continue this example, assume Brand A is composed of three unique items

of differing sizes (e.g. Small, Medium, Large), whereas Brand B is composed of only two

unique items of differing sizes (e.g. Small, Large). We first aim to show how the item-

level fixed effect for item j (e.g., β1 for j = 1) estimated from a five-equation system is

related to the brand-level fixed effect for Brand A (i.e., βA) estimated from a two-equation

system. We do this initially within the context of models that have fixed effects only (i.e.,

no covariates). For each model specification fixed effect parameters are identified through

use of data collected over some other dimension (e.g., time or market). In our empirical

application, observations for each item and composite are collected over time t = 1, . . . , T .

After first showing how to relate and recover fixed effects from models without covariates we

extend the analysis to the case of exogenous regressors. We again show how the fixed effects

and the true price (or marketing mix) coefficient that would arise from an item-level model

can be calculated without recourse to direct estimation, but by using parameter estimates

from the attribute-level model.

7The choice of attribute is immaterial. The researcher may choose any product attribute as the basis for

aggregation.

7

2.2 Fixed Effects for Items and Composites (Brands)

We begin by showing how the parameters from two models: (1) the disaggregate item-level

share model, and (2) the brand composite model are related when each model contains only

fixed effects. The goal is to show how item-level fixed effects may be recovered analytically

when one estimates only the attribute-level model. As noted earlier, we focus on brand as

the attribute over which aggregation takes place. This is for the purpose of exposition only

and we do so without loss of generality. Subsequently, we extend the analysis to the inclusion

of covariates, illustrating our approach using price as the variable of interest.

Fixed Effects: Attribute-Level Parameters from Item-Level Parameters

Following the standard approach in the literature we assume that the market share for item

j at time t can be expressed as

sjt =expµjt

1 + Jk expµkt

,(1)

where µjt is the item-specific utility relative to an outside good. In general, µjt can be a

function of both fixed effects for item j and time-dependent covariates and their associated

response parameters. Similarly, the share of the so-called outside good is given by

s0t =1

1 + Jk expµkt

.(2)

Parameter estimation is facilitated by a linearizing transformation (e.g., Besanko, Gupta

and Jain 1998)

log(sjt)− log(s0t) = µjt.(3)

Specifically, the parameters of the µjt component can then be estimated using OLS under

the assumption that the model also contains a structural error term jt that now takes on

the role of the error in the regression equation. Item-specific intercepts βj (excluding an

intercept for the outside good) can be estimated in a straightforward manner. Assume that

the market share for a particular brand b is simply the sum of the market shares for the first

8

two items. For brand b = 1 let

S1t = s1t + s2t(4)

=expµ1t + expµ2t1 + J

k expµkt.(5)

Clearly, the brand (attribute) share specification and the item share specification use the

same outside good value, which we can fix as

S0t = s0t =1

1 + Jk expµkt

.(6)

Applying the log transformation to the brand-level model of equation (5) we obtain the

brand-specific mean utility value for brand b (shared by items 1 and 2)

µbt = log(S1t)− log(S0t) = log(expµ1t + expµ2t).(7)

The key question of interest is: Can we recover the item-specific parameters from the brand

(attribute) share equations? Before addressing this however, let us first determine whether in

fact the reverse is possible: Can we recover the brand-specific parameters from the item-share

equations? The following calculation recovers the brand-specific intercepts for the attribute

share model using only the parameters and error terms from the item-specific model as inputs

(recall that in this example brand b is only available for items 1 and 2)

µbt = βb + bt = log(exp β1 exp 1t + exp β2 exp 2t)(8)

E[µbt] =1

T

T

t=1

βb + bt(9)

=1

T

T

t=1

log(expβ1 exp 1t + exp β2 exp 2t)(10)

= βb.

To confirm that this relation holds we obtain β1 and β2 and βb directly from OLS estimation

on the item-level and brand-level market share equations, respectively. We also compute

estimates of OLS residuals exp 1t and exp 2t from the item-level models. Using a simulated

dataset we computed these values for the right hand side of equation (10) and were able to

9

perfectly recover (to four decimal places) the brand-level intercepts βb estimated from the

brand share model, clearly demonstrating this link (details are given in Appendix A). Note

that this procedure explicitly requires both the parameter estimates βj and the residuals jt

to recover the brand-level parameters.

Fixed Effects: Item-Level Parameters from Attribute-Level Parameters

The next challenge is to recover item-specific intercepts via calculation, using only estimates

from the attribute-share model. For ease of exposition and without loss of generality we will

focus on the brand attribute and assume all Lb levels of this attribute are represented for

all periods of the data.8 Moreover, we note that the following information is available after

estimation of the attribute-share models

1. The estimated attribute-model intercepts for the brand levels, i.e., βb b = 1, . . . Lb.

2. The design matrix of the items (i.e., the description of each item in terms of a combi-nation of single unique levels for each attribute).

3. The error components of the data estimated from the attribute share models.

4. The outside good value.

5. The market share of each of the items, and the attribute share for each attribute level.

As we show below our method requires only 1 and 5 above in order to recover the item-level

fixed effects by calculation. Following our earlier analysis we have

expµbt = expµ1t + expµ2t.(11)

Let D = 1 + Jj=1 expµjt denote the denominator of the brand and item share models and

note that D can be rewritten as

D =expβb exp bt

Sbt.(12)

In general, exponentiating the mean utility for any product (whether the outside good, a

single item or a composite like a brand), and dividing it by the associated market share will

8This is much less stringent than requiring that all items are available in each and every period — something

that is clearly not true in practice (cf. Table 1 and Figure 1).

10

yield D. Next, using the definition of item share sjt given in equation (1) we recognize that

the product of the exponentiated (unobserved) item-level intercept and error component for

item j under brand b is equal to the product of D and the item-level share

exp βj exp jt = D · sjt.(13)

Taking the log of both sides we find that the item-level intercept is related to the brand-level

intercept as follows

βj + jt = βb + bt − log Sbt + log sjt.(14)

To obtain βj by calculation we take the average over t which yields

βj =1

T

T

t=1

[βb + bt − log Sbt + log sjt]

= βb +1

T

T

t=1

logsjtSbt.(15)

The last line of equation (15) is simply the sum of the estimated brand intercept and the

average log share of the item relative to the share of the brand. Because the denominator is

weakly greater than the numerator, the log sum will always be negative. Alternatively, we

could rewrite the expression as

βj = βb − 1

T

T

t=1

logSbtsjt.

Using simulated data the item intercepts calculated in this manner (see Appendix A) repro-

duce the directly estimated item intercepts almost exactly.9

2.3 Adding Covariate Effects

We now demonstrate how to recover covariate effects (e.g., price, feature and display). While

we focus on price for ease of exposition, our analysis is easily extended to any number of

9Calculated parameters are equal to estimated parameters to more than ten decimal places for all esti-

mates. We also tested the robustness of this relation given the presence of interactions among the attributes

(e.g. Colgate makes a better tartar control formulation than does Crest) and found that equation (15) is

still valid in this case.

11

covariates. The introduction of covariates complicates matters starting with recovery by

calculation of the item-level fixed effects βj, as the mean utilities for brand composite b and

item j now become

µbt = βb + γbPbt + bt and(16)

µjt = βj + γpjt + jt,(17)

where Pbt =1

Lb

Lb

i=1

sjtSbtpjt

is the item-share weighted price for the brand composite. First, we note how these changes

will affect the derivation that lead to equation (15) and the recovery of fixed-effects when

both the item-level and attribute-level models do not contain covariates. The parameters γ

and γb are the (true) item and brand-level price response parameters, respectively. That is,

γ is the parameter that would be obtained if one were to estimate an item-specific market

share model. We now ask the following questions. What is the relationship between βj and

βb when both models include price covariates? What is the relationship between γb and

γ? How do we recover the true γ from the estimate of the attribute-level counterpart γb?

Following the logic that lead to equation (12) we can still write, for item j

exp βj exp γpjt exp jt = D · sjt,(18)

however in this instance D is given by

D =exp βb exp γbPbt exp bt

Sbt.(19)

After taking logs and rearranging terms we find that

βj + γpjt + jt = βb + γbPbt + bt + log sjt − logSbt soβj = βb + γbPbt − γpjt − jt + log sjt − log Sbt.(20)

As before, the fixed effect values can be computed by taking averages over time

βj = βbestimated

+ γbestimated

¯Pbobserved

− γ

unknown

pj

observed

+1

Tlog sjt

observed

− 1Tlog Sbt

observed

,(21)

12

with the double bar indicating the average of the share-weighted price Pb taken over time

t = 1, . . . , T . We have been careful to distinguish what can be observed in the data, what can

be estimated and what quantities are unknown (and must be calculated). Unfortunately,

equation (21) clearly shows there is only one equation for each of the j items and two

unknowns (price effect and fixed effect).10 We will address this issue shortly, meanwhile we

note that the preceeding expression can be further refined as follows

βj = βb + γb¯Pb − γpj +

1

T

T

t=1

logsjtSbt.(22)

To understand how the identification problem caused by the introduction of regressors

complicates matters, consider the following example. Using simulated data we again esti-

mated the item-level and brand level-intercepts from the respective models, but this time

included the price variables. We found that calculating the item-level intercepts based on

the relationship in equation (15) derived earlier (for models that do not have covariates) and

ignoring the effect of price specified in equation (21) produces fixed effect estimates which are

consistently inaccurate. Figure 2 plots the estimated versus calculated item-level intercepts

resulting from this procedure.

––––––––––––––

[ Figure 2 about here ]

––––––––––––––

Clearly the difference between the points on the plot and the y = x line is due to the

inclusion of the price variable which renders the earlier derivation inappropriate. At the

same time equation (21) is not directly useful because the price and fixed effects cannot be

separately identified. Fortunately, we now demonstrate that both problems can be circum-

vented such that the true item-level fixed effects can be recovered from an appropriately

reformulated model. In addition, we show that a second-stage regression can be used to

recover the other parameter of interest – the unknown price effect, γ.

10It is also not possible to use information from other attribute equations (e.g., size) to identify the fixed

effect or price effect. This is because each attribute-level model yields equations of the form given in (21).

13

Fixed Effect Recovery: Model Reformulation With Mean-Centered Covariates

Continuing our discussion with price as the covariate, we reformulate the mean utilities of

the attribute and item-level models by including mean-centered prices rather than actual

prices to give

µbt = βb + γb(Pbt − ¯P b) + bt, and(23)

µjt = βj + γ(pjt − pj) + jt(24)

where βb and βj are defined here as the “true” fixed effects for each model. As indicated

above, mean-centering occurs at the appropriate level of aggregation – with respect to the

attribute-level average for each attribute level, and with respect to the item-level average for

the corresponding item. It is important to note that the covariate effect γb will be completely

unaffected by this change (e.g., Raudenbush and Bryk 2001). This can be seen explicitly by

observing that equation (23) can be rewritten as

µbt = βb − γb¯P b

β1b

+γbPbt + bt

where β1b is simply the fixed effect estimate that results when the covariate has not been

mean-centered. The difference between β1b and the “true” βb is simply equal to the price

estimate for the attribute multiplied by the average price for that level (and analogously for

the item-level model). It is still the case that: (1) the item-level and attribute-level price

effects will not be equal (i.e., γ = γb) and (2) the fixed effects βj will need to be calculated

from estimated values from the attribute-level models {βb, γb} and average log market sharesand prices { ¯P b, pj}. Using equation (18) but with the mean-centered covariates we obtain

βj + γ(pjt − pj) + jt = βb + γb(Pbt − ¯P b) + bt + log sjt − logSbt.

Taking expectations with respect to time we find 1T

Tt=1 γb(Pbt − ¯P b) = γb0 = 0 and

1T

Tt=1 γ(pjt − pj) = 0 which implies that item-level intercepts can be calculated from

E[βj] = βb +1

T

T

t=1

logsjtSbt.(25)

14

The analytical form of equation (25) is identical to that obtained for models that do not

include fixed effects, except that the covariates need to be mean-centered. We tested this

relation with simulated data (see Appendix A) and were able to recover the item-level in-

tercepts from the brand-level intercepts, just as we did for the models without covariates.

In addition, the true item-level fixed effects can be recovered using any attribute that the

researcher desires to use as a basis for aggregation, and with any number of appropriately

mean-centered regressors in the model.

Covariate Effect Recovery: Second Stage Regression Analysis

So far we have shown how true item-level fixed effects can be recovered using a parsimonious

attribute-level model. Recovery of the covariate effect – γ in our example – is still com-

plicated by the fact that while mean-centering solved the identification problem for the βj

parameters, it introduced an indeterminancy in process: γ × 0 = γb × 0. In fact, it was theintroduction of this indeterminancy in part, that made recovery of the βj possible. It is also

the case that knowledge of γb (nor any γa for any attribute a alone or in combination) tells

us nothing by itself about the true price effect, γ.

We can however obtain an estimate of the true price effect without recourse to an item-

level model, through the following second stage regression. First, having estimated the

attribute-level model we calculate the item-level fixed βj effects using equation (25). Second,

we subtract these calculated item-specific fixed effects from log-transformed share informa-

tion that enters the item-level regression model. That is, we use item-level data for each

item j and time period t and form the following regression equation

(yjt − βj) = γ(pjt − pj) + jt where

(yjt − βj) = log(sjt)− log(s0t)− βb +1

T

T

t=1

logsjtSbt

or

yjt = γpmcjt + jt,(26)

where pmc is the mean-centered price. OLS estimation retrieves the same γ as that obtained

from a regression model based on the original item-level market shares.11

11This approach has implications for the standard errors and these are given in Appendix B.

15

2.4 Summary

In the preceding sections we have developed the analytical and empirical relationships that

exist between market share models estimated at different levels of aggregation. In particular,

we show how one can recover via calculation both the underlying item-specific fixed effects

and the response parameters for covariates in an item-level model using a very parsimonious

model estimated at a higher level of aggregation. The implications for researchers and

practitioners are powerful: There is no need to resort to arbitrary aggregation schemes,

disregard data, or focus only on categories with a relatively small number of alternatives.

Our formulation allows the researcher to be true to the properties of the underlying data,

and implement models in a very straightforward fashion using standard software.

In the next sections we describe our empirical analysis of the toothpaste category and

also develop unique insights afforded by our model into how consumers value attributes. In

particular, we show that attribute-level equations are useful not only as a methodological

shortcut, but also as a tool for delivering substantive insights into consumer sensitivity to

attribute-level changes.

3 Data

We analyze toothpaste sales data from a single store in the Stanford Market Basket Data-

base.12 It is well known that retailers use some categories to drive store choice decisions

(Chen, Hess, Wilcox and Zhang 1999) and that toothpaste is one category typically not as-

sociated with traffic building objectives (Dreze and Hoch 1998). In addition, the toothpaste

category contains a good deal of variety and large changes in the total number of items

available due to stock outs, product introductions and deletions (see Table 1 and Figure

1). Collectively, these features make it an ideal candidate category for calibrating our new

12These data have been used in a number of studies published in the marketing and economics lit-

eratures. See Bell and Lattin (1998) for a more detailed description of the entire dataset and visit

http://wrds.wharton.upenn.edu for additional details and documentation. The dataset contains both panel

and store level data for each product category.

16

approach. Our model analyzes the market shares of items and is therefore conditional upon

consumer decisions to first chose the store. Furthermore, they are conditional upon the

product offerings available in a particular week.

Raw Data

Data Cleaning. In assembling the data for the study, we use information from Euromonitor,

the IRI Marketing Fact Book and the Stanford Market Basket Database.13 The toothpaste

sales data come from a large supermarket retailer operating in Chicago and cover 104 weeks

(week 1 is the week beginning 6/1/1991 and week 104 is the week beginning 5/22/1993).

All prices are in September 1991 dollars as the original price series were deflated using the

Bureau of Labor Statistics’ Consumer Price Index. Sales are in ounces, and prices are price

per ounce of toothpaste.

The “outside good” (measured in ounces) is created as follows. The dataset contains

information on the number of panelists visiting our chosen store in a particular week, as well

as total sales for each item in the toothpaste category. The group of panelists in the store

represents a sample of customers and we assume that their purchase behavior represents

that of the entire population of shoppers visiting the store. We use panelist sample means

to estimate the total amount spent by an average customer (average market basket dollar

value), and divide total dollar sales for each week by this value to generate an estimate for

the total number of visitors that week. We follow the standard approaches (e.g. Nevo 2000)

to estimate the outside good and include any shopper who visited the store in that week as

part of the “market potential.” We further assume that if such a shopper decides to purchase

in the toothpaste category, they purchase an average-sized pack.

To determine the item set we start by selecting all 246 items that were sold in this store

at some point during the observation period. From this group we deleted 40 items for which

we had no attribute information, leaving 206 items.14 Another 35 items for which we had

13The full dataset, documentation and MATLAB code for estimation are available from the authors upon

request.14All but one of these items had less than 0.3 percent category share overall.

17

category information but were otherwise problematic – either because of very small share

or because they were other types of tooth cleanser (e.g., liquid cleansers or teeth whiteners)

– were removed. Their combined category share was less than two percent. Finally, we

removed the brand “Topol” which is a small share brand designed for smokers. The end

result is a choice set of 168 unique items that represent just under 92 percent of the entire

category sales for our store. Although the potential choice set is 168 items, on average only

95 of these are available on the shelf at any one time. This discrepancy speaks to the large

amount of diversity in the category in terms of rate of entry and exit of products, and the

prevalence of stock outs.

Attributes and Levels. Using the definitions provided by IRI, we identified the unique set

of attributes and levels that form what we will refer to as the attribute basis for the product

category. Each item is described completely by a combination of unique single-valued levels

of each attribute. There are four attributes (in addition to brand)

1. Flavor – the taste of the toothpaste (e.g., mint or bubble gum).

2. Form – texture and product delivery (e.g., translucent gel or more traditional opaquepaste).

3. Function – the purpose of the toothpaste (e.g., “anti-plaque” to protect the teethagainst plaque build-up, a hard substance that would otherwise require the consumerto visit a dentist to remove it).

4. Size – the pack size in ounces. While there are numerous levels of size, many of theseare essentially identical (e.g., 3.45oz versus 3.50oz). We pre-classify size into mutuallyexclusive and collectively exhaustive categories: small, medium, large, and extra large.

As noted earlier, the number of unique levels for each attribute is as follows: brand (10),

flavor (14), form (3), function (7), and size (4).

Within Brand and Attribute Share Distributions

As noted in the Introduction, full enumeration of the attribute basis leads to 11,760 possi-

bilities yet only 168 are observed in the dataset, suggesting that choice of product location

by manufacturers and retailers is non-random. A typical approach to modeling demand is

to use a market share function where the individual equations are aggregated items within

18

an attribute (usually brand). This approach is more readily justified if the brand shares for

each level of each attribute possess the same distribution as brand shares within the brand

attribute (see, e.g. Kamakura and Russell, 1989), yet may be problematic if this condition

does not hold.

When we analyze these distributions for our data we find multiple “gaps” in the attribute

grid representing no presence at that level by the corresponding brand. Some brands do not

compete at all at some attribute levels (approximately 40 percent of the attribute-brand

levels are not currently occupied) which suggests that aggregation over brand and simple

execution of the analysis at this level alone may be problematic for a category such as

this. Moreover, there are many discrepancies in the shares of brands across attribute levels.

For example, Crest toothpaste sells relatively more “gel” than “paste” form toothpaste and

seems to dominate many of the flavor levels to a greater extent than Crest’s brand share

would predict. In short, this argues that one should attempt to build a model that draws

on all of the data and not resort to trimming or aggregation of an arbritrary nature as an

end in and of itself.

4 Empirical Results

Our empirical analysis of the toothpaste category proceeds in two parts. First, we estimate

the attribute-share models for all five attributes (brand, flavor, form, function and size) and

report the model fits and parameter estimates for both the fixed effects and the price response

parameter. Second, we show how these estimates can be used to recover via calculation the

underlying item-level fixed effects and true price response parameter from the item-level

model. We defer an analysis of the substantive implications of our empirical findings to the

next section.

Attribute Share Models

Table 3 summarizes the findings from the five attribute share models. Recall that in accor-

dance with our analytical development, these models are estimated with an appropriately

19

mean-centered price. If this were not the case, we would be unable to recover the item-level

parameters (details are given below). The number of parameters and observations differs

across models in accordance with the number of levels, La contained in each attribute a.

The brand model, for example, contains 10 levels and therefore 980 observations (all models

are estimated over 98 weeks of data) with a total of 11 parameters (10 fixed effects and 1

price effect).

––––––––––––––

[ Table 3 about here ]

––––––––––––––

For ease of exposition we do not report all attribute-level fixed effects (these are available

upon request), but simply summarize them according to the minimum, maximum and median

values. The model fit is good in all cases, with adjusted R2 ranging from 89% for the flavor

model to 94% for the brand model, and all fixed effects are statistically significantly different

from zero. The price response parameter γa varies across attributes a = 1, . . . , A and provides

some indication of differential price sensitivity with form appearing to be the most price-

responsive attribute.

Calculation of Item-Level Effects

Table 4 presents a comparison of the estimated item-level coefficients (fixed effects and price

effect) from the item level model and the calculated item-level coefficients from the attribute-

level models shown in Table 3. For ease of illustration we present the results from the brand

attribute only (results for other attributes are identical). Several interesting observations

are immediately apparent

• Fixed Effect Recovery. The item-level fixed effects recovered via calculation are essen-tially identical to those obtained by direct estimation. We summarize the 168 item-levelintercepts using the minimum, maximum, median and mean values. The column enti-tled MSE reports the average squared difference between item-specific estimates fromthe OLS item-level model and those generated through the attribute model. It is clearthat regardless of whether one specifies a brand-share or form-share model it makes nodifference to the ability to recover the item-specific estimates.

• Price Effect Recovery. The attribute-level model recovers the price effect exactly.Recall that this calculation proceeds in two steps – calculated fixed effects are fed

20

into a second stage regression of log-differenced item shares less calculated fixed effectson mean-centered item-level prices. The researcher is free to specify any convenientattribute-share model and can be confident of obtaining the true price coefficient.

• Fit and Precision. The R2 column indicates a value of 0.56 for the item-level modelwhich is well below that obtained for the attribute-level models (see Table 3), suggest-ing considerably more unexplained variation in items. For the attribute-share modelthe value of 0.07 is for the second-stage regression used to obtain the price effect (i.e.,equation 26). The item-level model does however produce the smallest standard er-rors for fixed effects suggesting that our calculation approach that proceeds from theattribute-level estimates suffers from a loss of precision (see Appendix B). Nevertheless,no loss of precision is observed for the price effect.

––––––––––––––

[ Table 4 about here ]

––––––––––––––

Attribute-Level Price Elasticities

Our model allows computation of “elasticities” for attributes and provides insights into con-

sumer sensitivity to price changes for different product freatures. Table 5 reports elasticities

derived from estimates in Table 3, and under the assumption of a temporary price change.

––––––––––––––

[ Table 5 about here ]

––––––––––––––

All the price elasticities are larger (more negative) than -1. On average, the elasticities

increase across attributes as follows: size, brand, function, flavor and form. Consumers

appear relatively more willing to switch among say gel and paste than they do trade among

sizes and as one might expect, price elasticity increases as sizes get smaller. It interesting that

the “triple” function yields the lowest price elasticity (anti-cavity, tartar control and anti-

plaque are all more price-elastic) which is consistent with the notion that customers readily

substitute from products with individual functions to those with the bundled function.

21

5 Discussion and Conclusion

In recent years researchers and practitioners alike have devoted considerable time and re-

sources to the development and estimation of market share models. An important substantive

challenge in this literature is the identification of patterns of choice and competition that

allow one to draw inferences about market structure and market response. Unfortunately,

in many instances researchers are forced to make data pruning or aggregation decisions that

facilitate model estimation but are not necessarily true to the underlying structure of the

data. Drawing on insights from the panel data literature (FH 1996) we develop a modeling

approach that exploits analytical relationships between item-level parameters and composite

or attribute-level parameters obtained from models estimated at higher levels of aggregation.

We are able to recover the item-level parameters using a parsimonious specification for

the market shares and exploiting information generated by attribute-level analysis. The first

order of contribution is therefore methodological as this recovery procedure is implemented

without recourse to estimation of the underlying model. This new procedure also offers

the potential for substantive contributions. By examining the attribute-level results we can

develop additional insights into to demand response to changes in product characteristics.

Attribute-Level and Item-Level Price Elasticities

Russell and Bolton (1988) examine item-level price elasticities as a function of “sub-market”

price elasticities and item-level conditional market share elasticities. They show analytically

how given a proper definition of sub-markets one can derive a hierarchy of own and cross-

price elasticities. In our setup there exists a large number of potential sub-markets so our

approach to uncovering the relationship between attribute-level elasticities and item-level

elasticities is an empirical one. We first compute item-level elasticities based on fixed effect

parameters recovered by our analytical procedure and then regress these on the appropriate

attribute-level elasticities. That is, the elasticty for the SKU with the following features:

Colgate (Brand), Mint (Flavor), Gel (Form), Tartar Control (Function) and Medium (Size)

is regressed on the attribute-level elasticities for Colgate, Mint, Gel, Tartar Control and

22

Medium. The estimated coefficients from this regression represent the importance weights

for the attributes in determining the overall item-level elasticity.

One possible application is the estimation of likely price response to new items that

are not currently part of the category. To examine this idea, we selected eight new items

introduced in the last seventeen weeks of the data and calibrated the model using the first

eighty-seven weeks of data. The new items included five by Colgate, two by Crest and one

by Arm & Hammer. Table 6 shows: (1) parameter estimates from the regression of item

elasticities on attribute elasticities (given in the first row), and (2) price elasticity estimates

for the eight new items. The weights indicate the order of attribute importance and show

that size, brand and function are statistically significant while flavor and form are not. Actual

and predicted elasticities for the eight new items are shown in the first two columns of Table

6 (actual elasticities are calculated using the procedure set out in Appendix C, while the

predicted values are obtained from the regression). We are able to recover the item-level

price elasticities in the holdout sample fairly well (with the MAD at around 0.45, and the

MSE at around 0.26).

––––––––––––––

[ Table 6 about here ]

––––––––––––––

Comparison with a Pure Characteristics Approach

The FH model estimated on panel data – a pure characteristics approach – has generated

a lot of interest among both academic researchers and practitioners. This owes a lot to the

parsimony of the formulation and simplicity of estimation. In the FH model individual items

are expressed as additive linear functions of parameter values for levels of attributes that

describe the item. A reasonable question to ask is: Why not implement a direct replication

of the pure characteristics model on market share data (i.e., regress log differenced market

shares on the dummy variables for the levels of the different attributes)? To address this

question Table 7 reports the “part-worths” from the pure characteristics approach. The first

three columns contain results for a model with no mean-centering for price. In this case, the

23

estimated price coefficient (γ = −6.539) is underestimated. The results based on data witha mean-centered price correct this and replicate the price coefficient value obtained earlier

and shown in Table 4.

––––––––––––––

[ Table 7 about here ]

––––––––––––––

Points of difference between these findings and those from our method emerge

• Fixed Effects. The attribue-level parameter estimates (or “part-worths”) are consis-tently over-estimated when no mean centering is performed for the covariate (as notedearlier this also results in a biased coefficient for the covariate). Furthermore, there is arelatively substantial difference between the true fixed effects and those implied by thepure characteristics model. As shown in Table 4 our recovery procedure reproduces thefixed effects exactly (MSE is essentially zero), however, under the pure characteristicsapproach the comparable MSE is 0.503.

• Model Fit. The model fit (with R2 = 35 − 40%) is considerably worse for the purecharacterstics specifation. The attribute-level models have values of 90% and above.

• Attribute-Level Price Response. As shown in Table 3 our model allows one to examineattribute level price sensitivity while a pure characteristics approach does not. More-over, our approach facilitates forecasts of the likely price sensitivity to new items (seeTable 6) by relating the item-level elasticities to their attribute-level counterparts.

More insight into the relative merits of these approaches can also be seen via an in-sample

comparison of predicted item-level fixed effects for the top and bottom ten selling items.

Table 8 reproduces item-level fixed effects for three models: (1) estimated effects from the

true item-level model (IL), (2) calculated fixed effects from the attribute-level model (AL),

and (3) estimated fixed effects from the pure characteristics model (PC). The MSE difference

for the IL and AL models (see also Table 4) is very small (0.00014). For the IL and PC

models however this is considerably larger (0.50). This is also reflected in the rank ordering

of items. Column 1 shows that the IL and AL models produce identical rank orderings for

items, whereas column 5 shows markedly different rankings for the PC model.

––––––––––––––

[ Table 8 about here ]

––––––––––––––

24

While these findings are based on in-sample prediction the superior performance of our

model is also evident in holdout tests. Specifically, we compute the out-of-sample mean

utilities and market shares for the eight new items discussed earlier. To generate forecasts

for these new items from the AL model we translate the recovered item-level intercepts to

a set of part-worths for attribute levels. To accomplish this, and to allow comparison with

the PC model, we regress the recovered item-level intercepts onto the design matrix used

by the PC model.15 Using these recovered estimates, we forecast the mean utilities and the

market shares of each item in the holdout sample. The calibration sample was constructed

from the first 87 weeks of data and the holdout sample from the last 17 weeks (all eight new

items were introduced in this period). For the PC model the MAD and MSE values for out

of sample prediction of mean utilities for the new items are 0.2738 and 0.3298, respectively.

The AL model produces better out of sample forecasts with values of 0.2598 and 0.3022.

This pattern is repeated in market share prediction with the AL again producing better

results.16

Conclusion

We show analytically that a market share model specified at the SKU level is related to

another market share model aggregated to the attribute level, and that this relationship holds

for any desired attribute. This knowledge facilitates construction of item-level parameter

values by calculation from estimates obtained from the simpler aggregated model and avoids

the need for direct estimation of the item level model. This approach is validated empirically

using store-level scanner data from the toothpaste category.

Although the number of items changes dramatically from week to week and there is

15In this regression the number of observations is equal to the number of recovered fixed effects (i.e., 168).

This step in our method is essentially identical to that employed by Nevo (2000).16These values are obtained assuming that the prices of the new items are not known in advance. This

assumption is handled naturally in the AL model as mean-centering enforces the notion that the calculated

fixed effects capture mean levels of item share given an “average” marketing mix environment. If we use

knowledge of prices then the PC forecasts of mean utilities improve to yield MAD and MSE values of 0.2708

and 0.3085, respectively, however these are still below the values for the AL model.

25

the possibility of considerable interaction among attributes (e.g. Colgate’s tartar control

toothpaste may be better than that produced by Crest), our methodology is able to recover

parameters of the item level market share model very well. Out of sample forecasts reveal that

the attribute level model produces better predictions that those obtained from an obvious

alternative – the pure characteristics approach. The attribute-level model’s simplicity,

parsimony and reliance on analytical relationships make it an appealing method for both

researchers and practitioners. In subsequent work we plan to apply this new method to

substantive problems that require use and understanding of data from all SKUs in a product

category.

26

References

[1] Bell, David R. and James M. Lattin (1998) “Consumer shopping behavior and retail

price format: Why ‘large basket’ shoppers prefer EDLP,” Marketing Science, 17 (1),

66-88.

[2] Besanko, David, Sachin Gupta and Dipak Jain (1998) “Logit demand estimation under

competitive pricing behavior: an equilibrium framework,” Management Science, 44 (11,

part 1 of 2), 1533-1547.

[3] Brodie, Rod and C. deKluyver (2001) “Attraction versus linear and multiplicative mar-

ket share models: an empirical evaluation,” Journal of Marketing Research, 21, 194-201.

[4] Bucklin, Randolph E. and Sunil Gupta (1999) “Commercial use of scanner data: indus-

try and academic perspectives,” Marketing Science , 18 (3), 247-273.

[5] Camerer, Colin and Ho, Teck-Hua (1999) “Experience-weighted attraction learning in

normal-form games,” Econometrica, 67 (4), 827-874.

[6] Chen, Yuxin, James D. Hess, Ronald T. Wilcox and Z. John Zhang (1999) “Accounting

profits versus marketing profits: a relevant metric for category management,”Marketing

Science , 18 (3), 208-229.

[7] Chintagunta, Pradeep K., Bonfrer, Andre and Inseong Song (2002) “Investigating the

effect of store-brand introduction on retailer demand and pricing behavior,” Manage-

ment Science, 48 (10), 1242-1267.

[8] Cooper, Lee G. and Masao Nakanishi (1974) “Parameter estimation for a multiplicative

competitive interaction model,” Journal of Consumer Research, 1 (2), 36-44.

[9] ––— and ––— (1982) “Simplified estimation procedures MCI models,” Marketing

Science , 1 (3), 314-323.

[10] Dreze, Xavier and Stephen J. Hoch (1998), “Exploiting the customer base using cross-

merchandising and category destination programs,” International Journal of Research

in Marketing , 15 (5), 459-472.

27

[11] Fader, Peter. S. and Bruce G. S. Hardie (1996) “Modeling consumer choice among

SKUs,” Journal of Marketing Research, 33, (4), 442-452.

[12] Ho, Teck-Hua, and Chong, J.K. (2003) “A Parsimonious Model of SKU Choice,” Work-

ing Paper, Haas School of Business, University of California, Berkeley.

[13] Kamakura, Wagner A. and Gary J. Russell (1989) “A probabilistic choice model for

market segmentation and elasticity structure,” Journal of Marketing Research, 26 (4),

379-391.

[14] Leeflang, Peter S.H., Dick R. Wittink, Michel Wedel and Philippe A. Naert (2000)

“Building Models for Marketing Decisions,” Kluwer Academic Publishers: Nordwell,

MA.

[15] Montgomery, Alan L. (1997) “Creating micro-marketing pricing strategies using super-

market scanner data,” Marketing Science, 14 (4), 315-337.

[16] Nevo, Aviv. (2000) “A practitioner’s guide to estimation of random-coefficients logit

models of demand,” Journal of Economics and Management Strategy , 9 (4) 513-548.

[17] ––— (2001) “Measuring market power in the ready-to-eat cereal industry,” Economet-

rica, 69 (2), 307-342.

[18] Raudenbush, Stephen W. and Anthony S. Bryk (2002) “Hierarchical linear models:

applications and data analysis methods,” Sage Publications: Thousand Oaks, CA.

[19] Russell, Gary. J. and Ruth N. Bolton (1988) “Implications of market structure for

elasticity structure” Journal of Marketing Research, 25 (3), 229-241.

28

A Model Simulations

We first generated a set of market shares for items j = 1, . . . J over time t = 1, . . . , T (with outsidegood) based on an additive specification for the indirect utility as a function of attribute utilities. Theattribute utilities, and the simulated item level utilities, are already relative to a hypothetical “outside”good. That is, if the utility of the consumer with this structure were to be smaller than zero, then nopurchase would occur of this item. We used a combination of brand and size, and had a complete designof two sizes and three brands for a total of six “items”. In addition, we generated a set of prices withaverage values for prices unrelated to the mean utilities for the items’ attribute levels. The indirectutilities thus consist of the two components

Ujt = βj + γpjt

where βj = f(bj , sj) is the item level intercept, as a linear additive function f() of brand and sizeattributes respectively. The γ and βj are parameters to be recovered in the simulation. We add astandard normal residual to Ujt and transform the data to obtain regression equations for item-levelshares (see equation eqn:trans). To obtain the simulated attribute share values, we simply add upmarket shares of items that belong to a common level (e.g., same brand or same size) of a particularattribute. Formally, this can be done using the following transformation matrices

Bx =

1 0 0

1 0 0

0 1 0

0 1 0

0 0 1

0 0 1

Sx =

1 0

0 1

1 0

0 1

1 0

0 1

The matrix Bx will be used to produce attribute share equations for brands and the Sxmatrix attributeshares for sizes. A kronecker delta operator multiplies the transformation matrix by an identity matrixIT of dimension (T ×T ) so that we have the mapping of item shares to brand shares Sb = (IT ⊗Bx) sand item shares to size shares Ss = (IT ⊗ Sx) s where s denotes the item share vector of dimension(J × T )× 1. The design matrix for brand and size regressions can be created using Bx and Sx, simplystacked T times. For the item market share regressions we simply stack the identity matrix IT , T times.

With the simulated matrix of market shares, we build the estimation equations using the transforma-tions log sjt − log s0t, where the outside good value for the both attribute share and the item sharemodels have the same values for each observation t = 1, . . . , T (see equation 6. The estimation procedureis as in equations (15) or (25) and (26).

B Standard Errors

The standard errors of the item level intercepts are equal to

Var[βj ] = E[β2j ]− E[βj ]2

which is

Var[βj ] = E βb + ( bt − jt) + logsjtSbt

2

− E βb +1

T

T

t=1

logsjtSbt

2

29

= E ( bt − jt)2 + 2E ( bt − jt) log

sjtSbt

+ E logsjtSbt

2

− 1

Tlog

sjtSbt

2

= E ( bt − jt)2 +

1

T

T

t=1

logsjtSbt

2

− 1

T

T

t=1

logsjtSbt

2

(27)

where the second step is a result of expanding the terms in brackets and taking expectations.

This is a decomposition of the standard error into a component due to the item (unobserved at thispoint) and a component due to the brand, plus a component due to a mean-centered log ratio ofthe item share to the attribute share b. For the standard error term jt we can use the estimatedOLS residual from the pricing equation (26). The estimate of bt comes from the residual of the OLS

estimation of brand share. The standard error of the price estimate, γ, is s2(X X)−1, where Xis the (mean-centered) covariate, and s2 is the estimated variance, e e/(N − k). We use X here forgeneralizability, since it is possible to add any mean-centered covariate and preserve the validity of theestimated item-intercepts.

C Price Elasticities

Within any market share model, the estimated price coefficient γ is the same regardless of whethermean-centering of the price covariate is done or not. The usual calculation of a point elasticity for pricein the market share model is

ηj,k =∂sj∂pk

pksj

where ηj,k is the price elasticity for price of option k on the share of option j, and sj and pj are themean share and price, respectively, for option j. In the specification of market share given in equation(1) this becomes

ηj,k = γpj(1− sj) ∀ j = k

= −γskpj ∀ j = k

In the case of the mean-centered market share specification, the price “point” at which we take thiselasticity is zero, so normally this implies the price elasticity is zero. Recall that the price coefficientfor the mean-centered price is identical to the price coefficient for the non-mean centered price marketshare model. Therefore, we can utilize this. Note, however, the mean utility of choice j in the non-meancentered model is related to the attribute utility βmcj of the mean-centered model in the following way:

µjt = βj

=βmcj −γpj

+γpjt + jt

Now we need to take into account this relation in our elasticity calculation. Note that any such marketshare models that do not mean-center prices, should take this into account. By adding the covariateinto the specification, unless we mean center, the elasticity should take into account the indirect effectof any price change via the average of the covariate, on mean utility. The significance of this problemis likely to be small if one were only to predict a temporary impact of a price change on market share.However, if the analyst were interested in understanding how a permanent shift in price would affectmarket share, the inference would be biased by this non-centering relation. Thus the own-price elasticitybecomes:

ηj,j =∂sjt∂pjt

pjsj

30

= (γ − γ∂pj∂pj

)sj(1− sj) pjsj

= γ(1− ∂pj∂pj

)(1− sj)pj

So this implies that we need to consider the impact of −γ ∂pj∂pj

in calculating the elasticity. This value is

likely to be small if one is considering a temporary change in price (e.g. trying to calculate the marketshare impact of running a price-based promotion), but this result would be more important given amore permanent price change. A similar effect is present in the cross-price elasticity:

ηj,k = −γ(1− ∂pk∂pk

)skpk

Again, if these are temporary price changes, the impact is likely to be small. If the price change is morepermanent, the cross-price elasticity will need to take into account the impact of the price change onthe average price.

31

Table 1: Summary Statistics for the Toothpaste Category

A: Availability of Items

Inventory Status # of Items % of Items Share

Occasional stock-out 27 16.1 20.7

Established (always on shelf) 22 13.1 30.9

Added during observation period 41 24.4 20.5

Deleted during observation period 49 29.2 15.6

Added then deleted 29 17.3 12.2

B: Attribute Levels and Item Locations

Flavor Form Function Size

Brand levels 10 10 10 10

Attribute levels 14 3 7 4

Possible locations 140 30 70 40

Locations used 29 18 26 26

Percentage used 20.7 60.0 37.1 65.0

32

Table 2: Item Ownership

Manufacturer Share

Block Drug Co Inc 0.009

Church & Dwight Co Ltd 0.078

Colgate Palmolive 0.339

Procter & Gamble 0.373

SmithKline Beecham 0.099

Unilever 0.102

Herfindahl Index (1/n) 0.280 (0.167)

Attribute-level Herfindahl (1/n)

Brand 0.251 (0.100)

Flavor 0.490 (0.071)

Form 0.376 (0.333)

Function 0.417 (0.143)

Size 0.294 (0.250)

33

Table 3: OLS Estimation Results for Attribute-Share Models

Parameter Attribute-Level Estimates (s.e.)

Brand Flavor Form Function Size

Intercept (min) -6.87 -7.74 -4.39 -8.07 -5.98

(0.04) (0.04) (0.03) (0.05) (0.04)

Intercept (max) -2.92 -2.42 -2.43 -2.72 -2.60

(0.04) (0.04) (0.03) (0.05) (0.04)

Intercept (med) -5.16 -5.54 -2.94 -4.83 -3.29

(0.04) (0.04) (0.03) (0.05) (0.04)

Price -6.79 -10.30 -14.15 -8.15 -4.70

(0.35) (0.64) (0.74) (0.51) (0.86)

Adj R2 0.94 0.89 0.92 0.93 0.93

N (La × T )1 980 995 294 682 392

1All levels for all periods are present for Brand, Form and Size.

Table 4: Item-Level Intercepts: Estimated and Calculated

Model Parameter Values

Fixed Effect Summary Price R2

Min Max Median Mean MSE1

Item-Level: Estimated -9.081 -4.898 -7.089 -7.114 -7.473 0.561

(s.e.) (0.08) (0.12) (2.64)

Brand-Level: Calculated -9.061 -4.902 -7.099 7.112 1.44e-04 -7.473 0.076

(s.e.) (0.76) (0.96) (2.64)

1Mean squared difference between estimates from item-level and attribute-level models.

34

Table 5: Attribute-Level Price Elasticities

Brand Aim Aquafresh A&H Closeup Colgate Crest Gleem

-2.0186 -1.9581 -2.9584 -2.0632 -1.9312 -1.9793 -1.9505

Pepsodent Sensodyne Ultrabrite

-1.2634 -5.9942 -1.8801

Flavor1 BGum BMint CMint XFresh FMint Mint Orig

-3.7297 -3.7600 -3.0015 -2.8740 -4.1894 -3.3991 -3.1723

Reg WFresh ClMint NatMint HPunch OBGum SMint

-2.7708 -3.0954 -3.4396 -2.5192 -3.0045 -2.9872 -3.0503

Form Gel Paste Gel + Pst

-4.2748 -4.0655 -4.0795

Function Cavity Plaque Reg Tartar Triple Plq+Tar Sens

-2.6623 -2.6904 -2.3530 -2.3914 -2.2506 -3.3939 -6.9949

Size Small Medium Large X-Large

-2.4989 -1.7425 -1.3156 -1.1953

1Non-abbreviated flavor descriptions can be found in Table 7.

35

Table 6: Forecasted Elasticities for New Items

Price elasticities Regression Coefficients

Intercept Brand Flavor Form Function Size

Item Actual Predicted 0.5715 0.3321 0.0645 -0.001 0.248 0.9942

(0.357) (0.065) (0.050) (0.079) (0.039) (0.085)

23 -3.364 -2.578 -2.958 -4.189 -4.275 -2.391 -1.316

90 -2.720 -2.136 -1.931 -2.771 -4.066 -2.353 -1.316

91 -2.720 -2.157 -1.931 -3.095 -4.275 -2.353 -1.316

92 -2.723 -2.146 -1.931 -2.771 -4.066 -2.391 -1.316

93 -2.720 -2.145 -1.931 -2.771 -4.275 -2.391 -1.316

94 -1.830 -2.016 -1.931 -2.771 -4.275 -2.353 -1.195

140 -2.116 -2.073 -1.979 -3.399 -4.275 -2.353 -1.195

141 -2.496 -2.192 -1.979 -3.399 -4.275 -2.353 -1.316

MAD 0.452

MSE 0.259 R2=0.7655

36

Table 7: Results for the Pure Characteristics Model

Standard Mean-centered PriceParameter1 Estimate s.e. t Estimate s.e. tPrice -6.539 0.179 -36.487 -7.473 0.330 -22.616

Brand: Aim -0.318 0.101 -3.153 -0.587 0.105 -5.597Aquafresh -0.900 0.088 -10.260 -1.591 0.089 -17.836A & H 0.559 0.079 7.101 -0.164 0.079 -2.070Closeup -0.062 0.077 -0.804 -0.324 0.080 -4.056Colgate 0.611 0.073 8.432 0.141 0.074 1.895Crest 0.559 0.061 9.169 0.085 0.062 1.377Gleem -0.514 0.097 -5.320 -0.729 0.100 -7.266Pepsodent -0.797 0.112 -7.150 -0.758 0.116 -6.527Sensodyne -1.169 0.159 -7.372 -5.642 0.105 -53.894

Flavor: Bubble gum -0.085 0.090 -0.943 -0.246 0.094 -2.612Bubblemint 1.200 0.139 8.638 1.356 0.145 9.375Cool mint -0.403 0.073 -5.555 -0.593 0.075 -7.874Extra fresh -0.278 0.097 -2.863 -0.507 0.101 -5.034Fresh mint 0.193 0.073 2.637 -0.250 0.075 -3.334Mint -0.117 0.067 -1.747 -0.412 0.069 -5.981Original 0.062 0.073 0.846 -0.316 0.076 -4.186Regular -0.014 0.069 -0.211 -0.280 0.071 -3.943Winterfresh -0.099 0.089 -1.117 -0.213 0.092 -2.313Clean mint -0.033 0.274 -0.122 -0.230 0.285 -0.806Natural mint -0.557 0.446 -1.251 -0.699 0.464 -1.507Hawaiian punch -0.128 0.197 -0.649 0.059 0.205 0.290Orange bubble gum 0.239 0.193 1.234 0.436 0.201 2.168

Form: Gel -0.636 0.056 -11.373 -0.978 0.057 -17.045Paste -0.385 0.050 -7.639 -0.620 0.052 -11.912Paste and gel 0.000 0.000

Function: Anti-cavity -4.673 0.104 -45.085 -5.602 0.105 -53.533Anti-plaque -3.993 0.103 -38.676 -4.910 0.104 -47.090Regular -4.110 0.095 -43.434 -4.867 0.096 -50.611Tartar control -4.095 0.094 -43.639 -4.817 0.096 -50.412Triple protection -3.866 0.102 -38.067 -4.435 0.105 -42.431Anti-plaque/tartar -4.098 0.125 -32.922 -5.175 0.126 -41.092

Size: Small -0.628 0.062 -10.218 -2.057 0.049 -41.678Medium -0.593 0.036 -16.473 -1.283 0.032 -40.269Large -0.161 0.032 -5.018 -0.515 0.032 -16.157Adjusted R2 0.401 0.351N 9348 9348

1 Ultrabrite, Smooth Mint, Sensitive and Extra-Large normalized to zero.

37

Table 8: Comparison of Item Fixed Effects

IL1 IL AL PC PC2 brand flavour form function size

Top 10 Ranked By Item Utility1 -4.898 -4.902 -5.669 9 Colgate Regular Paste Plaque X Large2 -5.193 -5.207 -6.140 26 Colgate Regular Paste Regular large3 -5.284 -5.288 -5.576 2 Colgate Regular Paste Tartar X Large4 -5.293 -5.307 -6.140 27 Colgate Regular Paste Regular large5 -5.325 -5.321 -6.140 28 Colgate Regular Paste Regular large6 -5.430 -5.428 -5.626 5 Colgate Regular Paste Regular X Large7 -5.517 -5.523 -5.626 6 Colgate Regular Paste Regular X Large8 -5.543 -5.543 -5.631 8 Crest Regular Paste Tartar X Large9 -5.575 -5.575 -6.182 32 Crest Original Paste Tartar large10 -5.615 -5.609 -5.576 3 Colgate Regular Paste Tartar X Large

Bottom 10 Ranked By Item Utility159 -8.370 -8.357 -7.971 157 Aquafresh Regular P+G Tartar medium160 -8.394 -8.371 -7.217 117 Colgate Regular Gel Tartar medium161 -8.452 -8.426 -7.202 113 A’fresh Regular P+G Tartar large162 -8.474 -8.483 -8.436 165 A’fresh X Fresh Paste Triple medium163 -8.491 -8.507 -7.935 154 Colgate Wtr frsh Gel Cavity medium164 -8.501 -8.469 -6.519 60 Crest B’Gum Gel Regular large165 -8.711 -8.717 -7.908 152 A & H Fr Mint Paste Tartar small166 -8.843 -8.820 -6.306 37 A’fresh Regular P+G Triple X Large167 -9.025 -8.992 -7.589 136 A’fresh Regular P+G Triple medium168 -9.081 -9.061 -9.364 168 A’fresh Regular Paste Tartar small1 Rank order for the IL and AL models (order is identical).2 Rank order for the PC model.

Figure 1:  Time Series of Toothpaste SKUs 

 Figure 2:  Calculated Versus Estimated Fixed Effects