Does Providing Competitive Information to Your Own Customers …ebusiness.mit.edu/research/papers/2010.04_Liberali_Urban... · 2010-08-05 · (USAM) to provide competitive. information

Does Providing Competitive Information to Your Own

Customers Increase Sales? Paper 272

Guilherme Liberali Glen L. Urban John Hauser

April 2010

A research and education initiative at the MIT Sloan School of Management

For more information,

[email protected] or 617-253-7054 please visit our website at http://digital.mit.edu

or contact the Center directly at

Does Providing Competitive Information to Your Own Customers Increase Sales?

by

Guilherme Liberali

Glen L. Urban

and

John R. Hauser

April 15, 2010 Guilherme (Gui) Liberali is a Visiting Scholar at MIT Sloan School of Management, and Assis-tant Professor of Marketing, Erasmus School of Economics, Erasmus University Rotterdam, 3000 DR Rotterdam, The Netherlands, (1.781) 632-7674, fax (+31)10-408-9169, [email protected]. Glen L. Urban is the David Austin Professor of Marketing, MIT Sloan School of Management, Massachusetts Institute of Technology, E40-159, 1 Amherst Street, Cambridge, MA 02142, (617) 253-6615, [email protected]. John R. Hauser is the Kirin Professor of Marketing, MIT Sloan School of Management, Massa-chusetts Institute of Technology, E40-179, 1 Amherst Street, Cambridge, MA 02142, (617) 253-2929, [email protected]. This research was supported by the MIT Sloan School of Management, the Center for Digital Business at MIT (ebusiness.mit.edu), and General Motors, Inc. We gratefully acknowledge the contributions of our industrial collaborators, research assistants, and faculty colleagues: Eric Bradlow (Wharton), Michael Braun (MIT), Luis Felipe Camargo (UNISINOS), Daria Dryabura MIT), Patricia Hawkins (GM), Daniel Roesch (GM), Dmitriy A Rogozhnikov (IBM), Catherine ucker (MIT), JuanJuan Zhang (MIT).

(T

Does Providing Competitive Information to Your Own Customers Increase Sales?

Abstract

We examine data from a field experiment and a quasi-experiment in which a major US

automaker provided competitive information to its own customers. The automaker believed that

it was offering excellent vehicles but that consumers would not consider its vehicles because of

past experiences. Competitive information took many forms. An Auto Show in Motion made it

easy for consumers to test drive competitive vehicles from Chrysler, Ford, General Motors, Hon-

da, Mercedes, Toyota and other manufacturers. eBrochures provided brochures on over 100

competitive vehicles. Auto Choice Advisor provided unbiased purchase recommendations. And a

community forum encouraged consumers to discuss buying and owning experiences.

In year 1 the automaker provided four treatments to consumers in a 2 x 2 x 2 x 2 field ex-

periment. In year 2, to simulate a national launch, the automaker used an opt-in quasi-experiment

with a control cell, a forced-exposure cell, and a pure opt-in cell. In both years data were col-

lected monthly for six months. We model consumer transitions to consideration and purchase us-

ing a (hidden) continuous-time Markov process which accounts for potential misclassifications

of consumers’ behavioral states. The analyses suggest that both competitive test drives and com-

petitive brochures are effective strategies for increasing consideration and purchase. The treat-

ments are mediated through trust. Neither the online advisor nor the community forum were ef-

fective in generating trust, consideration, or sales.

Keywords: Automobile Applications, Competitive Information, Continuous-time Markov

Processes, Communications, Electronic Marketing, Hidden States, Information

Search, Misclassification, Quasi-experiments, Trust

1

Competitive Information to Your Own Customers

INTRODUCTION AND MOTIVATION

Information is everywhere on the Internet. If a firm does not provide information to its

customers, someone else will. This is particularly true in the automotive market. Websites such

as Autotrader.com, Cars.com, ConsumerReports.org, Edmunds.com, Kelly Blue Book

(kbb.com), and TheAutoChannel.com compete to provide specifications, reviews, prices, and

availabilities for most makes and models. Automakers often do not control this information. On

the other hand, one of the most comprehensive sources of consumer information, test drives, are

offered only by manufacturer-affiliated dealers.

Suppose an automaker believes that its vehicles are more reliable and satisfy consumer

needs significantly better than key competitors and has evidence that consumers do not share this

belief. Suppose that by drawing on experiences with this automaker in the last 10-20 years con-

sumers reject the automaker’s vehicles before searching for information on potential purchases.

If consumers never search for information and never test drive an automaker’s vehicles, then

those consumers will never buy the automaker’s vehicles. Its sales will suffer. Unfortunately, this

scenario described key US automakers in the first decade of the 21st century and was a contribut-

ing factor in their bankruptcies. For example, despite the facts that Buick was tied with Lexus

for the top spot in J. D. Power’s 2007 vehicle dependability ranking, was the top US brand in

2008 Consumer Reports, and was the number one brand in China, roughly half of all US con-

sumers (and almost 2/3rds in California) would not even consider a Buick.

In this paper we analyze an experiment and a quasi-experiment by a US automaker

(USAM) to provide competitive information to consumers. USAM provided consumers with the

ability to test drive 90 competitive vehicles, get unbiased competitive eBrochures, have access to

an unbiased web-based advisor that often recommended competitive vehicles, and join an online

2


community forum that discussed both USAM vehicles and competitive vehicles. USAM wanted

to test whether such competitive information would increase consideration and purchase of

USAM’s vehicles.

In the first year USAM randomly assigned treatments in a 2 x 2 x 2 x 2 field experiment.

The year-1 results identify which strategies have the most potential. In the second year, USAM

used a business-driven quasi-experiment to determine whether or not the marketing actions could

be implemented nationwide. To mimic a national launch USAM encouraged consumers to opt-in

to treatments. To address opt-in self-selection, USAM used quasi-controls: a control group in

which consumers were not given the opportunities for the test drives, brochures, advisors, or

communities, and a forced-exposure group in which consumers were given strong incentives to

visit the website from which they could opt-in to such competitive information.

USAM realized that the impact of competitive information might be indirect. Competi-

tive information might encourage consumers to consider USAM but not affect purchase condi-

tioned on consideration. (This would still increase net purchases because the pool of consumers

who consider USAM is now larger.) Theoretically, competitive information should enhance con-

sideration because, according to the evaluation-cost theory of consideration, consumers are more

likely to consider a product if a firm lowers the cost of evaluating that product relative to the

brands already in the consideration set (Hauser and Wernerfelt 1990). Alternatively, competitive

information might increase consideration and purchase by increasing trust in USAM even if

there is no direct effect on consideration and purchase. Trust is a common mediating variable

(Bart, et al. 2005; Urban 2004).

Modeling these indirect effects requires non-conventional analyses. First, to model indi-

rect effects through consideration the model must account for consideration as a latent construct;

3


empirical measures might have misclassified “consideration” as “not considered,” or vice versa.

Second, because automotive purchases occur over many months, USAM used a panel in which

subjects report consideration and purchase monthly for six months both year 1 and in year 2.

Third, observed “flows” from “not consider” to “consider” and to purchase (or not) may occur

faster than the interval of observation. Our model uses discrete observations from a continuous

system to infer the net effect of multiple unobserved paths to purchase.

Our primary method of analysis is a continuous-time Markov process (CTMP) with dis-

crete-time observation and subject to misclassifications of behavioral states. We begin by de-

scribing USAM’s year-1 experimental treatments and the measurements. We present one- and

two-stage CTMP models and address the impact of misclassification analysis. We next describe

the year-2 quasi-experiment, examine potential self-selection tests, and compare analyses of the

year-2 quasi-experiment to analyses of the year-1 experiment. The results suggest that competi-

tive test drives and competitive brochures enhance consideration and sales, although the effect is

indirectly through trust. We close with a discussion of managerial implications.

YEAR-1 EXPERIMENTS: COMPETITIVE-INFORMATION STRATEGIES

The year-1 panel ran from October 2003 to April 2004. (This was five years prior to the

bankruptcies of two US automakers.) Members of Harris Interactive’s panel were screened to be

in the market for a new vehicle in the next year, on average within the next 6.6 months, and in-

vited to participate and complete six monthly questionnaires. In total, Harris Interactive enrolled

615 Los Angeles consumers of whom 317 completed all six questionnaires for an average com-

pletion/retention rate of 51.5%. USAM did not retain recruitment rate statistics for year 1, but,

based on year 2, we estimate an initial recruitment rate of about 40%. Consumers were assigned

randomly to experimental cells in the 2 x 2 x 2 x 2 field experiment.

4


Competitive Test Drives: Auto Show in Motion

Consumers were invited to an event at a California test track to drive vehicles from

BMW, Chrysler, Dodge, Ford, General Motors, Honda, Lexus, Mercedes, and Toyota without

sales pressure. In returned they received coupons good at Amazon.com and a chance to win

$10,000. Figure 1a reproduces information about the test drive, called Auto Show in Motion

(ASIM). ASIM had substantial fixed costs to set up the test track, procure and maintain over 100

vehicles, assure safety, and provide staff assistance. Per consumer costs, including recruiting, in-

centives, and lunch, were in the $50-100 range. ASIM was available in period 4 to 39.1% of the

consumers.

Insert Figure 1 about here.

Customized Brochures

In year 1 this experimental treatment provided information about USAM rather than

competitive vehicles. (In year 2 USAM used competitive brochures.) Specifically, year-1 con-

sumers received brochures that were targeted to their specific needs as determined by measures

prior to the experiments (Figure 1b). The brochures were mailed in either period 2 or 3 to 51.7%

of the consumers.

Competitive Online Advisor: Auto Choice Advisor

Consumers were invited to use a web-based advisor that recommended vehicles based on

a series of questions that revealed the consumers’ wants and needs. The web-based advisor,

known as the Auto Choice Advisor (ACA), was similar to the advisor described in Urban and

Hauser (2004). See Figure 1c. ACA was available in periods 2 through 6 to 49.2% of the con-

sumers.

5


Competitive Community Forum

Consumers participated in an CommuniSpaceTM online forum that discussed both USAM

and competitive vehicles. See Figure 1d. Consumers were free to participate in any of the over

30 dialogues which averaged over 30 comments in each. Consumers could discuss experiences

with their buying or ownership experience for any competitor and do so in an unbiased manner.

The community was available in periods 2 through 6 to 47.6% of the consumers.

Trust Covariate

Consumers have varied experiences in the auto market and varied perceptions of USAM.

USAM believed that it was possible that competitive information (and brochures) would affect

consideration and purchase by increasing consumers’ trust in USAM. Trust measures also enable

us to control for past history and to test for indirect effects. USAM measured consumers’ trust

using a five-item scale with items such as “I believe that this company is willing to assist and

support me.” or “Overall, this company has the ability to meet customer needs.” The items exhi-

bited high reliability: Cronbach’s α = 0.95.

Measures Used in the Analyses

Dependent measures. In each period consumers reported the vehicles that they were

“considering for your next purchase or lease.” They indicated the make-model combinations

from a drop-down menu of 348 make-model combinations. USAM was interested in whether or

not one of these vehicles was a USAM vehicle. With 348 make-model combinations on a drop-

down menu, misclassification was a real concern. The purchase dependent variable was based on

purchasing records maintained by USAM. For modeling purposes, we assume that misclassifica-

tion of the purchase observation is small compared to misclassification of consideration.

Covariates. Many automotive consumers are brand loyal. The first of six surveys col-

6


lected data on the vehicles consumers owned prior to the experiment. We dummy-code these da-

ta as “own USAM,” “own other American vehicle,” and “own Japanese.” The dummy variables

are relative to “own European.” USAM measured age with 11 categories, which we dummy-

code with 10 categorical variables. The covariates do not vary by observation period.

Experimentally-assigned treatments versus treatments that were received. Some

consumers could not access the treatment(s) to which they were assigned. For example, some

consumers experienced technical difficulties with ACA and some could not come to ASIM. For-

tunately, the data contain self-reported treatment effects which we use to improve precision on

which consumers actually received treatments. The self-reports match up well with the assigned

treatments and seem to capture phenomena where consumers were not able to access the treat-

ment. We ran analyses with dummy-coded treatment variables and with dummy-coded self-

reports. Because they were similar and provided the same strategic interpretations, we report on-

ly the analyses based on the dummy-coded self-reported treatment effects. We now describe the

model and use it to analyze the year-1 experiment and the year-2 quasi-experiment.

LATENT BEHAVIORAL STATES AND CONTINUOUS TRANSITIONS

USAM is interested in whether or not competitive information encourages consumers to

consider and/or purchase USAM vehicles. We represent this focus with the Markov diagram in

Figure 2. The diagram is Markov because flows among behavioral states depend only on the cur-

rent behavioral state (and the explanatory variables), not the entire past history of flows.


7

We index continuous time with t and we index behavioral states with i where i = 1, 2, and

3 index “do not consider USAM,” “consider USAM,” and “purchase USAM,” respectively. Let

1 if the consumer is in state i at time t, and let 0 otherwise. (To simplify notation


we suppress the subscript for consumer.) While consumers flow continuously among states and

may make multiple transitions in a given month, we only observe behavioral states at monthly

intervals. Let be the observation time for the monthly observation. Because we only ob-

serve the result of continuous transitions, we derive an expression for the probability, ,

that the consumer was in state i at and in state j at , where . Let be

the transition matrix of the ’s.

Competitive information affects the rate at which consumers flow among states. Let

be the instantaneous flow rate during the observation period and let An be the flow matrix of

the ’s. Mathematically for j ≠ i, Δ is the probability that the consumer flows from state i

to state j in the time period between t and t + Δ for very small Δ during the observation pe-

riod. For small Δt the only way to be in state j at time t + Δt is to be there at time t or move there

from another state in time Δ . This property gives the following differential flow equation:

(1) Δ 1 Δ Δ

Following Cox and Miller (1965) and Hauser and Wisniewski (1982) we let Δt → 0 to obtain a

differential equation for the transition ma ix. tr

(2)

The solution to the differential equation requires matrix exponentiation which is a difficult nu-

merical problem. Equation 3 provides the solution to the differential equation and provides two

ways by which the solution can be computed. ( is the matrix of Eigenvectors of and

expΛ is the matrix with the exponentiation of the Eigenvalues on the diagonal.)

8


(3) !

exp Λ

To capture the effect of competitive information, trust, and other covariates we let the

flows be a function of these variables. By definition, the off-diagonal elements of are posi-

tive, so we use logarithms. Let 1 if the consumer received experimental treatment k in ob-

servation period n. Otherwise 0. Let be the trust as measured at the beginning obser-

vation period n, that is, as measured at the end of the 1 period. Let ℓ be the ℓ covariate.

We represent flows by Equation 4 for the feasible behavioral-state transitions in Figure 2. We

seek to estimate the unknown parameters: , , , and ℓ .

(4) log ℓ ℓ ℓ

We code our dependent-variable observations such that 1 if the consumer is ob-

served in behavioral state i at and in behavioral state j at . Using Equations 3 and 4 we set

up the data likelihood. Although medical researchers have developed reversible jump Monte

Carlo Markov Chain estimation to obtain estimates of the parameters for moderately-sized

CTMP models (e.g., Suchard, Weiss and Sinsheimer 2001), matrix exponentiation presents prac-

tical problems for many empirical applications, especially when the number of observed transi-

tions are small compared to the sample size as is the case in our data with USAM purchases.1

We found maximum likelihood methods to be more stable for the USAM experiments and quasi-

experiments (and more common in the CTMP literature). See Kulkarni (1995) for a review of

1 Pn is a stochastic matrix (rows sum to 1) which implies that the rows of An sum to zero. Thus the first Eigenvalue of Pn is 1.0 and describes steady-state behavior. The remaining Eigenvalues are less than 1.0 and describe the dy-namic behavior. If tn is too large or too small relative to the dynamic behavior of the system there will be only one non-zero Eigenvalue. The remaining Eigenvalues will be close to zero. Slight numerical errors could make them negative, which would imply imaginary flows when we use the logarithmic representation in Equation 3. This caus-es problems with a Bayesian sampler.

9


computational methods to deal with matrix exponentiation. (Our one-stage models have a run-

ning time of 1-5 hours with maximum-likelihood estimation. Two-stage models are quicker.)

MODELING POTENTIAL MISCLASSIFICATION OF DEPENDENT MEASURES

Consideration is both a transient construct and one that is measured with potential error.

For example suppose a consumer is not quite sure whether he or she is seriously considering a

USAM vehicle. The consumer who is unsure might say he/she is considering a USAM vehicle in

period 2 but not say so in period 3. Even if the consumer is sure about consideration, he/she

might miss a vehicle in a drop-down menu of over 348 vehicles.2

Survey noise might cause us to infer phantom flows from “not consider USAM” to “con-

sider USAM” and back. These phantom flows might disguise the effects of the experimental

treatments, trust, or the covariates. To account for misclassification we define 1 if we

observe that the consumer says that he or she is in state i at the end of the observation period.

The true state, , is not observed. To simplify notation let and .

We seek to infer the probability of correct classification, Pr | , and of misclassifications,

Pr | for j ≠ i.

To model misclassification we adopt Jackson, et al.’s (2003) hidden Markov model. To

model misclassification we recognize that the likelihood of a series of observed states,

, , … , , is equal to the probability that we made those observations conditioned on a

set of true states, , , … , . If there were no restrictions, then for any observed state the

consumer could have been in any true state. For any sequence of observed states, if all transitions

were allowed, there is a non-zero probability for any of the 3x3x3x3x3x3 = 729 paths through

the true states at times , , … , . In our model “purchase” is a trapping state and assumed to 2 Toward the end of the decade USAM developed improved methods to measure consideration that mimicked the manner by which consumers choose consideration sets. See Hauser, et al. 2010.

10


have negligible observation error, so the number of paths is far less, but not trivial. To form the

likelihood, we sum over all feasible true paths. For example, the likelihood that we observe “not

consider” at the end of the first observation period and “consider” at the end of the second obser-

vation period is given by Equation 5 where Pr is a prior probability. ,

Pr ,Pr | Pr Pr | Pr | Pr | Pr |

Pr | Pr Pr | Pr | Pr | Pr | (5)

As we expand Equation 5 to all six periods the likelihood gets complicated, but is han-

dled easily by computer. Indeed, Jackson, et al. (2003, p. 197) provide a compact matrix notation

to sum the likelihood over all true paths. Based on the data likelihood we estimate the parameters

of the CTMP (the β’s) and the misclassification probabilities simultaneously.

TESTING THE DIRECT EFFECT OF COMPETITIVE INFORMATION

Table 1 summarizes the direct estimation. To simplify Table 1, we do not report ’s for

the covariates – none were significant at the 0.10 level. Because of the logarithmic specification,

a negative coefficient indicates that a flow rate decreased, not that the flow is negative. Misclas-

sification was moderate; approximately 12% of the consumers were estimated to be misclassified

as “consider USAM” when they did not yet consider USAM and approximately 6% were esti-

mated to be misclassified as “do not consider USAM” when they considered USAM.

Insert Table 1 about here.

Each of the four sets of two columns represents one of the four allowable flows in Figure

2. Table 1 indicates that there is no identifiable direct effect due to the competitive information

(the experimental treatments). However, lagged trust significantly increases flows from consid-

eration to purchase and significantly decreases flows from “consider” to “do not consider.” To

11


anticipate a two-stage model, we estimated a reduced-form CTMP model with only lagged trust

as a variable. The results, shown in the lower portion of Table 1, have similar implications: trust

is a key driver of purchase and of maintaining consideration.

COMPETITIVE INFORMATION ACTS THROUGH TRUST

To examine the indirect effects of competitive information we develop a two-stage mod-

el. For the first stage we regress trust on competitive test drives, customized brochures competi-

tive advisors, and competitive community forums. We include lagged trust because the experi-

mental treatments are likely to increase or decrease trust and we include covariates to account for

unobserved propensity for trusting USAM. We include dummy variables for observation periods

to account for unobserved advertising and other actions by USAM and to account for unobserved

environmental shocks. (Period 1 is a pre-measure and the period-2 dummy variable is set to zero

for identification.) The period dummy variables also account for any measurement artifact that

might boost trust (“Hawthorne” effect). The trust regression is summarized in Table 2. (To sim-

plify exposition we suppress the categorical age variables; none were significant.)


Table 2 suggests that both competitive test drives (ASIM) and customized brochures in-

crease consumers’ trust in USAM. Given the importance of trust in the first stage (CTMP) this is

an important finding. The impact of customized brochures is consistent with other published stu-

dies of customization (e.g., Ansari and Mela 2003; Hauser, et al. 2009). The significance of

ASIM demonstrates that competitive information is likely to maintain consideration and increase

purchases of USAM’s vehicles. We believe that such large-scale field-experiment evidence of

the positive effect of competitive information is relatively novel.

12

To examine the second stage of the CTMP analysis, we use estimated trust ( ̂ ) from


the first stage rather than measured trust ( ) in the CTMP conditional likelihood. These two-

stage estimates are limited-information maximum-likelihood (LIML) estimates. LIML estimates

are consistent but we need to use bootstrap methods to check the standard errors for the ’s

(Berndt, et al. 1974; Efron and Tibshirani 1994; Wooldridge 2002, p. 354, 414). Table 3 reports

both the LIML estimates and the means from 1,000 bootstrap replicates.


The estimates in Table 3, both LIML and bootstrap, are similar to the reduced model in

Table 1 confirming the impact of trust on consideration and purchase.3 The two-stage model

confirms that some competitive information (competitive test drives) increases consideration and

purchase, but that the effect is through increased trust. This important finding suggests that

USAM explore further strategies to increase trust. We discuss such strategies later in the paper.

We did not find an effect for the competitive advisor and the competitive forum. This is

intuitive for the 2004-2005 time period. Although USAM’s vehicles had improved relative to the

early 2000s, ACA was recommending other vehicles (particularly the Japanese vehicles) and

USAM did not always get positive reviews in community forums. Before we examine the impact

of competitive information in the year-2 quasi-experiments, we detour to examine the relative

value of modeling misclassification.

THE IMPACT OF MODELING MISCLASSIFICATION IN A CTMP MODEL

Many applications in various literatures have modeled misclassification including the di-

3 The LIML and reduced-form estimates are almost identical. The LIML and bootstrap coefficients of lagged trust are not significantly different. The constants representing flows between “not consider USAM” and “consider USAM” do differ, but offset. This difference is likely due to misclassification analysis interacting with the bootstrap procedure. Specifically, because bootstrap randomly selects observations, many observations are repeated in a given replicate causing that replicate to underestimate misclassification. The means of the 1,000 replicates were almost identical to the medians suggesting that outliers were not a problem. The running time is about 250 hours for 1,000 two-stage replicates.

13


agnosis of disease progression, the diagnosis of toxoplasmosis infection, the spread of HIV, the

probability of workers changing jobs, volcanic activity, and Monte Carlo studies of forecasting

error (Aspinall, et al. 2005; Bessec and Bouabdallah 2005; Chen and Sen 2007; Hausman, Abre-

vaya and Scott-Morton 1998; Jackson, et al. 2003; Rahme, Joseph and Gyorkos 2000; Tzavidis

and Lin 2006). We test the impact of modeling misclassification in USAM’s experiment by us-

ing 5-fold cross-validation to examine predictive ability. For each of five subsamples we esti-

mate the two-stage CTMP model with 80% of the data and predict behavior for the remaining

20%. Accounting for misclassification improves hit rates significantly from 66.1% to 77.1% (p <

0.001). The coefficients themselves also vary. When we ignore misclassification, the constants

and lagged (estimated) trust remain significant in the same pattern as in Table 3, but we underes-

timate the impact of trust for maintaining consideration.

YEAR 2 QUASI-EXPERIMENTS ON COMPETITIVE INFORMATION

Buoyed with the success of the year-1 experiment USAM sought to test further the im-

pact of competitive information. The year-2 quasi-experiment further analyzed competitive test

drives (ASIM) but expanded the competitive information treatments to include competitive bro-

chures. USAM maintained both the competitive advisor and the community forum even though

they had no identifiable effect in year 1.

The big difference in year 2 was that USAM sought to test the ability to launch competi-

tive-information strategies on a national basis by allowing consumers to opt-in to the strategies.

By its very nature, opt-in makes the test a quasi-experiment requiring USAM to test for potential

self-selection. USAM included two quasi-controls. Consumers were assigned randomly to one of

three cells. Consumers in the control cell received no treatments. Consumers in the forced-

exposure cell were invited to an “Internet study” that included a visit to USAM’s “My Auto Ad-

14


vocate” website at which they could opt-in to competitive information treatments. See Figure 3a.

Consumers in the pure opt-in cell received an advertisement inviting them visit the “My Auto

Advocate” website.


The year-2 panel ran from January to June, 2005. Members of Harris Interactive’s panel

were again screened to be in the market for a new vehicle, on average within the next 2.2 years,

and invited to participate and complete six monthly questionnaires. (The year-1 sample was re-

stricted to 12-month intenders, the year-2 sample was not.) Incentives were similar to year 1. In

total, Harris Interactive invited 6,092 Los Angeles consumers of which 1,720 completed all six

questionnaires for an average response/completion/retention rate of 28.2%. This rate was not

significantly different across the three conditions.

Competitive Test Drives: Auto Show in Motion

Consumers were invited to an event at one of three test tracks. Otherwise ASIM was sim-

ilar to year 1. Opt-in respondents received 20 reward certificates for participating.

Competitive Brochures and USAM Booklets

In year 2 USAM continued to offer brochures (called eBooklets), but this time on an opt-

in basis. To test competitive information, consumers could also download or obtain a CD with

competitive brochures. Although many competitive brochures were available on manufacturers’

websites, USAM’s single-source webpage made it more convenient for consumers to compare

vehicles. Opt-in consumers received 5 reward certificates for downloading a USAM brochure.

See Figure 3b.

15


Competitive Online Advisor: My Product Advisor

USAM updated ACA to “My Product Advisor (MPA)” and made it available directly

from the “My Auto Advocate” website. Besides an improved interface, MPA had a “garage” at

which consumers could store vehicle descriptions. Like ACA, MPA was unbiased. Opt-in con-

sumers received 5 reward certificates for using MPA. See Figure 3c.

Competitive Community Forum

The community forum was updated and integrated with the “My Auto Advocate” web-

site. Opt-in consumers received at least 5 reward certificates for participating, but could earn up

to 20 reward certificates for active participation. See Figure 1d.

Trust, Dependent Measures, and Other Covariates

Trust, consideration, and purchase were measured as in year 1, as were prior ownership

of USAM, other American, and Japanese vehicles. Age was measured directly rather than by

categories and sex of the respondent was recorded.

EXAMINING ISSUES OF SELF-SELECTION TO TREATMENTS

It is possible that consumers who take advantage of competitive information are more

likely to be interested in purchasing an automobile in the near future, but this self-selection may

or may not translate into greater consideration for or purchase of USAM vehicles.

The Effect of Cell Assignment for Non-Participating Consumers

Many consumers in the opt-in (35.9%) and forced-exposure (45.0%) cells and all con-

sumers (100%) in the control cell did not participate in the opt-in treatments. If self-selection im-

pacts the dependent measures, then we should have a non-random removal of consumers from

the opt-in and forced-exposure cells. If self-selection compromises the quasi-experiment these

consumers should be less likely to consider and/or purchase USAM vehicles. To analyze this ef-

16


fect we created dummy variables for experimental cell and estimated a CTMP model with mis-

classification. There are no significant effects due to cell assignment (all p’s > 0.14 for all dum-

mies for all flows; often much higher). We obtain similar results controlling for trust.

Exposure to My Auto Advocate

We have already established that the there is no significant difference in net response rate

between those consumers who (a) were required to visit “My Auto Advocate” and completed the

six questionnaires (forced exposure) and (b) those who could opt-in to “My Auto Advocate” and

completed the six questionnaires (p > 0.17). To examine whether “My Auto Advocate” itself had

a significant effect on consideration and purchase we repeat the dummy-variable CTMP model

with misclassification, but for all consumers. The cell-assignment dummies remain insignificant

(all p’s > 0.23) with and without controlling for trust.

While these tests do not unequivocally rule out self-selection, they suggest it is reasona-

ble to examine the potential impact of providing consumers with competitive information and to

compare the effects to the year-1 experiment.

YEAR-2 ANALYSIS OF COMPETITIVE INFORMATION

We repeated all year-1 analyses with the year-2 data. As in year 1, none of the quasi-

experimental treatments had direct significant effects in the CTMP model and the constants and

coefficients of lagged trust in the reduced model were similar to those in the full model. Misclas-

sification was slightly lower than in year 1; approximately 9% of the consumers were estimated

to be misclassified as “consider USAM” when they did not yet consider USAM and approx-

imately 3% were estimated to be misclassified as “do not consider USAM” when they actually

considered USAM. Accounting for misclassification increased the predictive ability in a five-

fold cross-validation from 74.7% to 87.1% (p < 0.001). Because the two-stage LIML and boot-

17


strap estimates are similar to the reduced model we simplify exposition and report only the two-

stage estimates in Tables 4 and 5.4

Insert Tables 4 and 5 about here.

In year 1 both lagged trust and estimated lagged trust had significant effects on considera-

tion and purchase (e.g., p ≤ 0.03, bootstrap estimates). In year 2 both lagged trust and estimated

lagged trust also have significant effects on consideration and purchase (p ≤ 0.05, bootstrap esti-

mates). (The bootstrap standard errors are more accurate than the LIML standard errors.) In-

creasing trust is clearly beneficial for USAM in year 2 as it was in year 1.

Competitive test drives still increase trust in year 2, although the effect is now only mar-

ginally significant (p = 0.10). The interesting new implication is that competitive brochures in-

creased trust in USAM significantly in year 2 (p = 0.04). Sex and age are significant in year 2

possibly reflecting unobserved changes in USAM’s product mix or advertising. Overall, the

year-2 two-stage model is remarkably similar to the year-1 model despite changes in the experi-

mental treatments, unobserved changes in the environment, the change from an experiment to a

quasi-experiment, and a sample that is not limited to consumers who plan to purchase in 12

months. Tables 1 through 5 suggest strongly that competitive information is an effective strategy

to increase trust (in USAM) and, through trust, to increase consideration and purchase of USAM

vehicles.

These quantitative conclusions are consistent with qualitative comments by consumers

who participated in the quasi-experiment:

4 All reduced-model and LIML estimates were not significantly different (all p’s > 0.32). Differences in the LIML and bootstrap constants are explained in footnote 3. The mean and median bootstrap estimates are almost identical. One coefficient of lagged trust was significant for the bootstrap estimates, but not for the LIML estimates, however, the value of the coefficient itself is almost identical (0.468 vs. 0.474).

18


My Auto Advocate: “I've learned a lot more about USAM. I didn't realize how many dif-

ferent models of cars they own. It was eye opening. I enjoyed it and I have a more positive view

of USAM than previously.”

Competitive test drives: “Please don't stop doing these events, USAM. This was the one

and only reason we purchased a USAM car over a Mustang GT or a Dodge Magnum R/T. There

was no way we would have test-driven a USAM car had it not been for ASIM. It's the best expe-

rience I can imagine for overcoming people's prejudices against USAM and selling them on your

many terrific products (like the …). I doubt I'd make the trip to different dealers (…) to drive

those models.”

Competitive Brochures: “The CD was downright entertaining! I liked how it got a lot of

info across in a concise, easy-to-follow fashion, and made the specifics I was seeking easy to

find.”

MANAGERIAL IMPLICATIONS

US automobile manufacturers face challenges as they try to improve sales and profit post

bankruptcy. Our analyses have two implications. First, trust is key. Those consumers who trust

USAM are more likely to consider and purchase USAM vehicles. Second, competitive test

drives and competitive product brochures influence consumers to avoid rejecting USAM ve-

hicles before they gather information. The effect is predominately through increased trust in

USAM. USAM’s managers felt the results were consistent with qualitative data and sufficiently

compelling to investigate further.

Competitive test drives and competitive brochures were not rolled out nationally in large

part because of other distractions during the automotive and financial crises at the end of the first

decade of the 21st century. Competitive test drives are extremely expensive adding substantially

to the cost of selling a vehicle. Competitive brochures are more likely to be cost effective and,

with the growth of computer power and Internet use, are becoming more relevant. At minimum

USAM’s experiments and quasi-experiments provide evidence that honest competitive compari-

19


sons are effective for firms that are disadvantaged by low consideration despite having good

products.

As the new USAM seeks to regain profitability they have implemented many strategies to

encourage consumers to actively compare USAM vehicles to competitors. In 2009 a series of

advertisements starring a well-known sports announcer made explicit comparisons to competi-

tors featuring “surprises” such as good fuel economy. Another campaign was based on “May the

Best Car Win.” In the fall of 2009 USAM implemented a policy by which consumers could try

vehicles for 60 days and return them if they were not satisfied. USAM also ran successful mini-

experiments in which USAM dealers invited customers to test drive competitive vehicles. All of

these marketing tactics sought to encourage competitive comparisons and/or increase trust in

USAM.

SUMMARY

If a firm has products that are much better than consumers perceive them to be, our re-

sults suggest that consideration and purchase increase if the firm makes competitive information

available. (If a firm has inferior products then providing competitive information to consumers is

unlikely to be profitable.) Competitive information reduces search and evaluation costs making it

more likely for consumers to add another good product to their consideration sets. The firm rea-

lizes added benefits if it is transparent that the firm is providing competitive information. “Cre-

dit” for providing information likely leads consumers to place more trust in the firm. This ap-

pears to be what happened in USAM’s experiments and quasi-experiments. Competitive infor-

mation increased demand through consideration and purchase and did so by increasing trust in

USAM.

Managerial, the open question is whether providing competitive information is cost effec-

20


tive. Paying for competitive test drives does not appear to be cost effective, but encouraging con-

sumers to test drive both competitive and USAM vehicles does appear to be cost effective. Pro-

viding eBrochures appears to be a cost effective strategy.

To analyze the quasi-experiment we used CTMP models to address the data constraint

that transitions occur continuously while we only observe the results of these transitions at

monthly intervals. While CTMP and related models, including hidden Markov models, have

been used successfully to analyze other marketing issues, we are unaware of other applications in

marketing that address potential misclassification errors in CTMPs (Ding and Eliashberg 2008;

Eliashberg, et al. 2000; Hauser and Wisniewski 1982; Netzer, Lattin and Srinivasan 2008; Ro-

berts, Morrison and Nelson 2004, 2005; Srinivasan and Kim 2009; Weerahandi and Moitra

1995). The analysis of misclassification proved to be a key methodological component of our

analyses.

CTMP, accounting for misclassification, is a powerful tool. Because flows were sparse in

the USAM quasi-experiment, stability issues made maximum likelihood estimation the best nu-

meric technique. When flows are less sparse, matrix exponentiation is more stable numerically

and MCMC analysis will be computationally feasible. When there is more data per consumer

than in the USAM quasi-experiment, CTMP analyses could be extended to include heterogeneity

in flow rates among consumers. Perhaps new data will become available to develop these exten-

sions for CTMP models which account for misclassification.

21


REFERENCES

Ansari, Asim and Carl F. Mela (2003), “E-Customization,” Journal of Marketing Research, 40,

(May), 131-145.

Aspinall, W. P., R. Carniel, O. Jaquet, G. Woo, and T. Hincks (2006), “Using Hidden Multi-

State Markov Models with Multi-parameter Volcanic Data to Provide Empirical Evi-

dence for Alert Level Decision-Support,” Journal of Volcanology and Geothermal Re-

search, 153, 112-124.

Bart, Yakov, Venkatesh Shankar, Fareena Sultan, and Glen L. Urban (2005), “Are the Drivers

and Role of Online Trust the Same for All Web Sites and Consumers? A Large-Scale

Exploratory Empirical Study,” Journal of Marketing, 69, (October), 133-152.

Berndt, Ernie K., B. H. Hall, R. E. Hall, and Jerry Hausman (1974), “Estimation and Inference in

Nonlinear Structural Models,” Annals of Social Measurement, 3, 653-665.

Bessec, Marie and Othman Bouabdallah (2005), “What Causes The Forecasting Failure of Mar-

kov-Switching Models? A Monte Carlo Study,” Studies in Nonlinear Dynamics & Eco-

nometrics, 9, 2, Article 6.

Chen, Pai-Lien and Pranab K. Sen (2007), “Markov Chain Model Selection by Misclassified

Model Probabilities,” Communications in Statistics—Theory and Methods, 36, 143–153.

Cox, David R. and Hilton D. Miller (1965), The Theory of Stochastic Processes, (London, UK:

Chapman & Hall).

Ding, Min and Jehoshua Eliashberg (2008), “A Dynamic Competitive Forecasting Model In‐

corporating Dyadic Decision Making,” Management Science, 54, 4, 820‐834.

Efron, Bradley and Robert J. Tibshirani (1994), An Introduction to the Bootstrap, (New York,

NY: Chapman & Hall/CRC).

22


Eliashberg, Jehoshua, Jedid‐Jah Jonker, Mohanbir S. Sawhney, and Berend Wierenga (2000),

Marketing Science, 19, 3, 226‐243.

23

Hausman, Jerry A., Jason Abrevaya and Fiona M. Scott-Morton (1998), “Misclassification of the

Dependent Variable in a Discrete-response Setting,” Journal of Econometrics, 87, 239-

269.

Hauser, John R., Olivier Toubia, Theodoros Evgeniou, Daria Dzyabura, and Rene Befurt (2010),

“Cognitive Simplicity and Consideration Sets,” Forthcoming Journal of Marketing Re-

search, 47, (June).

------, Glen L. Urban, Guilherme Liberali, and Michael Braun (2009), “Website Morphing,”

Marketing Science, 28, 2, (March-April), 202-224.

------ and Birger Wernerfelt (1990), "An Evaluation Cost Model of Consideration Sets," Journal

of Consumer Research, Vol. 16, (March), 393-408.

------ and Wisniewski, Kenneth J. (1982), “Dynamic Analysis of Consumer Response to Market-

ing Strategies,” Management Science, 28, 5, 455-484.

Jackson, Christopher H., Linda D. Sharples, Simon G. Thompson, Stephen W. Duffy and Elisa-

beth Couto (2003), “Multistate Markov Models for Disease Progression with Classifica-

tion Error,” The Statistician, 52, 193-209.

Kulkarni, V. (1995), Modeling and Analysis of Stochastic Systems, (London, UK: Chapman &

Hall/CRC).

Netzer, Oded, James M. Lattin and V. Srinivasan (2008),” A Hidden Markov Model of Customer

Relationship Dynamics,” Marketing Science, 27, 2, 185-204.

Rahme, Elham, Lawrence Joseph, and Theresa W. Gyorkos (2000), Applied Statistics, 49, 119-

128.


24

Roberts, John H., Pamela D. Morrison and Charles J. Nelson (2004), “Implementing A Pre‐

launch Diffusion Model: Measurement And Management Challenges Of The Telstra

Switching Study,” Marketing Science, 23, 2, 186‐191.

-----, -----, and ----- (2005), “A Prelaunch Diffusion Model for Evaluating Market Defense Strat-

egies,” Marketing Science, 24, 1, 150-164.

Srinivasan, V. and Sang-Hoon Kim (2009), “A Conjoint‐Hazard Model of the Timing of Buy‐

ers' Upgrading to Improved Versions of High‐Technology Products,” Journal of

Product Innovation Management, 26, 3, 278‐290.

Suchard, Marc A., Robert E. Weiss and Janet S. Sinsheimer (2001), “Bayesian Selection of Con-

tinuous-Time Markov Chain Evolutionary Models,” Molecular Biology Evol., 18, 6,

1001-1013.

Tzavidis, Nikos and Yan-Xia Lin (2006), “Estimating from Cross-Sectional Categorical Data

Subject to Misclassification and Double-Sampling: Moment-based, Maximum Likelihood

and Quasi-Likelihood Approaches,” Journal of Applied Mathematics and Decision

Sciences, 2006, 1-13.

Urban, Glen L. (2004), “The Emerging Era of Customer Advocacy,” MIT Sloan Management

Review, (Winter), 45, 2, 77-82.

------ and John R. Hauser (2004), “’Listening-In’ to Find and Explore New Combinations of Cus-

tomer Needs,” Journal of Marketing, 68, (April), 72-87.

Weerahandi, Samaradasu and Soumyo Moitra (1995), “Using Survey Data to Predict Adoption

and Switching for Services,” Journal of Marketing Research, 32, 1, 85-96.

Wooldridge, Jeffrey M. (2002), “Econometric Analysis of Cross Section and Panel Data,” (Cam-

bridge, MA: MIT Press).

TABLE 1 ONE-STAGE ANALYSIS OF COMPETITIVE INFORMATION

CTMP with Misclassification Modeled

State at Tn-1: From “Do Not Consider USAM” From “Consider USAM”

State at Tn: to Consider USAM to Purchase USAM to Do Not Consider USAM to Purchase USAM

Full Model p value p value p value p value

Constant 0.005 b 0.10 0.022 b 0.10 0.013 b 0.09 0.084 a 0.01

Lagged Trust -0.043 c 0.84 0.607 c 0.42 -0.470 b 0.10 0.278 a 0.04

Competitive Test Drives

-0.071 c 0.97 -2.138 c 0.86 -0.058 c 0.94 -0.359 c 0.50

Customized Brochures

0.747 c 0.28 1.639 c 0.25 -4.832 c 0.60 0.189 c 0.53

Competitive Online Advisor

-1.130 c 0.21 6.075 c 0.55 -0.639 c 0.47 -0.138 c 0.64

Competitive Forum

0.365 c 0.57 0.288 c 0.85 0.767 c 0.35 0.185 c 0.53

Reduced Model

Constant 0.043 a <0.01 0.001 b 0.10 0.066 a <0.01 0.121 a <0.01

Lagged Trust 0.06 c 0.75 0.556 c 0.63 -0.502 a 0.01 0.255 a 0.04

a Significant at the 0.05 level (shown in bold). b Significant at the 0.10 level, but not 0.05 level (shown in bold italics)

TABLE 2 TRUST AS A FUNCTION OF COMPETITIVE INFORMATION (YEAR 1)

Key Variables Covariates Effect p value Effect p value

Intercept 0.640 b 0.06 Own USAM 0.000 c 0.84

Lagged Trust 0.860 a <0.01 Own American 0.021 c 0.59

Competitive Test Drives 0.380 a <0.01 Own Japanese -0.019 c 0.62

Customized Brochures 0.171 a <0.01 Period 3 -0.220 a <0.01

Competitive Advisor -0.057 c 0.39 Period 4 -0.295 a <0.01

Competitive Forum 0.045 c 0.22 Period 5 -0.127 a <0.01

Adjusted R2 0.749 c Period 6 -0.251 a <0.01


TABLE 3 SECOND-STAGE ANALYSIS BASED ON ESTIMATED LAGGED-TRUST (YEAR 1)




LIML p value p value p value p value

Constant 0.042 a <0.01 0.001 b 0.08 0.066 a <0.01 0.120 a <0.01

Lagged Trust (estimated)

0.123 c 0.60 -0.380 c 0.65 -0.471 b 0.06 0.275 b 0.06

Bootstrap (1,000 replicates)

Constant 0.100 a <0.01 0.001 c 0.36 0.156 a <0.01 0.129 a <0.01


0.209 c 0.13 -0.130 c 0.69 -0.325 a 0.03 0.275 a 0.01


TABLE 4 TRUST AS A FUNCTION OF COMPETITIVE INFORMATION (YEAR 2)

Key Variables Covariates Effect p value Effect p value

Intercept 0.902 a <0.01 Own USAM 0.132 c <0.01

Lagged Trust 0.824 a <0.01 Own American 0.023 c 0.19

Competitive Test Drives 0.089 b 0.10 Own Japanese -0.059 c <0.01

Competitive Brochures 0.050 b 0.04 Sex 0.037 a 0.02

USAM Booklets 0.017 c 0.52 Age -0.088 a 0.02

Competitive Advisor -0.020 c 0.39 Age-Sq/100 0.009 a 0.01

Competitive Forum 0.003 c 0.91 Period 3 0.038 c 0.15

Period 4 0.040 c 0.13

Period 5 0.028 c 0.30

Adjusted R2 0.700 c Period 6 0.050 b 0.06


TABLE 5 SECOND-STAGE ANALYSIS BASED ON ESTIMATED LAGGED-TRUST (YEAR 2)




LIML p value p value p value p value

Constant 0.025 a <0.01 0.001 a <0.01 0.099 a <0.01 0.006 a <0.01


0.989 a <0.01 0.468 c 0.36 -0.425 a <0.01 -0.018 c 0.96

Bootstrap (1,000 replicates)

Constant 0.072 a <0.01 0.001 a 0.01 0.188 a <0.01 0.006 a <0.01


0.614 a <0.01 0.474 a 0.05 -0.362 a <0.01 0.060 c 0.87


FIGURE 1 YEAR-1 COMPETITIVE-INFORMATION EXPERIMENTAL TREATMENTS

(a) Competitive Test Drive (b) Customized Brochures

Site contained over 60

dialogues averaging over 60 comments.

(c) Competitive Online Advisor (d) Competitive Community Forum

F

YEAR

(a

(c

CONTI

R-2 COMPE

a) My Auto A

c) Competitiv

NUOUS-TIIME FLOWIGURE 2

WS AMONGG BEHAVIOORAL STATTES

ETITIVE-IN

Advocate Ho

ve New-Veh

FFORMATIO

omepage (Op

hicle Advisor

IGURE 3 ON QUASI

pt-in)

-EXPERIMMENTAL TRREATMENTTS

(b) Competitive E-Brrochures

r

(d) Competitive Commmunity Foruum

Documents

Does Providing Competitive Information to Your Own Customers …ebusiness.mit.edu/research/papers/2010.04_Liberali_Urban... · 2010-08-05 · (USAM) to provide competitive. information