Experimentation in Market Reasearch

8/4/2019 Experimentation in Market Reasearch

1/86

Business Research Method

Experimentation


2/86

An Introduction to Experimental Design

Statistical studies can be classified as being either

experimental or observational.

In an experimental study, one or more factors(

Independent Variables) are controlled so that data

can be obtained about how the factors influencethe variables of interest(Dependent Variable).

In an observational study, no attempt is made to

control the factors. Cause-and-effect relationships are easier to

establish in experimental studies than in

observational studies.


3/86

Experiment

. Experiment: A research method inwhich conditions are controlled so that

one or more independent variables can

be manipulated and their effect ondependent variable is measured .

To study impact of a sports coach oncricket teams performance an

experiment can be conducted


4/86

Experiment

.-A research investigation in which conditionsare controlled

-One independent variable is manipulated

(sometimes more than one)-Its effect on a dependent variable is

measured

-To test a hypothesis


5/86

Some Definitions

Absolute Experiment: If we want to know

impact of a sports coach on cricket teamsperformance it is an absolute experiment

Comparative experiment :If we want toknow impact of one sports coach on cricket

teams performance compared to impact ofanother sports coach on cricket teamsperformance it is a comparative experiment

Experimental units: Subjects or entitiessubjected to various treatments are calledexperimental units


6/86

Some Definitions

- Experimental group: The group ofsubjects exposed

to an experimental treatment.Group of students provided with a sport coach

- Control Group :A group of subjects who are

exposed to the control conditions in an experiment,that is they are subjects not exposed to the

experimental treatment

Group of students not provided with a sports coach


7/86

Treatment

- Treatment: It refers toconditions to which

experimental groups aresubjected and control groups

are not subjected

Provision of sports coach


8/86

Extraneous Variable

Extraneous variable: Independent variables that

are not directly linked with the study but mayinfluence dependent variable:

If the study wants to test a hypothesis that a

relationship exists between academicachievement and self concept of children then

academic achievement is dependent and self

concept is independent variable.

Intelligence may also affect academic

achievement but it is not related to studys

purpose and hence it is an extraneous variable.


9/86

Experimental Design

Experimental Design is a set of

procedures specifying

--Manipulation of the Independent

Variable--Selection of Dependent Variable

--Experimental groups & Control groups

--Control Over Extraneous Variables


10/86

Manipulation of independent Variable

The experimenter has some degree of control over the

independent variable The variable is independent because its value can be

manipulated by the experimenter.

To test relationship between package design & sales

Expose customers to three package designs A,B &C.These packs are placed on shelves of select outlets

Consumers response is measured

Package design is independent variable ,which is

manipulated & there are 3 treatment levels A,B & C


11/86

Selection of Dependent Variable

Selection of dependent variable is another important

decision Sales volume of the product is considered as dependent

variable in package design example

Selecting dependent variable may not be easy in all

cases

To evaluate effectiveness of various ad programmes

dependent variable can be brand image, brand

awareness & product sales Purpose of the research helps in selection of

appropriate dependent variable


12/86

Experimental & Control Groups Experimental group is a group of test units exposed to

change in independent variable Control group is a group of test units not exposed to

change in independent variable

In package design example a group of supermarkets

(Experimental group) is selected & each package designis displayed for a month

Another group of supermarkets (Control group)continue to carry regular package design for that period

Sales of product are measured for both groups &difference in sales is analysed to find out impact ofdesign on sales


13/86

Controlling Extraneous Variables

Extraneous (Confounding)variables need to be controlled

otherwise it is difficult to find out

whether the change in dependent

variable is due to change in

independent variable or change inextraneous variable


14/86

Ways to control extraneous variable

Randomization Hold Conditions Constant

Matching Subjects

Design Control

Statistical Control


15/86

Randomisation Process of assigning test units randomly to

experimental treatments and assigning

experimental treatments randomly to test units It helps to spread effects of extraneous variable

equally over the test units

Most popular method

Not effective for small samples----Department stores assigned randomly toExperimental & Control groups

----If 3 versions of test commercials are to be

assigned to 3 groups then one commercial isselected at random & assigned to a randomlyselected group & process is repeated


16/86

Physical Control of Extraneous Variables

Hold Conditions Constant: Thelevel of extraneous variables is

kept constant throughout the

experiment

-- If price is the extraneous variable

prices are not changed duringexperimental period


17/86

Physical Control of Extraneous Variables

Matching Subjects: Judgmental sampling isused to assign test units to both

experimental group & control group so that

both groups are matched in terms of

characteristics of test units

---Department stores are matched on the

basis of annual sales, size and location. One

store from each matched pair would beassigned to each experimental group &

other to control group


18/86

Statistical Control

Extraneous variables affecting dependent

variable are identified & measured usingappropriate statistical tools like ANOVA. Then

effects of extraneous variable on dependent

variable are removed by statistical adjustments

If 5 departmental stores are used to study salesof products before & after in- house sales

promotion then impact of the characteristics of

the stores can be removed through use of

ANOVA-randomised block design using

department stores as blocks


19/86

Design Control

Selecting appropriate experimental

designs to conduct experiments helps incontrolling specific extraneous variables .

Experimental Designs are of 4 types

-Pre Experimental-True Experimental

-Quasi experimental

-Statistical


20/86

Experimental Validity Validity: Extent to which a research process

is accurate & reflects actual marketconditions

Internal Validity: It measure the extent of

change in dependent variable in anexperiment caused by manipulation ofindependent variable .( Not vitiated byextraneous variables)

External Validity :It measures to what extentinferences derived from experiment can begeneralised to real environment


21/86

Internal Validity

Control of extraneous variable is a

necessary condition for establishing

internal validity

Internal validity is the basic

minimum requirement of an

experiment

Six major types of extraneousvariables are threats to internal

validity


22/86

Factors Influencing Internal

Validity History

Maturation

Testing

Instrumentation

Selection

Mortality


23/86

Type of Extraneous Variable Example

History - Specific events in theenvironment between the Beforeand After measurement that arehistoric in nature (rare occurrence)

Maturation - Subjects change

during the course of the experimentdue to passage of time. Subjectsbecome tiredTesting - The Before measure alertsor sensitizes subject to nature of

experiment or second measure.

While test marketing

new farm equipmentarea gets affected byfloodsImpact of promotional

campaign for a car getsaffected by steep price risein petrol by Iraq warDrug trials carried over longeperiod of time results in som

effect on account ofphysiological changesQuestionnaire about thetraditional role of women

triggers enhanced awarenes


24/86

Instrument -Changes in

instrument to minmise testingeffect result in response bias

Selection

test units selected forexperimental group differ fromthose assigned to control group

Mortality - Sample attrition; somesubjects withdraw from experiment

New questions about

women are interpreteddifferently from earlierquestions.

Heavy users in experimentagroup & moderate & lowusers in control group

Subjects in TV viewing studdrop out before completion


25/86

External Validity

Experiments conducted in natural

settings offer more external

validity as compared to

experiments in controlledconditions.

Since generalisation is easier with

field conditions

i i


26/86

Experimental Environment Laboratory Environment: Experiments conducted in controlled

conditions

---Showing ads or products to select consumers in controlled

conditions----blind taste tests

Advantages-

Effects of extraneous variables can be minimised

-History effects eliminated

-Provides more internal validity

-time & cost effective

-reduces the risk of information about products & ideas being passedon to competitors

Disadvantages -Expts conducted in artificial conditions &results may not hold well

in actual conditions

-less external validity

-results influenced by testing effects ,Test units aware of being tested

& may not respond naturally


27/86

Field Environment Experiments conducted in natural settings

--Launching products in select regions

--Observing consumer behaviour regarding a POP display

in supermarkets

--analysing customer response to trial offers Advantages

-high degree of external validity

Disadvantages -Low degree of internal validity

-Require greater time& efforts and are expensive


28/86

Laboratory Experiment Field Experiment

Artificial-Low Realism

Few Extraneous

Variables

High control

Low Cost

Short Duration

Subjects Aware of

Participation

Natural-High Realism

Many Extraneous

Variables

Low control

High Cost

Long Duration

Subjects Unaware of

Participation


29/86

Different Basic Experimental Designs

Pre-Experimental Designs

True Experimental Designs

Quasi Experimental Designs

Statistical Designs

N i


30/86

Notations

X: Treatment or manipulation of independent

variable O: Observation or measurement of dependent

variable

R: Random assignment of test units to

experimental groups

EG : Experimental group exposed to experimental

treatment

CG :Control group not exposed to experimentaltreatment

Notations in the diagram are read from left to right


31/86

Conventions

Movement from left to right indicates

movement through time

Horizontal alignment of symbols implies

that all those symbols refer to specific

treatment group

Vertical alignment of symbol implies that

those symbols refer to activities or events

that occur simultaneously

Examples


32/86

Examples

X O1 O2 means that a given group of test

units was exposed to treatment variable X

and the response was measured at two

different points in time ,O1 and O2

R X1 O1

R X2 O2 means that two groups of

test units were randomly assigned to two

different treatment groups at the same time

and the dependent variable was measured in

two groups simultaneously


33/86

Pre-Experimental Designs

---They lack proper control mechanism to deal

with effects of extraneous variables

--- Randomization is absent

One Shot Design (After Only)

One Group Pretest-Posttest

Static Group Design


34/86


It involves exposing experimental group to treatment X

after which the measurement (O1) of dependent variable istaken .

EG : X O1

To measure effectiveness of a test commercial

TV programme containing test Ad (X) is selected

Telephone interviews are conducted of respondents whoreport having watched the TV programme previous night

Respondents are asked whether they recall having seen theAd. Measurements of Dependent variable are recalls (O1)

Recalls are compared with norm scores for interpretation.

One Shot Design (After Onl )


35/86


Drawbacks

No random assignment of test units Extraneous variables are not controlled

Internal validity low

There is no other measurement against

which O1 can be compared (what would

happen if X was absent)

More appropriate for exploratory studiesand not conclusive studies

O G P P


36/86

One Group Pretest -Posttest Exposing an experimental group to experimental treatment (X) and

taking measurements of dependent variable before(O1) and after(O2)the treatment

EG: O1 X O2

Difference between O1 and O2 measures the impact of treatment ondependent variable

To measure effectiveness of a test commercial

Respondents are recruited on the basis of convenience Administered a personal interview to measure attitude towards the

product

Then they watch a TV programme containing Commercial

After watching TV programme they are again administered a personal

interview to measure attitude towards product Effectiveness of test commercial is measured as O2-O1

Influenced by maturation,testing ,mortality, selection bias affecting IV


37/86

Static Group Design Two groups of test units are involved EG &CG

EG is exposed to treatment CG is not exposed

Measurements are taken for both groups after Expt.

EG: X O1

CG: O2

Difference between O1& O2 is effect of treatment To test effectiveness of TV commercial

Two groups of respondents are recruited

EG is exposed to TV programme containing Ad

Attitude towards product of both EG &CG are measured Effectiveness of test Ad is measured as O1-O2

Selection bias ,mortality affects results with less IV


38/86

True Experimental Designs

---Use randomisation & one or more controlgroups to reduce effect of extraneous variables

Pre testPost test Control Group Design

Posttest only Control Group DesignSolomon Four Group design

Pretest-Posttest Control Group Design


39/86

Pretest Posttest Control Group Design

Two groups of test units EG &CG are considered

Test units are assigned to these 2 groups randomly Pre test measurements of dependent variable are

taken for both groups

EG is exposed to treatment Post test measurements of dependent variable aretaken for both groups

EG: R O1 X O2

CG: R O3 O4

Treatment Effect (TE) is (O2-O1)-(O4-O3)

Pre Test Post Test Control Group Design


40/86

Pre Test Post Test Control Group Design

To measure effectiveness of a TV commercial

Sample of respondents is selected at random

Half of these would be randomly assigned to EG & other halfwould form CG

Both groups would be administered a questionnaire to obtain pre-test measurement on attitude towards the product

Only EG would be exposed to TV programme containingcommercial

Then respondents in both groups would be administered aquestionnaire to obtain post test measurements on attitude towardsproduct

Effectiveness measure is (O2-O1)-(O4-O3)

This design provides accurate results & addresses most ofextraneous variables accept testing effects

P tt t t t l C t l G D i


41/86

Posttest test only Control Group Design

Both EG & CG are formed by random assignment

of test units EG is exposed to experimental treatment CG is not

Post test measurement of dependent variable is

taken for both the groups EG: R X O1

CG: R O2

Treatment Effect (TE) equals O1-O2

P O l C l G D i


42/86

Posttest Only Control Group Design

To measure effectiveness of a TV commercial

Sample of respondents is selected at random Half of these would be randomly assigned to EG & other

half would form CG

Only EG would be exposed to TV programme containing

commercial Then respondents in both groups would be administered a

questionnaire to obtain post test measurements on attitudetowards product

Difference in attitudes of EG &CG is used aseffectiveness measure

P O l C l G D i


43/86

Posttest Only Control Group Design

Involves only 2 groups & one measurement

Significant advantage in terms of time ,cost& sample size

Selection bias & mortality which are the

major drawbacks can be controlled throughcarefully designed experimental procedures

Because of its simplicity it is most popular

in marketing design

Solomon Four Group Design


44/86

Solomon Four Group Design It involves 4 groups ,2 EGs & 2 CGs

Six measurements are taken 2 pretest &4 posttest

Also known as 4 group 6 study design EG: R O1 X O2

CG: R O3 O4

EG: R X O5

CG: R O6

Design provides various observations which can beanalysed

O2-O1; O4-O3; O2-O4; O5-O6

Addresses all extraneous variables but is expensive& consumes time & efforts

Q i E i t l D i


45/86


Are used when randomisation is not possible & timing oftest presentation lacks control of researcher

Not as effective as true designs but better than pre-design

Prominent is Time Series Design

A series of measurements are taken on dependent variablefor a group of test units before & after the experimentaltreatment

O1,O2,O3 X O4, O5, O6

Trends before treatment are compared with trends aftertreatment to determine effectiveness of treatment

Aids in identifying permanent change & temporary change

Threats on account of history & instrumentation



46/86


To test effectiveness of a test commercial

Broadcasting test commercial a predetermined number oftimes & examining data from a pre-existing test panel

Scheduling of test commercial can be controlled but it isuncertain when or whether panel members are exposed toit

Purchases of panel members before ,during and after thecampaign are examined to find out whether testcommercial has a short term effect ,long term effect or noeffect

To find out impact of price change on sales of a product, aseries of observations are taken before the price change&after the price change. Trends before treatment arecompared with trends after treatment to determine change


47/86

Multiple Time Series Design

Similar to time series except that another group

of test units is added to serve as control group EG:O1,O2,O3 X O4,O5,O6

CG:O7,O8,O9 O10,O11,O12

Test commercial would be shown only in a few

of test cities

Panel members in these cities would form EG

Panel members in the cities where commercial

was not shown would constitute control group


48/86

One-Shot Design

Internal Validity Problems

History

weak

Maturation weak

Testing

not relevant

Instrumentation

not relevant

Selection weak

Mortality

weak


49/86

One-Group Pretest-Posttest


History

weak

Maturation weak

Testing

weak

Instrumentation

weak

Selection controlled

Mortality

controlled


50/86

Static-Group Design


History

controlled

Maturation possible source of

concern

Testing

controlled

Instrumentation

controlled

Selection weak

Mortality

weak


51/86

Pretest-Posttest Control

Internal Validity Problems History

controlled

Maturation controlled

Testing

controlled

Instrumentation

controlled


Mortality

controlled


52/86

Posttest-Only Control


History

controlled


Testing

controlled

Instrumentation

controlled


Mortality

controlled


53/86

Solomon Four-Group Design

Internal Validity Problems History

controlled


Testing

controlled

Instrumentation

controlled


Mortality

controlled

Statistical Designs


54/86

Statistical Designs

Aids in measuring effect of more than one

independent variable Helps in isolating effects of most

extraneous variables

Four prominent designs in this category

-Completely Randomised

-Randomised Block

-Latin Square

-Factorial


55/86

Advanced Experimental Designs are

More Complex

Completely randomized

Randomized block design

Latin square

Factorial


56/86

Completely Randomized Design

An experimental design that uses a random process

to assign subjects (test units) and treatments toinvestigate the effects ofonly one independent

variable (factor).

There are n test units & ktreatments (levels offactor)

The n test units are assigned to k treatments

randomly. Post test measurements are evaluated


57/86


3 test cities are randomly exposed to 3 test

commercials

Then measurements are taken on sales in these 3

cities

Differences in sales in 3 cities are analyzed to seewhich AD is effective

Example: Home Products, Inc.


58/86


Home Products, Inc. is considering marketing along-lasting car wax. Three different waxes (Type

1, Type 2,and Type 3) have been developed.

In order to test the durability of these waxes, 5new cars were waxed with Type 1, 5 with Type 2,

and 5 with Type 3. Each car was then repeatedlyrun through an automatic carwash until the waxcoating showed signs of deterioration. Thenumber of times each car went through the

carwash is shown on the next slide.Home Products, Inc. must decide which wax to

market. Are the three waxes equally effective?



59/86

p

WAX WAX WAX

Observation Type 1 Type 2 Type 3

1 48 73 51

2 54 63 63

3 57 66 61

4 54 64 54

5 62 74 56

Sample Mean 55 68 57Sample Variance 26.0 26.5 24.5



60/86


ANOVA Table

Source of Sum of Degrees of Mean

Variation Squares Freedom Squares F

Treatments 490 2 245 9.55

Error 308 12 25.667

Total 798 14


E l E t Oil C


61/86

Example: Eastern Oil Co. Completely Randomized Design

Rejection RuleAssuming = .05, F.05 = 3.88 (2 d.f. numerator

and 12 d.f. denominator). RejectH0 ifF>

3.88.

Test Statistic

F= MSTR/MSE = 245/25.67= 9.55

Conclusion

Since 9.55 > 3.88, we rejectH0 and conclude

that the miles per gallon ratings differ for the

three gasoline blends.



62/86

Completely Randomized Design To evaluate efficacy of a weight loss drug

A sample of 40 consumers is selected

Assigned randomly to two treatment levels 25 to treatment 1 & 15 to treatment 2

Consumers in group 1 are asked to take drug for onemonth

Consumers in group 2 are not given any drug After experiment measurements are taken for both groups

& differences if any are analysed to find out effectivenessof the drug

Applicable when test units are homogeneous Extraneous variables can be controlled

A single variable is evaluated


63/86

Randomized Block Design

An extension of the completely randomized

design in which a single extraneous variablethat might affect test units response to the

treatment has been identified and the effects

of this variable are isolated by blocking outits effects.

Example Randomised Block Design


64/86


A consumer product company wants to examine

impact of price change on sales of its newly

developed health drink

Store type is extraneous variable which would also

affect product sales

Company has decided to conduct experiment using 3price levels & 3 store types

Nine retail outlets are segregated according to store

type Treatment levels (prices) are assigned randomly to

each test unit ( retail Outlet)


65/86


Price

Change

Drug Store Supermarkets Malls

Rs 10

Rs 20

Rs 30


66/86

E l E t Oil C


67/86

Example: Eastern Oil Co.

utomobile Type of Gasoline (Treatment) Blocks

(Block) Blend X Blend Y Blend Z Means

1 31 30 30 30.333

2 30 29 29 29.333

3 29 29 28 28.667

4 33 31 29 31.000

5 26 25 26 25.667

reatment

Means 29.8 28.8 28.4

The ANOVA Procedure


68/86

he ANOVA procedure for the randomized block desig

equires us to partition the sum of squares total (SST) in

hree groups: sum of squares due to treatments, sum of

quares due to blocks, and sum of squares due to error.

he formula for this partitioning is

SST = SSTR + SSBL + SSE

he total degrees of freedom, n - 1, are partitioned suchhat k- 1 degrees of freedom go to treatments,

- 1 go to blocks, and (k- 1)(b - 1) go to the error term.

The ANOVA Procedure

Example: Eastern Oil Co


69/86

Example: Eastern Oil Co.


Mean Square Due to TreatmentsThe overall sample mean is 29. Thus,

SSTR = 5[(29.8 - 29)2 + (28.8 - 29)2 + (28.4 - 29)2] = 5.2

MSTR = 5.2/(3 - 1) = 2.6 Mean Square Due to Blocks

SSBL = 3[(30.333 - 29)2 + . . . + (25.667 - 29)2] = 51.33

MSBL = 51.33/(5 - 1) = 12.8

Mean Square Due to Error

SSE = 62 - 5.2 - 51.33 = 5.47

MSE = 5.47/[(3 - 1)(5 - 1)] = .68



70/86


ANOVA Table

Source of Sum of Degrees of Mean

Variation Squares Freedom Squares F

Treatments 5.2 2 2.6 3.82

Block 51.3 4 12.8

Error 5.5 8 0.68

Total 62 14

p


71/86


72/86

Factorial Design

An experiment that investigates the

interaction of two or more independentvariables on a single dependent variable.


73/86

F i l D i


74/86

Factorial Design Researcher is investigating believability of Ads on 0-100

scale Two magazine Ads A & B are to be compared

Gender of the reader is another factor

This is a 2x2 factorial experiment Permits to test 3 hypothesis

Which Ad is more believable (Main)

Which gender tends to believe magazine Ads more(Main)

Which gender finds which ad more

believable(Interaction)


75/86

Men

Women

Ad A Ad B

65

65

70 60

Main Effectsof Gender

Main Effects of Ad

>

2 x 2 Factorial Design


76/86

Price Red Gold

$25 Cell 1 Cell 4$30 Cell 2 Cell 5$35 Cell 3 Cell 6

Package Design

Factorial Design -- Roller Skates


77/86

Latin Square Design

A balanced, two-way classification scheme

that attempts to control or block out the

effect oftwo extraneous factors byrestricting randomization with respect to the

row and column effects.

Latin Square Example


78/86

To find out impact of three different Ads on sales

Pricing & income levels of consumers are EV

In latin square levels of all the 3 variables must be same (2EV and 1 INDEP)

A table is then developed with levels of one EVrepresenting rows ,levels of second EV representing

columns . Levels of independent variable (treatments) are exposed toeach cell on a random basis so that each row has alltreatments & each column has all treatments

Treatment effect is determined for each cell

Analysis shows which treatment level influences dependentvariable more


79/86

Latin Square Example

Pricing

Levels

Low

Income

Middle

Income

High

Income

Rs 10000 Ad-B Ad-A Ad-C

Rs 12000 Ad-C Ad-B Ad-A

Rs 14000 Ad-A Ad-C Ad-B

L ti S


80/86

Latin Square

A, B & C identify three treatments

Rows & Columns of the table identify

confounding factors (extraneous variables)

A taste test may be confounded by order of testing

A taste test may also be confounded by individual

test preference

To control for these 2 factors each subject is

exposed to every treatment & order in whichsubjects test is randomised.


81/86

1 2 3

1 A B C2 B C A3 C A B

Order of Usage

SU

BJECT

Latin Square


82/86

Latin Square

Interaction effects are assumed to be minimal or

absent

Number of treatment levels for both confounding

factors must be equal

Suppose a retail grocery chain wishes to controlfor shelf space & city where product is sold

It markets in only 3 cities & wishes to experiment

with 4 levels of shelf space

Having unequal number of levels for each factor

will eliminate latin square as a possibility


83/86

TEST MARKETING

Controlled experimentation

Not just tryingsomething

out

But scientifictesting


84/86

Test Marketing

An experimental procedure that provides an

opportunity to test a new product or a new

marketing plan under realistic marketconditions to measure sales or profit

potential.


85/86

Selecting a Test Market

Population size

Demographic composition

Lifestyle considerations

Competitive situation

Media

Self-contained trading area

Secrecy

C l M h d f T


86/86

Control Method of Test

Marketing Small city Low chance of being detected

Distribution is forced (guaranteed)

Documents

Experimentation in Market Reasearch