Embed Size (px)
Citation preview
Chapter Seven
Causal Research Design:Experimentation
7-2
Concept of Causality
A statement such as "X causes Y " will have thefollowing meaning to an ordinary person and to ascientist.
____________________________________________________Ordinary Meaning Scientific Meaning
____________________________________________________X is the only cause of Y. X is only one of a number of
possible causes of Y.
X must always lead to Y The occurrence of X makes the (X is a deterministic occurrence of Y more probablecause of Y). (X is a probabilistic cause of Y). It is possible to prove We can never prove that X is athat X is a cause of Y. cause of Y. At best, we can
infer that X is a cause of Y.____________________________________________________
7-3Types of Evidence That Supports a Causal Inference
Concomitant variation is the extent to which a cause, X, and an effect, Y, occur together or vary together in the way predicted by the hypothesis under consideration.
The Time order of occurrence condition states that the causing event must occur either before or simultaneously with the effect; it cannot occur afterwards e.g. in-store service & sales
The Absence of other possible causal factors means that the factor or variable being investigated should be the only possible causal explanation e.g pricing, advertising, quality, competition etc.
X—the presumed cause (independent variable)Y—the presumed effect (dependent variable)
7-4Evidence of Concomitant Variation betweenPurchase of Fashion Clothing influenced by Education
High
High Low
363 (73%) 137 (27%)
322 (64%) 178 (36%)
Purchase of Fashion Clothing, Y
500 (100%)
500 (100%)Low
Ed
uca
tion
, X
7-5Evidence of Concomitant Variation betweenPurchase of Fashion Clothing influenced by Education
Based on evidence can we say high education causes high purchase of fashion clothing?
Certainly not! We can only say that the association makes the hypothesis
more tenable, it does not prove it There may be other causal factors e.g. income Next example we have kept effect of income constant We see that difference between high & low education
respondents reduced considerably Evidence of concomitant variation, time order of occurrence,
elimination of other causal factors even if combined still do not demonstrate conclusively that a causal relationship exists
However if all evidence is strong & consistent, it may be reasonable to conclude that there is a causal relationship
7-6Purchase of Fashion Clothing ByIncome and Education
Low Income
Purchase
High Low
High
LowEd
ucati
on
200 (100%)
300 (100%)
300
200
122 (61%)
171 (57%)
78 (39%)
129 (43%)
High Income
Purchase
High
High
Low
Low
241 (80%)
151 (76%)
59 (20%)
49 (24%)
Ed
ucati
on
7-7
Definitions and Concepts Independent variables or treatments are variables
or alternatives that are manipulated and whose effects are measured and compared e.g. price levels.
Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g. consumers or stores.
Dependent variables are the variables which measure the effect of the independent variables on the test units, e.g. sales, profits, and market shares.
Extraneous variables are all variables other than the independent variables that affect the response of the test units, e.g. store size, store location, and competitive effort.
7-8
Experiment
A research investigation in which conditions are controlled
One independent variable is manipulated
Its effect on a dependent variable is measured
To test a hypothesis
7-9
Types of Experiments
Scientific investigation in which an investigator manipulates and controls one or more independent variables and observes the degree to which the dependent variable or variables change
Experiment
Field ExperimentResearch study in a realistic situation in which one or more independent variables are manipulated by the experimenter under as carefully controlled conditions as the situation will permit
Research investigation in which Investigator creates a situation with exact conditions so as to control some, and manipulate other, variables
Laboratory Experiment
7-10
Laboratory Experiment Field Experiment
Artificial-Low Realism
Few ExtraneousVariables
High control
Low Cost
Short Duration
Subjects Aware ofParticipation
Natural-High Realism
Many ExtraneousVariables
Low control
High Cost
Long Duration
Subjects Unaware ofParticipation
7-11Basic Issues of Experimental Design
Manipulation of the Independent Variable
Selection of Dependent Variable Assignment of Subjects (or other
Test Units) Control Over Extraneous Variables
7-12
Independent Variable:The experimenter has some degree of control over the independent variable. The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be.
7-13
Experiment Treatment
Alternative manipulations of the independent
variable being investigated
7-14
Dependent Variable
Its value is expected to be dependent on the experimenter’s manipulation
• Criterion or standard by which the results are judged e.g... sales volume, awareness, recall etc
7-15
Test Units
Subjects or entities whose response to the experimental treatment are measured or observed.
7-16
Symbols X = independent variable, treatment, or event, the effects of which are to
be determined O = observation or measurement of the dependent variable on the test
units R = the random assignment of test units to separate treatments Movement from left to right indicates movement through time E.g. X O1 O2 Means test units were exposed to treatment/independent variable (X) &
response was measured at two different pts of time O1 & O2 Horizontal alignment of symbols implies that all those symbols refer to a
specific treatment group Vertical alignment of symbols implies that those symbols refer to
activities or events that occur simultaneously E.g. R X1 O1 R X2 O2 Means that two groups of test units were randomly assigned to two different
treatment groups at the same time & the dependent variable was measured in the two groups simultaneously
7-17
Validity in Experimentation
Two goals in experimentation: Draw valid conclusions about the
effects of independent variables on the study group (internal validity)
Make generalizations to a larger population of interest (external validity)
7-18
Validity in Experimentation Internal validity refers to whether the
manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity.
External validity refers to whether the cause-and-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?
7-19Factors Influencing Internal Validity
History Maturation Testing Instrumentation Selection Mortality
7-20
Type of Extraneous Variable
History (H)- Specific events that are external to the experiment but occur at the same time as the experiment & may affect the dependent variable e.g. A major employer closes its plant in test market area
Maturation (MA)- Subjects/Test units change during the course of the experiment e.g. Subjects become tired, stores change in terms of layout, décor, composition etc
Testing - The Before measure alerts or sensitizes subject to nature of experiment or second measure.
- Main testing effect (MT), when a prior observation affects a latter observation e.g. Measure effect of advt on attitudes towards a brand, respondents given a questionnaire measuring background info & attitude, then exposed to ad & again asked qs, may be no change in their attitude as they may try to maintain consistency or may show change just because it is being measured
- Interactive testing effect (IT), a prior measurement affects the test units response to the independent variable e.g. When people are asked to indicate their attitudes towards a brand they become sensitized /aware of the brand & pay more attention to the test commercial than they normally would
7-21
Type of Extraneous Variable
Instrument (I)- Changes in instrument result in response bias e.g. in advt experiment if newly designed questionnaire is used to measure post treatment attitudes.
Statistical Regression(SR) – when test units with extreme scores move closer to the average score during the experiment e.g. in advt experiment if some respondents had either very favorable of very unfavorable attitudes, post treatment their attitudes might have moved towards average. People with extreme attitudes have more room for change, so variation more likely
Selection Bias(SB) – improper assignment of test units to treatment conditions e.g. different store sizes
Mortality(MO)- loss of test units while the experiment is in progress
What are the Different Basic Experimental Designs?
7-23A Classification of Experimental Designs
Pre-experimental designs do not employ randomization procedures to control for extraneous factors: the one-shot case study, the one-group pretest-posttest design, and the static-group.
In true experimental designs, the researcher can randomly assign test units to experimental groups and treatments to experimental groups: the pretest-posttest control group design, the posttest-only control group design, and the Solomon four-group design.
7-24A Classification of Experimental Designs
Quasi-experimental designs result when the researcher is unable to achieve full manipulation of scheduling or allocation of treatments to test units but can still apply part of the apparatus of true experimentation: time series and multiple time series designs.
A statistical design is a series of basic experiments that allows for statistical control and analysis of external variables: randomized block design, Latin square design, and factorial designs.
7-25A Classification of Experimental Designs
Pre-experimental
One-Shot Case Study
One Group Pretest-Posttest
Static Group
True Experiment
alPretest-Posttest Control Group
Posttest: Only Control Group
Solomon Four-Group
Quasi Experimental
Time Series
Multiple Time Series
Statistical
Randomized Blocks
Latin Square
Factorial Design
Experimental Designs
7-26Pre-experimental designsOne-Shot Case Study
Measure effectiveness of a test commercial (X)for a deptt. store X 01
A single group of test units is exposed to a treatment X. A single measurement on the dependent variable is taken (01). There is no random assignment of test units. The one-shot case study is more appropriate for exploratory than
for conclusive research. E.g. measure the effectiveness of a test commercial for a brand Test commercial is X, Dependent variable(Os) are unaided &
aided recall of the ad
7-27Pre-experimental designs One-Group Pretest-Posttest Design
01 X 02 A group of test units is measured twice. There is no control group. First a pre-treatment measure is taken(O1), then
group is exposed to treatment (X) Finally a post-treatment measure is taken (O2),
e.g. sales after the ad The treatment effect is computed as
02 – 01. The validity of this conclusion is questionable
since extraneous variables are largely uncontrolled.
7-28One-Group Pretest-Posttest Design: Example
Measure effectiveness of a test commercial for a deptt. store
Respondents first administered a personal interview to measure attitudes towards the store (O1)
Then they watch the test commercial (X) After that they are again administered a
personal interview to measure attitudes towards the store (O2)
Effectiveness of the commercial is measured as O2 - O1
7-29Pre-experimental designsStatic Group Design
EG: X 01
CG: 02
A two-group design. The experimental group (EG) is exposed to
the treatment, and the control group (CG) is not.
Measurements on both groups are made only after the treatment.
Test units are not assigned at random. The treatment effect would be measured
as 01 - 02.
7-30
Static Group Design: Example
To measure the effectiveness of a test commercial for a deptt. Store
Two groups of respondents would be recruited on the basis of convenience
Only the experimental group would be exposed to the test commercial
Then attitudes towards the store of both experimental & control group would be measured
Effectiveness would be measured as O1 – O2
7-31True Experimental Designs: Pretest-Posttest Control Group Design
Distinguishing feature of this design compared to pre-experimental design is randomization
Here the researcher randomly assigns test units to experimental groups
The pretest-posttest control group design, the posttest-only control group design, and the Solomon four-group design.
7-32True Experimental Designs: Pretest-Posttest Control Group Design
EG: R 01 X 02
CG: R 03 04
Test units are randomly assigned to either the experimental (exposed to test ad) or the control group (not exposed).
A pretreatment measure (attitude towards a brand) is taken on each group.
Only the experimental group is exposed to the treatment but post-test measures are taken on both.
The treatment effect (TE) is measured as:(02 - 01) - (04 - 03).
Selection bias is eliminated by randomization.
7-33Posttest-Only Control Group Design
EG : R X 01
CG : R 02
Does not involve any pre measurement Sample randomly split, half in EG (shown the advt.) & other
half in CG (not shown), then questionnaire administered to both to obtain posttest measures on attitudes towards the brand, difference in attitudes of the EG & CG used as a measure of the effectiveness of the advt.
The treatment effect is obtained byTE = 01 - 02
Except for pre-measurement, the implementation of this design is very similar to that of the pretest-posttest control group design.
Simple to implement
7-34
Solomon Four Group Design
Experimental Group 1: R O1 X O2
Control Group 1: R O3 O4
Experimental Group 2: R X O5
Control Group 2: R O6
When we need to examine changes in attitudes of respondents then this design is used
Expensive & time consuming hence not popular
7-35
Quasi-Experimental Designs
There is no randomization of test units to treatments: Researcher can control when measurements are taken & on whom
Researcher lacks control over the scheduling of the treatments & also is unable to expose test units to treatments randomly
Quicker & less expensiveTime series & Multiple time series designs
7-36
Time series & Multiple Time Series Design
Time Series Design: series of periodic measurements on the dependent variable for a group of test units
01 02 03 04 05 X 06 07 08 09 010
Multiple Time Series Design: similar to time series design except that another group of test units is added to serve as a control group
EG : 01 02 03 04 05 X 06 07 08 09 010
CG : 01 02 03 04 05 06 07 08 09 010
If the control group is carefully selected, this design can be an improvement over the simple time series experiment.
Can test the treatment effect twice: against the pretreatment measurements in the experimental group and against the control group.
7-37Multiple Time Series Design: Example
Examine the buildup effect of increased advertising One group of households was the EG & equivalent group was
CG Groups were matched on demographic variables Data collected for 76 weeks Both recd same level of advt for first 52 weeks for the brand
in qs Next 24 weeks, experimental group (EG) exposed to twice as
much advt as the Control group (CG) Results indicated that buildup effect of advt was immediate
with a duration of the order of the purchase cycle Useful for selecting advt timing patterns
7-38
Statistical Designs
Statistical designs consist of a series of basic experiments that allow for statistical control and analysis of external variables and offer the following advantages:
The effects of more than one independent variable can be measured.
Specific extraneous variables can be statistically controlled.
Economical designs can be formulated when each test unit is measured more than once.
The most common statistical designs are the randomized block design, the Latin square design, and the factorial design.
7-39
Randomized Block Design
Is useful when there is only one major external variable, such as store size or income, that might influence the dependent variable & needs to be blocked
The test units are blocked, or grouped, on the basis of the external variable e.g. Impact of humor on the effectiveness of advertising for store Z- store patronage identified as the blocking variable
The researcher must be able to identify & measure the blocking variable
By blocking, the researcher ensures that the various experimental and control groups are matched closely on the external variable.
Limitation is that researcher can control for only one external variable, when more than one variable must be controlled must use Latin square or factorial designs
7-40
Randomized Block Design
Treatment Groups Block Store Commercial Commercial Commercial Number Patronage A B C 1 Heavy A B C 2 Medium A B C 3 Low A B C 4 None A B C
7-41
Latin Square Design Allows the researcher to statistically control two non interacting
external variables as well as to manipulate the independent variable. E.g. wants to control two variables- store patronage & interest in store to measure impact of humor on the effectiveness of advertising for store Z
Each external or blocking variable is divided into an equal number of blocks, or levels.
The independent variable is also divided into the same number of levels- assignments of 3 test commercials can be made- A, B, C
To implement this design store patronage would also have to be blocked at three rather than four levels (e.g. by combining the low & non patrons into a single block)
A Latin square is conceptualized as a table with the rows and columns representing the blocks in the two external variables.
The levels of the independent variable are assigned to the cells in the table.
The assignment rule is that each level of the independent variable should appear only once in each row and each column
7-42
Latin Square Design
Interest in the Store Store Patronage High Medium
Low
Heavy B A C Medium C B
A Low and none A C
B
7-43
Factorial Design: Example Used to measure the effects of two or more independent variables at
various levels. Allows for interactions between variables- interaction takes place when the
simultaneous effect of two or more variables is diff from the sum of their separate effects e.g. a person may prefer coffee & fav. temperature may be cold but does not mean he likes cold coffee
In a two-factor design, each level of one variable represents a row and each level of another variable represents a column
A cell for every possible combination of treatment variable Suppose in addition to examining effect of humor on advt, researcher was
also interested in simultaneously examining the amount of store info in an advt & its impact- store info is also varied at 3 levels. This would require a 3x3 = 9 cells
Thus 9 diff. commercials will be produced, each having a specific level of store info & amount of humor
Respondents would be randomly selected & randomly assigned to the 9 cells & respondents in each cell would receive a specific treatment combination
E.g. respondents in upper left hand corner will view an advt that had no humor & low store info
7-44
Factorial Design
Amount of Humor Amount of Store No Medium High Information Humor Humor Humor
Low A B C Medium D E FHigh G H I
7-45Laboratory versus Field Experiments
Factor Laboratory Field
Environment Artificial RealisticControl High Low Reactive Error High Low Demand Artifacts High Low Internal Validity High LowExternal Validity Low HighTime Short LongNumber of Units Small LargeEase of Implementation High Low Cost LowHigh
7-46
Limitations of Experimentation Experiments can be time consuming,
particularly if the researcher is interested in measuring the long-term effects.
Experiments are often expensive. The requirements of experimental group, control group, and multiple measurements significantly add to the cost of research.
Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment.
Competitors may deliberately contaminate the results of a field experiment.
7-47
Activities
You are a marketing research manager for the Coca-Cola Company. The company would like to determine whether it should increase, decrease, or maintain the current level of advertising dollars spent on Coke.
Design a field experiment to address this issue. What potential difficulties do you see in
conducting the experiment just described? What assistance would you require from the
Coca- Cola management to overcome these difficulties?
7-48
Activities
A pro-life group wanted to test the effectiveness of an anti-abortion commercial. Two random samples, each of 250 respondents, were recruited in Atlanta. One group was shown the anti-abortion commercial. Then attitudes toward abortion were measured for respondents in both groups.
A. Identify the independent & dependent variables in this experiment
B. Design a field experiment to address this issue.
7-49
Fieldwork
Select two different advertisements of any brand
Design and conduct an experiment to determine which ad is more effective.
Use a student sample with 10 students being exposed to each ad (treatment condition).
Develop your own measures of advertising effectiveness in this context
