19
DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment is a study in which a treatment, procedure, or program is intentionally introduced and a result or outcome is observed. True experiments have four elements: manipulation, control, random assignment, and random selection. The most important of these elements are manipulation and control. Manipulation means that something is purposefully changed by the researcher in the environment. Control is used to prevent outside factors from influencing the study outcome. When something is manipulated and controlled and then the outcome happens, it makes us more confident that the manipulation “caused” the outcome. In addition, experiments involve highly controlled and systematic procedures in an effort to minimize error and bias which also increases our confidence that the manipulation “caused” the outcome. Another key element of a true experiment is random assignment. Random assignment means that if there are groups or treatments in the experiment, participants are assigned to these groups or treatments, or randomly (like the flip of a coin). This means that no matter who the participant is, he/she has an equal chance of getting into all of the groups or treatments in an experiment. This process helps to ensure that the groups or treatments are similar at the beginning of the study so that there is more confidence that the manipulation (group or treatment) “caused” the outcome. Random selection is a form of sampling where a representative group of research participants is selected from a larger group by chance. This can be done by identifying all of the possible candidates for study participation (e.g., people attending the County fair on a Tuesday) and randomly choosing a subset to participate (e.g., selecting every 10th person who comes through the gate). This allows for each person to have an equal chance of participating in the study. Allowing each person in the group an equal chance to participate increases the chance that the smaller group possesses characteristics similar to the larger group. This produces findings that are more likely to be representative of and applicable to the larger group. Experimental studies – Example 1 An investigator wants to evaluate whether a new technique to teach math to elementary school students is more effective than the standard teaching method. Using an experimental design, the investigator divides the class randomly (by chance) into two groups and calls them “group A” and “group B.” The students cannot choose their own group. The random assignment process results in two groups that should share equal characteristics at the beginning of the experiment. In group A, the teacher uses a new teaching method to teach the math lesson. In group B, the teacher uses a standard teaching method to teach the math lesson. The investigator compares test scores at the end of the semester to evaluate the success of the new teaching method compared to the standard teaching method. At the end of the study, the results indicated that the students in the new teaching method group scored significantly higher on their final exam than the students in the standard teaching group. Experimental studies – Example 2 A fitness instructor wants to test the effectiveness of a performance-enhancing herbal supplement on students in her exercise class. To create experimental groups that are similar at the beginning of the study, the students are assigned into two groups at random (they cannot choose which group they are in). Students in both groups are given a pill to take every day, but they do not know whether the pill is a placebo (sugar pill) or the herbal supplement. The instructor gives Group A the herbal supplement and Group B receives the placebo (sugar pill). The students' fitness level is compared before

DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

  • Upload
    lamdang

  • View
    225

  • Download
    4

Embed Size (px)

Citation preview

Page 1: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE

Unlike a descriptive study, an experiment is a study in which a treatment, procedure, or program is intentionally introduced and a result or outcome is observed.

True experiments have four elements: manipulation, control, random assignment, and random selection. The most important of these elements are manipulation and control. Manipulation means that something is purposefully changed by the researcher in the environment. Control is used to prevent outside factors from influencing the study outcome. When something is manipulated and controlled and then the outcome happens, it makes us more confident that the manipulation “caused” the outcome. In addition, experiments involve highly controlled and systematic procedures in an effort to minimize error and bias which also increases our confidence that the manipulation “caused” the outcome.

Another key element of a true experiment is random assignment. Random assignment means that if there are groups or treatments in the experiment, participants are assigned to these groups or treatments, or randomly (like the flip of a coin). This means that no matter who the participant is, he/she has an equal chance of getting into all of the groups or treatments in an experiment. This process helps to ensure that the groups or treatments are similar at the beginning of the study so that there is more confidence that the manipulation (group or treatment) “caused” the outcome.

Random selection is a form of sampling where a representative group of research participants is selected from a larger group by chance. This can be done by identifying all of the possible candidates for study participation (e.g., people attending the County fair on a Tuesday) and randomly choosing a subset to participate (e.g., selecting every 10th person who comes through the gate). This allows for each person to have an equal chance of participating in the study. Allowing each person in the group an equal chance to participate increases the chance that the smaller group possesses characteristics similar to the larger group. This produces findings that are more likely to be representative of and applicable to the larger group.

Experimental studies – Example 1

An investigator wants to evaluate whether a new technique to teach math to elementary school students is more effective than the standard teaching method. Using an experimental design, the investigator divides the class randomly (by chance) into two groups and calls them “group A” and “group B.” The students cannot choose their own group. The random assignment process results in two groups that should share equal characteristics at the beginning of the experiment. In group A, the teacher uses a new teaching method to teach the math lesson. In group B, the teacher uses a standard teaching method to teach the math lesson. The investigator compares test scores at the end of the semester to evaluate the success of the new teaching method compared to the standard teaching method. At the end of the study, the results indicated that the students in the new teaching method group scored significantly higher on their final exam than the students in the standard teaching group.

Experimental studies – Example 2

A fitness instructor wants to test the effectiveness of a performance-enhancing herbal supplement on students in her exercise class. To create experimental groups that are similar at the beginning of the study, the students are assigned into two groups at random (they cannot choose which group they are in). Students in both groups are given a pill to take every day, but they do not know whether the pill is a placebo (sugar pill) or the herbal supplement. The instructor gives Group A the herbal supplement and Group B receives the placebo (sugar pill). The students' fitness level is compared before

Page 2: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

and after six weeks of consuming the supplement or the sugar pill. No differences in performance ability were found between the two groups suggesting that the herbal supplement was not effective.

Discussion questions 1. What makes both of these studies experimental? 2. What type of information might the investigator collect in these two studies to see if the treatment

(e.g. new teaching method or herbal supplement) is effective? 3. Can the researcher establish cause and effect in either or both of these two studies? 4. What would happen if the researcher allowed the students to study together or talk about the

different methods that were being used to teach the math lesson? Would this be a good or a bad idea? How would this influence the study results?

5. What if the fitness instructor allowed participants to take other herbal supplements in addition to the supplements being tested? Would this be a good or a bad idea? How would this influence the study results?

Reading assignment:

1. Threats to internal and external validity of experiments in the social sciences.

2. Quasi-experimental designs

Terminologies Experiment – a well-defined process or planned inquiry undertaken to obtain new facts or to

confirm or deny the results of previous studies Experimental unit/subject – is the object on which a measurement or measurements is taken; it

is the material on which the treatment is applied Factor – an independent variable whose effect on the dependent variable we want to study Level – is the intensity setting of a factor Treatment – level of a factor or combination of factors whose effect we want to measure and

compare with others Replication – number of times a treatment is applied to experimental units in an experiment Response – is the dependent variable being measured by the experimenter.

Principles of Experimental Design

1. Replication: application of each individual level of the factor to multiple subjects 2. Randomization: random assignment of the experimental units to levels of the factor 3. Local control: treatments should be applied uniformly and under standardized conditions. That

is, make the observations as homogeneous as possible so that error due to one or more assignable causes may be removed from the experimental error

The Completely Randomized Design: A One-way Classification The simplest experimental design and is a direct extension of the independent sampling design.

Samples are randomly selected from t independently populations (random selection). The experiment

involves only one factor. Each level of the factor represents a treatment. In most cases, subjects are

randomly assigned to the treatments (random assignment).

Example In an experiment to determine the effect of nutrition on the attention spans of elementary students, a group of 15 students were randomly assigned to each of three meal plans: no breakfast, light

Page 3: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

breakfast, and full breakfast. Their attention spans (in minutes) were recorded during a morning reading period.

The Randomized Complete Block Design: A Two-way Classification Recall that the completely randomized design is a generalization of the two independent samples design and is meant to be used when the experimental units are quite similar or homogeneous in their makeup and when there is only one factor- the treatment- that might influence the response. Any other variation in the response is due to random variation or experimental error. Sometimes it is clear to the researcher that experimental units are not homogeneous. Experimental units often add their own variability to the response. Although the researcher is not really interested in this source of variation, but rather in some treatment he chooses to apply, he may be able to increase the information by isolating this source of variation using the randomized complete block design, a direct extension of the matched-pairs design. Prior to randomly assigning or identifying the subjects to be assigned to the treatments, the overall set of subjects are first divided into groups (blocks) such that subjects within a block are more or less homogeneous and subjects between blocks are heterogeneous. Subjects within each block are then randomly assigned to the treatment conditions. It is worthy to remember that the number of subjects in a block is equal to the number of treatments (complete block). The blocking variable is one which could possibly affect the measurements on the response variable but of less importance than the treatment (factor).

Example In the previous experiment, suppose there is reason to believe that male and female students vary in their attention span. The experimental design has to be revised to take into consideration this “prior” variation. To do this, we can follow either one of the approaches below:

a. Select 6 students- 3 males and 3 females. One of the 3 males is randomly assigned to each of the meal plans: no breakfast, light breakfast, and full breakfast. Repeat for the group of 3 females. (RCBD without subsamples)

b. Select 30 students- 15 males and 15 females. Randomly assign 5 males to each of the meal plans. Repeat for the group of 15 females. (RCBD with subsamples)

For each design, attention spans (in minutes) of the subjects were recorded during a morning reading period. Analysis of Variance

It is a statistical method of dividing the total variation observed from experimental data into different components, each component assignable to a known source, cause, or factor.

For a CRD experiment, the total variance observed in the response measurements are divided into variation between treatments and the experimental error-the cumulative effect of all other factors not controlled in the experiment.

While for an experiment in RCBD, the total variance in the response variable is partitioned into three components: between treatments, between blocks, and experimental error.

The outline of the ANOVA table for CRD and RCBD experiments, respectively, are shown below:

Table 1. ANOVA for CRD experiments

Source of Variation df SS MS F

Between Treatments t – 1 SST MST MST/MSE

Experimental Error n – t SSE MSE

Total n – 1 Total SS

Page 4: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Table 2. ANOVA for RCBD experiments

Source of Variation df SS MS F

Between Treatments t– 1 SST MST MST/MSE

Between Blocks b – 1 SSB MSB MSB/MSE

Experimental Error (b –1)(t – 1) SSE MSE

Total n – 1 = bt – 1 Total SS

Assumptions for ANOVA 1. Normality 2. Homogeneity (equality) of variances 3. Independence 4. Linearity

Example In an experiment to determine the effect of nutrition on the attention spans of elementary students, a group of 15 students were randomly assigned to each of three meal plans: no breakfast, light breakfast, and full breakfast. Their attention spans (in minutes) were recorded during a morning reading period and are shown in the table below. Do the data provide sufficient evidence to indicate a difference in the average attention spans depending on the type of breakfast eaten by the student?

No Breakfast

Light breakfast

Full Breakfast

8 10 14

7 12 16

9 16 12

13 15 17

10 12 11

Analysis using STATA

1. Enter the data into Stata Data Editor or type the data in MS Excel and import into Stata.

Page 5: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

2. Test for homogeneity of variances.

Command: robvar attnspan, by(breakfast)

Output:

Interpretation: The variances of the attention span of students are homogeneous (not significantly

different) across treatments (W0=0.1198, p-value=0.8881).

3. Test for normality.

Command: by breakfast, sort: swilk attnspan

Output:

W10 = 0.11985019 df(2, 12) Pr > F = 0.88810181

W50 = 0.08571429 df(2, 12) Pr > F = 0.91841326

W0 = 0.11985019 df(2, 12) Pr > F = 0.88810181

Total 12.133333 3.04412 15

None 9.4 2.3021729 5

Light 13 2.4494897 5

Full 14 2.5495098 5

Breakfast Mean Std. Dev. Freq.

Summary of Attnspan

. robvar attnspan, by(breakfast)

attnspan 5 0.94273 0.676 -0.483 0.68530

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> breakfast = None

attnspan 5 0.94531 0.646 -0.535 0.70368

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> breakfast = Light

attnspan 5 0.94365 0.665 -0.501 0.69186

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> breakfast = Full

. by breakfast, sort : swilk attnspan

Page 6: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Interpretation: The attention span of students in the three treatments are normally distributed (all

three p-values>0.05).

4. Run one-way ANOVA with post hoc

Command: oneway attnspan breakfast, scheffe

Output:

Interpretation: Analysis of variance indicates significant differences in the attention span of

students (F=4.93, p-value=0.0273). We would then say that nutrition has an effect

on the attention spans of elementary school students.

In addition, the Scheffe test shows that the mean attention span of students with

Light and Full breakfast are not significantly different (p-value=0.813) and the

mean attention span of students with No breakfast is not significantly different from

the mean attention span of students with Light breakfast (p-value=0.105).

However, the mean attention span of students with No breakfast is statistically

different from the mean attention span of students with Full breakfast (p-

value=0.036).

Example: An accounting firm, prior to introducing in the firm widespread training in statistical sampling for

auditing, tested three training methods: 1. study at home with programmed training materials 2. training sessions at local offices conducted by local staff 3. training sessions in Manila conducted by national staff

Thirty auditors were grouped into 10 groups (blocks) of 3, according to time elapsed since college

graduation, and the auditors in each block were randomly assigned to the 3 training methods. Block 1 consists of auditors graduated most recently, …, block 10 consists of those graduated most distantly. At the end of the training, each auditor was asked to analyze a complex case involving statistical application;

0.036 0.105

None -4.6 -3.6

0.813

Light -1

Col Mean Full Light

Row Mean-

(Scheffe)

Comparison of Attnspan by Breakfast

Bartlett's test for equal variances: chi2(2) = 0.0378 Prob>chi2 = 0.981

Total 129.733333 14 9.26666667

Within groups 71.2 12 5.93333333

Between groups 58.5333333 2 29.2666667 4.93 0.0273

Source SS df MS F Prob > F

Analysis of Variance

. oneway attnspan breakfast, scheffe

Page 7: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

a proficiency measure based on this analysis was obtained for each auditor. The results are given in the table below.

Block

Training Method

1 2 3

1 73 81 92

2 76 79 89

3 75 76 87

4 74 77 90

5 76 71 88

6 73 75 85

7 68 72 88

8 64 71 82

9 65 73 81

10 62 69 78

Analysis using Stata

1. Enter the data into Stata Data Editor or type the data in MS Excel and import into Stata.

Page 8: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

2. Test for normality

Command: by trngmethod, sort : swilk profeciency

Output:

Interpretation: Proficiency scores are normally distributed (all three p-values>0.05).

3. Test for homogeneity of variances

Command: robvar profeciency, by(trngmethod)

Output:

Interpretation: The variation in proficiency scores is uniform across training methods (W0=1.3669,

p-value=0.2720).

profeciency 10 0.94871 0.790 -0.394 0.65325

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> trngmethod = 3

profeciency 10 0.96836 0.488 -1.151 0.87524

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> trngmethod = 2

profeciency 10 0.90546 1.457 0.671 0.25126

Variable Obs W V z Prob>z

Shapiro-Wilk W test for normal data

-> trngmethod = 1

. by trngmethod, sort : swilk profeciency

W10 = 0.81669956 df(2, 27) Pr > F = 0.45250832

W50 = 0.29752066 df(2, 27) Pr > F = 0.7450607

W0 = 1.36691542 df(2, 27) Pr > F = 0.27197195

Total 77 7.9956885 30

3 86 4.4221664 10

2 74.4 3.8643671 10

1 70.6 5.3374984 10

Trngmethod Mean Std. Dev. Freq.

Summary of Profeciency

. robvar profeciency, by(trngmethod)

Page 9: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

4. Two-way ANOVA (without replication)

Menu:

1. Click on Statistics>>Linear models and related>>ANOVA/MANOVA>>Analysis of

variance and covariance

2. On the dialog box, select proficiency in the Dependent variable drop-down menu and

trngmethod and block in the Model drop-down menu. See the screenshot below.

3. Click OK.

Command window: anova proficiency trngmethod block

Output:

Interpretation: Proficiency scores differ significantly among training methods (F=114.17, p-

value=0.000).

NOTE: Having observed that proficiency scores vary depending on the training method. It is often

interesting to see which particular methods differ. We accomplish this by doing pairwise

comparison (post hoc) tests.

Total 1854 29 63.931034

Residual 101.46667 18 5.637037

block 465.33333 9 51.703704 9.17 0.0000

trngmethod 1287.2 2 643.6 114.17 0.0000

Model 1752.5333 11 159.32121 28.26 0.0000

Source Partial SS df MS F Prob>F

Root MSE = 2.37424 Adj R-squared = 0.9118

Number of obs = 30 R-squared = 0.9453

. anova proficiency trngmethod block

Page 10: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Menu:

1. Click on Statistics>>Postestimation

2. In the dialog box, double click on Test, contrasts, and comparisons of parameter

estimates.

3. Double click on Pairwise comparisons and in the Main tab of the dialog box which pops

up on screen, select trngmethod in the pull down menu for Factor terms to compute

pairwise comparisons for. In the Multiple comaprsions pull-down menu select Scheffe’s

method.

Page 11: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

4. In the same dialog box, click on the Reporting tab and check the boxes opposite Specify

additional tables (default….), Show effect table with p-values, and Show table of

margins and group codes.

5. Click OK.

Output:

Interpretation: Proficiency scores differ significantly between Training methods 1 and 2 (p-

value=0.008); between Training methods 3 and 1 (p=0.000), and between

methods 3 and 2 (p=0.000).

3 vs 2 11.6 1.061794 10.92 0.000

3 vs 1 15.4 1.061794 14.50 0.000

2 vs 1 3.8 1.061794 3.58 0.008

trngmethod

Contrast Std. Err. t P>|t|

Scheffe

Page 12: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Factorial Experiments

Experiments where the effects of more than one factor are considered together are called factorial experiments and may sometimes be analyzed with the use of factorial ANOVA. For instance, the academic achievement of a student depends on study habits of the student as well as home environment. We may have two simple experiments, one to study the effect of study habits and another for home environment.

But these experiments will not give us any information about the dependence or independence of the two factors, namely study habit and home environment. In such cases, we resort to Factorial ANOV A which not only helps us to study the effect of two or more factors but also gives information about their dependence or independence in the same experiment. There are many types of factorial designs like 22, 23, 32, etc. The simplest of them all is the 22 or 2 x 2 experiment. Factors and Levels - An Example

Let us suppose that the Human Resources Department of a company desires to know if occupational stress varies according to age and gender. The dependent variable of interest is therefore occupational stress as measured by a scale.

There are two factors being studied - age and gender. Further suppose that the employees have been classified into three groups or levels:

age less than 40,

40 to 55

above 55 In addition employees have been labeled into gender classification (levels):

male female

In this design, factor age has three levels and gender two. In all, there are 3 x 2 = 6 groups or

cells. With this layout, we obtain scores on occupational stress from employee(s) belonging to the six cells.

Main Effects and Interaction

A main effect is an outcome that can show consistent difference between levels of a factor. In our example, there are two main effects ­ effect of age and effect of gender.

Factorial ANOVA also enables us to examine the interaction effect between the factors. An interaction effect is said to exist when differences on one factor depend on the level of the other factor. However, it is important to remember that interaction is between factors and not levels. We know that there is no interaction between the factors when we can talk about the effect of one factor without mentioning the other factor. Hypothesis Testing

In the above example, there are three hypotheses to be tested. These are:

H01: Interaction effect of age and gender is not present. (Effect of age is the same regardless of gender. Or, gender differences in occupational stress is the same across age groups.)

H02: Main effect of age is not significant. (There is no significant difference in mean occupational

stress scores among the three age groups.) H03: Main effect of gender is not significant. (There is no significant difference in mean

occupational stress scores between male and females employees.)

Page 13: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Analysis of Variance for Two-factor Factorial Experiments Table 3. ANOVA for two-factor factorial experiments

Source of Variation df SS MS F

Factor A a – 1 SSA MST MSA/MSE

Factor B b – 1 SSB MSB MSB/MSE

Interaction of A and B (a – 1) (b – 1) SSAB MSAB MSAB/MSE

Experimental Error ab(r – 1) SSE MSE

Total n – 1 = abr – 1 Total SS

Example:

Let us suppose that the Human Resources Department of a company desires to know if occupational stress varies according to age and gender. Thirty rank-and-file employees were selected and five were randomly assigned to each combination of gender and age group. Occupational stress measured on a scale of 1 (lowest) to 20 (highest) and are presented in the table below.

Gender

Age group

Below 40 40 to 55 Above 55

Male

15 15 18

13 14 16

14 10 10

11 9 12

9 8 16

Female

10 13 11

7 14 10

9 16 13

7 17 12

8 14 11

Analysis using STATA Here we assume all the assumptions of the ANOVA are met and proceed to the analysis of variance.

1. Data Entry

Page 14: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

Menu:

1. Click on Statistics>>Linear models and related>>ANOVA/MANOVA>>Analysis of

variance and covariance

2. On the dialog box, select stress in the Dependent variable drop-down menu and gender

and agegrp in the Model drop-down menu. See the screenshot below.

3. Click OK. Command window: anova stress gender agegrp gender#agegrp Output:

Interpretation: The interaction effect of gender and age on occupational stress is significant.

Gender difference in occupational stress is not the same across age groups. NOTE: If interaction effect is significant, do not test significance of main effects. Instead, compare

the mean response for different levels of one factor for each (fixed) level of the other factor. For example, we compare mean stress level of males and females for each age group.

Total 273.86667 29 9.4436782

Residual 128 24 5.3333333

gender#agegrp 88.2 2 44.1 8.27 0.0019

agegrp 46.866667 2 23.433333 4.39 0.0237

gender 10.8 1 10.8 2.03 0.1676

Model 145.86667 5 29.173333 5.47 0.0017

Source Partial SS df MS F Prob>F

Root MSE = 2.3094 Adj R-squared = 0.4352

Number of obs = 30 R-squared = 0.5326

. anova stress gender agegrp gender#agegrp

Page 15: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

In order to understand better the interaction effect of age and gender on occupational

stress, we type the following command tabulate gender agegrp, summarize(stress) nostandard nofreq

The above command will generate the following table:

If we type this command

margins gender#agegrp followed by

marginsplot, xdimension(gender) plotdimension(agegrp) the following plot is generated:

This plot further confirms the presence of interaction effect of age and gender on job stress. The

lines intersect each other in contrast to parallel lines when there is no interaction effect.

Suppose the researcher is interested to know if there is difference in the level of job stress between male and female employees for each age group. We do this using a T-test (male vs female) for each age group. But there is some minor recalculation of the t test statistic. The results of the analysis using Stata are shown below.

Total 10.3 13 12.9 12.066667

2 8.2 14.8 11.4 11.466667

1 12.4 11.2 14.4 12.666667

Gender 1 2 3 Total

Agegrp

Means of stress

. tabulate gender agegrp, summarize(stress) nostandard nofreq5

10

15

20

Lin

ear

Pre

dic

tio

n

Male FemaleGender

Below 40 40 to 55

More than 55

Adjusted Predictions of gender#agegrp with 95% CIs

Page 16: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

882

5

1

5

1335

24.

.

.t

; p-value=0.0205

472

5

1

5

1335

63.

.

.t

; p-value= 0.0387

Pr(T < t) = 0.9955 Pr(|T| > |t|) = 0.0090 Pr(T > t) = 0.0045

Ha: diff < 0 Ha: diff != 0 Ha: diff > 0

Ho: diff = 0 degrees of freedom = 8

diff = mean(1) - mean(2) t = 3.4293

diff 4.2 1.224745 1.375733 7.024267

combined 10 10.3 .9073772 2.869379 8.24737 12.35263

2 5 8.2 .5830952 1.30384 6.581068 9.818932

1 5 12.4 1.077033 2.408319 9.409677 15.39032

Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]

Two-sample t test with equal variances

-> Agegrp = 1

Pr(T < t) = 0.0258 Pr(|T| > |t|) = 0.0516 Pr(T > t) = 0.9742

Ha: diff < 0 Ha: diff != 0 Ha: diff > 0

Ho: diff = 0 degrees of freedom = 8

diff = mean(1) - mean(2) t = -2.2860

diff -3.6 1.574802 -7.231499 .0314989

combined 10 13 .9545214 3.018462 10.84072 15.15928

2 5 14.8 .7348469 1.643168 12.75974 16.84026

1 5 11.2 1.392839 3.114482 7.332859 15.06714

Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]

Two-sample t test with equal variances

-> Agegrp = 2

Page 17: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

052

5

1

5

1335

03.

.

.t

; p-value= 0.0745

Interpretation: There is a significant difference in the level of job stress between male and female

employees who are below 40 years old (p-value=0.0205) and among employees 40 to 50 years old (p-value=0.0387). But there is no significant difference in the level of job stress between male and female employees who are more than 50 years old (p-value=0.0745).

On the other hand, suppose the researcher is interested to know if there is difference in the level

of job stress in the three age groups for each gender. We do this using One-way ANOVA (Below 40 vs 40 to 50 vs Above 50) for each gender. But there is some minor recalculation of the F test statistic. The results of the analysis using Stata are shown below.

F=13.07/5.33=2.45; p-value= 0.1282

Pr(T < t) = 0.9550 Pr(|T| > |t|) = 0.0899 Pr(T > t) = 0.0450

Ha: diff < 0 Ha: diff != 0 Ha: diff > 0

Ho: diff = 0 degrees of freedom = 8

diff = mean(1) - mean(2) t = 1.9285

diff 3 1.555635 -.5873006 6.587301

combined 10 12.9 .8875685 2.806738 10.89218 14.90782

2 5 11.4 .509902 1.140175 9.984285 12.81571

1 5 14.4 1.469694 3.286335 10.31948 18.48052

Group Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]

Two-sample t test with equal variances

-> Agegrp = 3

Bartlett's test for equal variances: chi2(2) = 0.3720 Prob>chi2 = 0.830

Total 131.333333 14 9.38095238

Within groups 105.2 12 8.76666667

Between groups 26.1333333 2 13.0666667 1.49 0.2641

Source SS df MS F Prob > F

Analysis of Variance

-> Gender = 1

Page 18: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

F=54.47/5.33=10.22; p-value= 0.0026

Interpretation: There is no significant difference in the level of job stress of male employees across

different age groups (p-value=0.1282). However, level of job stress of female employees varies across different age groups (p-value=0.0026).

Example: Two-factor factorial experiment with no significant interaction

A food technologist conducted a taste test to answer the age old question: chocolate or vanilla? You suspect that the answer, however, may depend on what type of dessert it is. So you recruit 20 people and assign them to one of four conditions: chocolate cake, chocolate ice cream, vanilla cake, or vanilla ice cream. After each person tastes their assigned dessert they rate how much they like it on a scale of 1-9. The output of the analysis is given below. Determine if there is a main effect of flavor or dessert type and/or an interaction between the two.

Interpretations: 1. Flavor and type of dessert do not have significant interaction effect on taste scores (p-

value=0.6739). That is, the difference in the taste scores between chocolate and vanilla is consistent for each type of dessert. In fact,

Bartlett's test for equal variances: chi2(2) = 0.5015 Prob>chi2 = 0.778

Total 131.733333 14 9.40952381

Within groups 22.8 12 1.9

Between groups 108.933333 2 54.4666667 28.67 0.0000

Source SS df MS F Prob > F

Analysis of Variance

-> Gender = 2

Total 56.55 19 2.9763158

Residual 39.2 16 2.45

flavor#dessert .45 1 .45 0.18 0.6739

dessert 2.45 1 2.45 1.00 0.3322

flavor 14.45 1 14.45 5.90 0.0273

Model 17.35 3 5.7833333 2.36 0.1099

Source Partial SS df MS F Prob>F

Root MSE = 1.56525 Adj R-squared = 0.1768

Number of obs = 20 R-squared = 0.3068

. anova liking flavor dessert flavor# dessert

Page 19: DESIGN OF EXPERIMENTS AND ANALYSIS OF …bertmilla.weebly.com/uploads/6/4/6/4/64640057/design_of...DESIGN OF EXPERIMENTS AND ANALYSIS OF VARIANCE Unlike a descriptive study, an experiment

2. There is no significant difference in the taste scores between ice cream and cake (p-value=0.3322).

3. There is a significant difference in the taste scores between vanilla and chocolate (p-value=0.0273). Chocolate tastes better than vanilla be it ice cream or cake.

Total 4.8 5.5 5.15

2 5.8 6.2 6

1 3.8 4.8 4.3

Flavor 1 2 Total

Dessert

24

68

Lin

ear

Pre

dic

tio

n

1 2Dessert

Flavor=1 Flavor=2

Adjusted Predictions of Dessert#Flavor with 95% CIs

Dessert=1 (Ice cream) Dessert=2 (Cake) Flavor=1 (Vanilla) Flavor=2 (Chocolate)