1 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely

1

Chapter 1: Introduction to Design of Experiments

1.1 Review of Basic Statistical Concepts (Optional)

1.2 Introduction to Experimental Design

1.3 Completely Randomized Design

2


1.1 Review of Basic Statistical Concepts (Optional)1.1 Review of Basic Statistical Concepts (Optional)



Objectives Understand basic statistical concepts such as samples

and populations, statistics and parameters, normal distribution, central limit theorem, and hypothesis testing.

Recognize a completely randomized design.

3

Populations and SamplesA population is defined by the researcher and is usually the set of all possible outcomes of an experiment or process.

A sample is a subset of a population. The researcher would like the sample to be representative of the population. Normally, you as researchers deal with samples rather

than populations. With designed experiments, you will actually create

two or more populations. Each test, or run, in the experiment is a sample.

4

Statistics and ParametersStatistics are measurements or attributes about a sample. For example, you might say that the variance of the sample values is 25.

Parameters are measurements or attributes about a population. Parameters are usually not known and are estimated by sample statistics.

With a designed experiment, you will determine if changing a variable manifests itself in the parameters you estimate.

5

Normal Distribution

6

Normal Distribution and Central Limit Theorem The normal distribution is without question the most

important of all of the probability distributions. It is the basis for the majority of statistical inferences you make as a researcher.

The central limit theorem (CLT) states that the sum of independent and identically distributed random variables is approximately normally distributed. Thus, means calculated from samples of size n from a population can be modeled with a normal distribution and the approximation will improve as the sample size, n, increases.

7

Central Limit Theorem

8

Central Limit Theorem

This demonstration illustrates the concepts discussed previously.

9

10

Hypothesis or Significance Testing A hypothesis test is used to assess the evidence provided by the data in favor of some claim about the population.

There are four steps to conduct a hypothesis test.

1. State the null (H0) and alternative (Ha) hypotheses.

2. State alpha (significance level).

3. Collect data, compute sample statistics, and compute the p-value under H0.

4. Make decision: If p-value<α, there is sufficient evidence to reject H0. If p-value≥α, there is not sufficient evidence to reject H0.

You can never prove the null hypothesis. You can only state whether or not you have evidence to reject that hypothesis.

11

12

1.01 Multiple Choice PollIf a p-value is less than your stated alpha, then which of the following is true?

a. There is sufficient evidence to reject the null hypothesis.

b. There is not sufficient evidence to reject the null hypothesis.

13

1.01 Multiple Choice Poll – Correct AnswerIf a p-value is less than your stated alpha, then which of the following in true?

a. There is sufficient evidence to reject the null hypothesis.

b. There is not sufficient evidence to reject the null hypothesis.

14

Types of Errors and PowerYou perform a hypothesis test and make a decision, but was the decision correct?

The probability of Type I error is denoted by .

15

Types of Errors and PowerYou perform a hypothesis test and make a decision, but was the decision correct?

The probability of Type II error is denoted by .

The power of the statistical test is 1 - .

16

How Many Observations?

17

Level of Significance

Level of Significance

Effect SizeEffect Size

PowerPower

VariabilityVariabilityRequired Sample Size

Required Sample Size

18

1.02 QuizGiven that you hold all other terms constant, how does the required sample size change?

Match the number with the appropriate letter:

1. The effect size (the difference that is practically important to you) is increased.

2. The significance level (alpha) is decreased.

3. The desired power is increased.

4. The variability in the process decreases.

A: a higher sample size is necessary

B: a lower sample size is sufficient

19

1.02 Quiz – Correct AnswerGiven that you hold all other terms constant, how does the required sample size change?

Match the number with the appropriate letter:

1. The effect size (the difference that is practically important to you) is increased.

2. The significance level (alpha) is decreased.

3. The desired power is increased.

4. The variability in the process decreases.

A: a higher sample size is necessary

B: a lower sample size is sufficient

1-B, 2-A, 3-A, 4-B

20

Hardness Measurement Procedure

21

Data Model and Assumptions

i iY

22

This model represents what you think is true about your population. Specifically, you assume that there is a single population with values that are normally distributed. This normal distribution has a mean µ and standard deviation σ that affects the observed errors, εi.

Demonstration Information Preliminary sample information indicates that the

hardness difference you need to detect is about 0.45 and the standard deviation is about 0.5.

The company would like the power of the test to be at least 0.80 and to test at a level of significance of 0.05.

This information will be used to determine the required sample size.

23

Determining Power and Sample Size


24

Completely Randomized Design Twelve sheets are randomly selected from a randomly

selected lot, and measurements of hardness are recorded and stored in Hardness.jmp.

Conduct a significance test to determine if the average Hardness is not equal to 9.5.

25

Analyzing Data from a Completely Randomized Design


26

Conclusions Based on the analysis performed and additional

information about the measurement process, a decision was made to perform additional experiments.

The manufacturer would like to know if different drill tips affect the measurement of Hardness.

27

28

29



1.2 Introduction to Experimental Design1.2 Introduction to Experimental Design


Objectives Understand experimental design and its importance. Outline the experimental process.

30

Why Do You Experiment? To discover the sources of variation in a measured

response To collect evidence to support or rebut a theory To determine a consistent result of a system, product

design, or process To find conditions that yield a maximum or minimum

response in a specified range To compare values of the response at different

settings of the controllable variables To build a predictive model

31

What Is Design?Experimental design is the planning phase of data collection. It defines the structure of the experiment to ensure the efficient use of collected data.

32

Ad-Hoc Analysis versus Experimental Design

Ad-Hoc Regression Experimental Design

Process outcomes are measured

Process outcomes are measured

Objective of the process is to make all units identical

Objective of the experimental design is to discern which input variables influence a response

The ranges of input values are limited

The ranges of the input variables are purposefully manipulated

The values of the input variables might be correlated

There is zero, or near zero, correlation between input variables

33

Experiment: Design and AnalysisDesign and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple.

34 ...

Experiment: Design and AnalysisDesign and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple.

35

You begin with design!You begin with design!

The Process of Experimenting1. Define the purpose of the experiment.

2. Document the specific questions to be answered.

3. Define the population of interest.

4. Determine the need for sampling.

5. Define the data collection protocol.

6. Collect the data.

7. Analyze the data.

36

Define the Data Collection Protocol1. Describe the process.

2. Identify sources of variability in the process.

3. Determine the “best” design for the experiment.

4. Delineate the experimental procedure.

37

Identifying Sources of Variability

38

People

Training Shift

Hardness

Drill Tips

Within Lot

Between Lot

Materials

Drill Press

Equipment

Basic TermsSome basic terms used in experimental design are factor factor level treatment or design point response effect experimental unit run replication.

39

40

1.03 QuizSuppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg.

Match the term on the left with the appropriate experimental component on the right.

1. Factor

2. Factor Levels

3. Response

4. Run

5. Experimental Unit

41

A. 3 mg, 5 mg, and 7 mgB. Blood pressure readingC. DrugD. Dosage and corresponding blood pressure readingE. A woman

1.03 Quiz – Correct AnswerSuppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg.

Match the term on the left with the appropriate experimental component on the right.

1. Factor

2. Factor Levels

3. Response

4. Run

5. Experimental Unit

1-C, 2-A, 3-B, 4-D, 5-E

42

A. 3 mg, 5 mg, and 7 mgB. Blood pressure readingC. DrugD. Dosage and corresponding blood pressure readingE. A woman

Three Basic Principles of Designs Randomization of runs prevents systematic biases

from being introduced into the experiment. Randomization refers not only to performing the runs in a random order, but also to resetting the conditions after each run.

Blocking is a design technique used to reduce or control variability from nuisance factors.

Replication enables the experimenter to obtain an estimate of experimental error.

43

44

Exercise

This exercise reinforces the concepts discussed previously.

45

46

1.04 QuizThe output from the exercise is given below.Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days?

47

1.04 Quiz – Correct AnswerThe output from the exercise is given below.Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days?

No. The p-value is .0544, which is greater than .05. Therefore, there is not sufficient evidence to reject Ho (μ=120) in favor of Ha (μ>120).

48

49




1.3 Completely Randomized Design1.3 Completely Randomized Design

Objectives Understand basic terminology of experimental design. Follow the experimental design process to set up an

experiment and determine the appropriate design. Generate and analyze a completely randomized

design.

50

Drill Tip Example

51

Define the Purpose of the ExperimentThe company wants to determine whether Hardness readings from four types of drill tips are different.

52

Document the Specific QuestionsAre the average Hardness readings from the four types of drill tips significantly different from each other?

53

Define the Populations of InterestThe company is interested in all drill tips of these four types produced by its supplier, the XYZ Corporation.

54

...

...

...

...

Determine the Need for SamplingIt is physically impossible to collect information on all of the drill tips made by the XYZ Corporation; therefore, you need to use a sample of each of the populations of drill tips.

A sample of the Purple tips is shown below. Each of the other three populations would be sampled the same way.

55

...

Define the Data Collection Protocol – Describe the Process

56 ...


57 ...


58 ...


59

Define the Data Collection Protocol – Identify the Sources of Variability in the Process

Identify variability caused by the factor(s) of interest other factors, known as nuisance factors.

60

Define the Data Collection Protocol – Determine the “Best” Design for the Experiment The experiment has one factor, Tip Type, with four

levels, Purple, Green, Orange, and Blue. These factor levels are easy to change from run to run.

The experimental unit is a quadrant of a metal sheet. The experimental units are believed to be

homogeneous. The completely randomized design is appropriate.

61

Define the Data Collection Protocol – Delineate the Experimental Procedure

62

Experimental Units and Replication

63

VERSUSVERSUS

Determining Power and Sample SizeIn preparation for this experiment, you have consulted with industry experts and reviewed previous experiments on drill tips. The following information is determined: The expected standard deviation for each treatment

group is approximately 0.2. Alpha=0.05. Power needs to be at least 85%. The estimates for the means for each Tip Type are

given by 9.0, 9.1, 9.4, and 9.6.

64

Determining Power and Sample Size


65

66


Creating a Cause-and-Effect Diagram (Optional)

Generating a Completely Randomized Design


67

The Design

68

P G P G

P OP OG BG B

G PG P

B PB P

B BB BO OO O

G OG O

The ANOVA Model

69

ijiijY

Now there is a belief that each population (each value of Tip Type, i) has a unique mean, given by µ + αi. Each population is normally distributed, but there is a shift in the mean for each population.

The ANOVA Hypothesis

70

Ho: All means equal

20

15

10

5

0

H1: Not all means are equal

20

15

10

5

0

H0: µPurple = µGreen = µOrange = µBlue

Partitioning Variability in ANOVAIf the between-group variability is larger than the within-group variability, reject H0.

71

Between

WithinTotal

The variability estimated under H0.

The variability estimated under Ha.

Assumptions of ANOVA

72

independent observations

normally distributed residuals

equal variances for each group

Comparing Populations

A B C

2

1

3

D

4

73

1.05 Multiple Choice PollWhat is the test statistic for an ANOVA?

a. t

b. z

c. F

d. W

74

1.05 Multiple Choice Poll – Correct AnswerWhat is the test statistic for an ANOVA?

a. t

b. z

c. F

d. W

75

Analyzing Data from a Completely Randomized Design


76

Prospective versus Retrospective Power

77

Prospective Retrospective

Alpha (α) .05 .05

Error Standard Deviation .2 .274

Mean for Orange 9.0 9.450

Mean for Purple 9.1 9.575

Mean for Green 9.4 9.600

Mean for Blue 9.6 9.875

Sample Size (n) 16 16

Power (1-β) .9374 .3357

78

Exercise

This exercise reinforces the concepts discussed previously.

79

80

1.06 PollIs it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?

Yes

No

81

1.06 Poll – Correct AnswerIs it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?

Yes

No

82

Documents

1 Chapter 1: Introduction to Design of Experiments 1.1 Review of Basic Statistical Concepts (Optional) 1.2 Introduction to Experimental Design 1.3 Completely