View
247
Download
3
Category
Tags:
Preview:
Citation preview
1
Chapter 1: Introduction to Design of Experiments
1.1 Review of Basic Statistical Concepts (Optional)
1.2 Introduction to Experimental Design
1.3 Completely Randomized Design
2
Chapter 1: Introduction to Design of Experiments
1.1 Review of Basic Statistical Concepts (Optional)1.1 Review of Basic Statistical Concepts (Optional)
1.2 Introduction to Experimental Design
1.3 Completely Randomized Design
Objectives Understand basic statistical concepts such as samples
and populations, statistics and parameters, normal distribution, central limit theorem, and hypothesis testing.
Recognize a completely randomized design.
3
Populations and SamplesA population is defined by the researcher and is usually the set of all possible outcomes of an experiment or process.
A sample is a subset of a population. The researcher would like the sample to be representative of the population. Normally, you as researchers deal with samples rather
than populations. With designed experiments, you will actually create
two or more populations. Each test, or run, in the experiment is a sample.
4
Statistics and ParametersStatistics are measurements or attributes about a sample. For example, you might say that the variance of the sample values is 25.
Parameters are measurements or attributes about a population. Parameters are usually not known and are estimated by sample statistics.
With a designed experiment, you will determine if changing a variable manifests itself in the parameters you estimate.
5
Normal Distribution
6
Normal Distribution and Central Limit Theorem The normal distribution is without question the most
important of all of the probability distributions. It is the basis for the majority of statistical inferences you make as a researcher.
The central limit theorem (CLT) states that the sum of independent and identically distributed random variables is approximately normally distributed. Thus, means calculated from samples of size n from a population can be modeled with a normal distribution and the approximation will improve as the sample size, n, increases.
7
Central Limit Theorem
8
Central Limit Theorem
This demonstration illustrates the concepts discussed previously.
9
10
Hypothesis or Significance Testing A hypothesis test is used to assess the evidence provided by the data in favor of some claim about the population.
There are four steps to conduct a hypothesis test.
1. State the null (H0) and alternative (Ha) hypotheses.
2. State alpha (significance level).
3. Collect data, compute sample statistics, and compute the p-value under H0.
4. Make decision: If p-value<α, there is sufficient evidence to reject H0. If p-value≥α, there is not sufficient evidence to reject H0.
You can never prove the null hypothesis. You can only state whether or not you have evidence to reject that hypothesis.
11
12
1.01 Multiple Choice PollIf a p-value is less than your stated alpha, then which of the following is true?
a. There is sufficient evidence to reject the null hypothesis.
b. There is not sufficient evidence to reject the null hypothesis.
13
1.01 Multiple Choice Poll – Correct AnswerIf a p-value is less than your stated alpha, then which of the following in true?
a. There is sufficient evidence to reject the null hypothesis.
b. There is not sufficient evidence to reject the null hypothesis.
14
Types of Errors and PowerYou perform a hypothesis test and make a decision, but was the decision correct?
The probability of Type I error is denoted by .
15
Types of Errors and PowerYou perform a hypothesis test and make a decision, but was the decision correct?
The probability of Type II error is denoted by .
The power of the statistical test is 1 - .
16
How Many Observations?
17
Level of Significance
Level of Significance
Effect SizeEffect Size
PowerPower
VariabilityVariabilityRequired Sample Size
Required Sample Size
18
1.02 QuizGiven that you hold all other terms constant, how does the required sample size change?
Match the number with the appropriate letter:
1. The effect size (the difference that is practically important to you) is increased.
2. The significance level (alpha) is decreased.
3. The desired power is increased.
4. The variability in the process decreases.
A: a higher sample size is necessary
B: a lower sample size is sufficient
19
1.02 Quiz – Correct AnswerGiven that you hold all other terms constant, how does the required sample size change?
Match the number with the appropriate letter:
1. The effect size (the difference that is practically important to you) is increased.
2. The significance level (alpha) is decreased.
3. The desired power is increased.
4. The variability in the process decreases.
A: a higher sample size is necessary
B: a lower sample size is sufficient
1-B, 2-A, 3-A, 4-B
20
Hardness Measurement Procedure
21
Data Model and Assumptions
i iY
22
This model represents what you think is true about your population. Specifically, you assume that there is a single population with values that are normally distributed. This normal distribution has a mean µ and standard deviation σ that affects the observed errors, εi.
Demonstration Information Preliminary sample information indicates that the
hardness difference you need to detect is about 0.45 and the standard deviation is about 0.5.
The company would like the power of the test to be at least 0.80 and to test at a level of significance of 0.05.
This information will be used to determine the required sample size.
23
Determining Power and Sample Size
This demonstration illustrates the concepts discussed previously.
24
Completely Randomized Design Twelve sheets are randomly selected from a randomly
selected lot, and measurements of hardness are recorded and stored in Hardness.jmp.
Conduct a significance test to determine if the average Hardness is not equal to 9.5.
25
Analyzing Data from a Completely Randomized Design
This demonstration illustrates the concepts discussed previously.
26
Conclusions Based on the analysis performed and additional
information about the measurement process, a decision was made to perform additional experiments.
The manufacturer would like to know if different drill tips affect the measurement of Hardness.
27
28
29
Chapter 1: Introduction to Design of Experiments
1.1 Review of Basic Statistical Concepts (Optional)
1.2 Introduction to Experimental Design1.2 Introduction to Experimental Design
1.3 Completely Randomized Design
Objectives Understand experimental design and its importance. Outline the experimental process.
30
Why Do You Experiment? To discover the sources of variation in a measured
response To collect evidence to support or rebut a theory To determine a consistent result of a system, product
design, or process To find conditions that yield a maximum or minimum
response in a specified range To compare values of the response at different
settings of the controllable variables To build a predictive model
31
What Is Design?Experimental design is the planning phase of data collection. It defines the structure of the experiment to ensure the efficient use of collected data.
32
Ad-Hoc Analysis versus Experimental Design
Ad-Hoc Regression Experimental Design
Process outcomes are measured
Process outcomes are measured
Objective of the process is to make all units identical
Objective of the experimental design is to discern which input variables influence a response
The ranges of input values are limited
The ranges of the input variables are purposefully manipulated
The values of the input variables might be correlated
There is zero, or near zero, correlation between input variables
33
Experiment: Design and AnalysisDesign and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple.
34 ...
Experiment: Design and AnalysisDesign and analysis go hand-in-hand. You analyze data to answer your questions. You design an experiment so that the analysis is simple.
35
You begin with design!You begin with design!
The Process of Experimenting1. Define the purpose of the experiment.
2. Document the specific questions to be answered.
3. Define the population of interest.
4. Determine the need for sampling.
5. Define the data collection protocol.
6. Collect the data.
7. Analyze the data.
36
Define the Data Collection Protocol1. Describe the process.
2. Identify sources of variability in the process.
3. Determine the “best” design for the experiment.
4. Delineate the experimental procedure.
37
Identifying Sources of Variability
38
People
Training Shift
Hardness
Drill Tips
Within Lot
Between Lot
Materials
Drill Press
Equipment
Basic TermsSome basic terms used in experimental design are factor factor level treatment or design point response effect experimental unit run replication.
39
40
1.03 QuizSuppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg.
Match the term on the left with the appropriate experimental component on the right.
1. Factor
2. Factor Levels
3. Response
4. Run
5. Experimental Unit
41
A. 3 mg, 5 mg, and 7 mgB. Blood pressure readingC. DrugD. Dosage and corresponding blood pressure readingE. A woman
1.03 Quiz – Correct AnswerSuppose an experiment is to be conducted to test the effect of a particular drug on the blood pressure of women. The three dosages are 3, 5, and 7 mg.
Match the term on the left with the appropriate experimental component on the right.
1. Factor
2. Factor Levels
3. Response
4. Run
5. Experimental Unit
1-C, 2-A, 3-B, 4-D, 5-E
42
A. 3 mg, 5 mg, and 7 mgB. Blood pressure readingC. DrugD. Dosage and corresponding blood pressure readingE. A woman
Three Basic Principles of Designs Randomization of runs prevents systematic biases
from being introduced into the experiment. Randomization refers not only to performing the runs in a random order, but also to resetting the conditions after each run.
Blocking is a design technique used to reduce or control variability from nuisance factors.
Replication enables the experimenter to obtain an estimate of experimental error.
43
44
Exercise
This exercise reinforces the concepts discussed previously.
45
46
1.04 QuizThe output from the exercise is given below.Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days?
47
1.04 Quiz – Correct AnswerThe output from the exercise is given below.Given α=.05, is there sufficient evidence that the average shelf life is greater than 120 days?
No. The p-value is .0544, which is greater than .05. Therefore, there is not sufficient evidence to reject Ho (μ=120) in favor of Ha (μ>120).
48
49
Chapter 1: Introduction to Design of Experiments
1.1 Review of Basic Statistical Concepts (Optional)
1.2 Introduction to Experimental Design
1.3 Completely Randomized Design1.3 Completely Randomized Design
Objectives Understand basic terminology of experimental design. Follow the experimental design process to set up an
experiment and determine the appropriate design. Generate and analyze a completely randomized
design.
50
Drill Tip Example
51
Define the Purpose of the ExperimentThe company wants to determine whether Hardness readings from four types of drill tips are different.
52
Document the Specific QuestionsAre the average Hardness readings from the four types of drill tips significantly different from each other?
53
Define the Populations of InterestThe company is interested in all drill tips of these four types produced by its supplier, the XYZ Corporation.
54
...
...
...
...
Determine the Need for SamplingIt is physically impossible to collect information on all of the drill tips made by the XYZ Corporation; therefore, you need to use a sample of each of the populations of drill tips.
A sample of the Purple tips is shown below. Each of the other three populations would be sampled the same way.
55
...
Define the Data Collection Protocol – Describe the Process
56 ...
Define the Data Collection Protocol – Describe the Process
57 ...
Define the Data Collection Protocol – Describe the Process
58 ...
Define the Data Collection Protocol – Describe the Process
59
Define the Data Collection Protocol – Identify the Sources of Variability in the Process
Identify variability caused by the factor(s) of interest other factors, known as nuisance factors.
60
Define the Data Collection Protocol – Determine the “Best” Design for the Experiment The experiment has one factor, Tip Type, with four
levels, Purple, Green, Orange, and Blue. These factor levels are easy to change from run to run.
The experimental unit is a quadrant of a metal sheet. The experimental units are believed to be
homogeneous. The completely randomized design is appropriate.
61
Define the Data Collection Protocol – Delineate the Experimental Procedure
62
Experimental Units and Replication
63
VERSUSVERSUS
Determining Power and Sample SizeIn preparation for this experiment, you have consulted with industry experts and reviewed previous experiments on drill tips. The following information is determined: The expected standard deviation for each treatment
group is approximately 0.2. Alpha=0.05. Power needs to be at least 85%. The estimates for the means for each Tip Type are
given by 9.0, 9.1, 9.4, and 9.6.
64
Determining Power and Sample Size
This demonstration illustrates the concepts discussed previously.
65
66
This demonstration illustrates the concepts discussed previously.
Creating a Cause-and-Effect Diagram (Optional)
Generating a Completely Randomized Design
This demonstration illustrates the concepts discussed previously.
67
The Design
68
P G P G
P OP OG BG B
G PG P
B PB P
B BB BO OO O
G OG O
The ANOVA Model
69
ijiijY
Now there is a belief that each population (each value of Tip Type, i) has a unique mean, given by µ + αi. Each population is normally distributed, but there is a shift in the mean for each population.
The ANOVA Hypothesis
70
Ho: All means equal
20
15
10
5
0
H1: Not all means are equal
20
15
10
5
0
H0: µPurple = µGreen = µOrange = µBlue
Partitioning Variability in ANOVAIf the between-group variability is larger than the within-group variability, reject H0.
71
Between
WithinTotal
The variability estimated under H0.
The variability estimated under Ha.
Assumptions of ANOVA
72
independent observations
normally distributed residuals
equal variances for each group
Comparing Populations
A B C
2
1
3
D
4
73
1.05 Multiple Choice PollWhat is the test statistic for an ANOVA?
a. t
b. z
c. F
d. W
74
1.05 Multiple Choice Poll – Correct AnswerWhat is the test statistic for an ANOVA?
a. t
b. z
c. F
d. W
75
Analyzing Data from a Completely Randomized Design
This demonstration illustrates the concepts discussed previously.
76
Prospective versus Retrospective Power
77
Prospective Retrospective
Alpha (α) .05 .05
Error Standard Deviation .2 .274
Mean for Orange 9.0 9.450
Mean for Purple 9.1 9.575
Mean for Green 9.4 9.600
Mean for Blue 9.6 9.875
Sample Size (n) 16 16
Power (1-β) .9374 .3357
78
Exercise
This exercise reinforces the concepts discussed previously.
79
80
1.06 PollIs it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?
Yes
No
81
1.06 Poll – Correct AnswerIs it always necessary to follow an ANOVA with a multiple comparisons test, such as Tukey's HSD?
Yes
No
82
Recommended