58
Engineering Statistics Training Introduction to Design of Experiments Planning and Executing Dave Jackson, M.S. EW Engineering Statistics Introduction to Design of Experiments, Planning and Execution, July ‘05 1

Planning for DOE, Lesson 1

Embed Size (px)

DESCRIPTION

Design of Experiments

Citation preview

Page 1: Planning for DOE, Lesson 1

Engineering Statistics Training

Introduction to Design of Experiments

Planning and Executing

Dave Jackson, M.S. EW Engineering Statistics

Introduction to Design of Experiments, Planning and Execution, July ‘05 1

Page 2: Planning for DOE, Lesson 1

Contents Introduction to DOE -------------------------------------------------------------- 4

Prerequisites ------------------------------------------------------------------- 5 Training objective ------------------------------------------------------------- 6 What is DOE? ------------------------------------------------------------------ 7 Applications of DOE ---------------------------------------------------------- 8 DOE overview ----------------------------------------------------------------- 9 Why make an effort to learn DOE?---------------------------------------10 DOE vocabulary --------------------------------------------------------------13 2 and 3 level designs -------------------------------------------------------14 The empirical model ---------------------------------------------------------16 The steps for completing a DOE -----------------------------------------18 The sequential nature of DOE --------------------------------------------19

Planning for DOE ----------------------------------------------------------------20

• DOE planning prior to selecting a design --------------------------21 • Define the process you want to study -------------------------------22 • Define the experiment objectives ------------------------------------23 • Identify responses -------------------------------------------------------24 • The sample mean --------------------------------------------------------25 • The sample standard deviation ---------------------------------------36 • The %CV -------------------------------------------------------------------38 • Pass/fail responses ------------------------------------------------------39 • Sample size ---------------------------------------------------------------30 • Response calculation worksheet (example) -----------------------33 • Measuring the response ------------------------------------------------34

Introduction to Design of Experiments, Planning and Execution, July ‘05 2

• Response worksheet ----------------------------------------------------40

Page 3: Planning for DOE, Lesson 1

Contents Planning for DOE ------------------------------------------------------ continued

• Process variables worksheet (example) ----------------------------42 • Process variables (noise) ----------------------------------------------45 • The importance of wide factor level ranges -----------------------46 • Signal-to-noise review --------------------------------------------------48 • Factor that are a nuisance to vary -----------------------------------49 • Additional notes on factors --------------------------------------------50

Executing a DOE ----------------------------------------------------------------51

• Checklist for executing a DOE ----------------------------------------52

Final Thoughts --------------------------------------------------------------------53 Appendix A ------------------------------------------------------------------------54 References ------------------------------------------------------------------------55

Introduction to Design of Experiments, Planning and Execution, July ‘05 3

Page 4: Planning for DOE, Lesson 1

Introduction to DOE

Introduction to Design of Experiments, Planning and Execution, July ‘05 4

Page 5: Planning for DOE, Lesson 1

Prerequisites

• An interest in using Design of Experiments (DOE)

• An expectation for using DOE in the near future

• The ability to navigate in a Windows environment

• A working knowledge of basic statistical concepts. The training will reinforce and expand on these concepts using practical examples.

Introduction to Design of Experiments, Planning and Execution, July ‘05 5

Page 6: Planning for DOE, Lesson 1

Training Objective The objective of this course is to increase the likelihood of successful

experiment outcomes at Edwards.

The course will cover DOE planning, design selection, proper

execution, and analysis of DOE data using the Statgraphics software

package.

A successful experiment,

• Yields useful information that moves the project forward

• Does not have to be repeated

Introduction to Design of Experiments, Planning and Execution, July ‘05 6

Page 7: Planning for DOE, Lesson 1

What is DOE?

DOE is a procedure for learning how process variables acting

simultaneously affect, and interact to affect process performance.

Introduction to Design of Experiments, Planning and Execution, July ‘05 7

Estimated Response SurfaceMixer Speed=25.0

TemperatureTimeA

f Tem

pera

ture

556 558 560 562 564 240250260270280290300-11-10-9-8-7-6-5

Page 8: Planning for DOE, Lesson 1

Applications of DOE • Determine the process variable settings that simultaneously

optimize several performance characteristics • Screen out the important process variables from a large set of

candidate variables. • Reduce development time for new processes • Minimize cost by finding process variable settings that reduce the

cost of running a process • Reduce variation in a process

Introduction to Design of Experiments, Planning and Execution, July ‘05 8

Page 9: Planning for DOE, Lesson 1

Why Make the Effort to Learn DOE? DOE is a vast improvement over one-factor-at-a-time experimentation

and/or a series of trial-and-error efforts:

• More powerful – DOE gets the right answer more often

• More efficient – DOE gets the answer in fewer runs, with

fewer test articles and overall shortens

development time

• More actionable – DOE results are easier to interpret

Introduction to Design of Experiments, Planning and Execution, July ‘05 9

Page 10: Planning for DOE, Lesson 1

Why Make the Effort to Learn DOE? cont. Statistically Designed Experiments Can Detect and Describe

Interactions Between Input variables

Two variables interact if changing the level of one variable changes

the relationship between a second variable and the response. Time

and Temperature interact in the process of baking cookies.

Introduction to Design of Experiments, Planning and Execution, July ‘05 10

highlow

Flavor

Oven Temperature

Oven Time = short

Oven Time = long

Page 11: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 11

Why Make the Effort to Learn DOE? cont. Statistically Designed Experiments Can Deal With Curvature in the

Response.

Flavor

low

Amount of Salt in Food

medium high

Page 12: Planning for DOE, Lesson 1

ction to Design of Experiments, Planning and Execution, July ‘05 12

DOE Overview

A statistically designed experiment consist of a series of runs, or trials, in which (1) purposeful changes are made simultaneously to the input variables of a process, and (2) process performance is observed.

Introdu

Tissue Thickness

Flow Rate Power Pressure Duration

Ave. Lesion Width

SD Lesion Width

Ave. Lesion Depth

SD Lesion Depth

9 1 8 225 60 12 6 8 225 60

9 1 8 75 90 12 1 8 225 90 12 1 12 225 60

9 6 8 75 60 12 6 12 225 90

9 6 12 75 90 10.5 3.5 10 150 75 12 1 8 75 60 12 6 8 75 90 12 1 12 75 90

9 6 8 225 90

Page 13: Planning for DOE, Lesson 1

DOE Vocabulary

• There are typically many process variables that affect a response. Those process variables whose levels you change per your experiment design are called factors.

• Responses are the process performance characteristics that

you are interested in studying.

• An experiment design consists of the entire set of runs in the DOE.

• Runs are different combinations of factor levels. Identical back-

to-back runs require a complete reset of the process between runs.

• Center points are runs located in the center of the experiment

space.

• The experiment space is the multidimensional envelope

extending to the low and high levels of the factors.

Introduction to Design of Experiments, Planning and Execution, July ‘05 13

Factor B

Factor A

Factor C

Center points

Experiment space

Page 14: Planning for DOE, Lesson 1

2 and 3-level Designs

Two Level Designs:

• Factors are run at two levels (low and high) in the base design.

• Even when center points are added, the design is still

considered two-level.

• Used to model responses with, at most, a small amount of curvature.

• Screening experiments1

• Range finding

Three Level Designs:

• Factors are run at three or more levels in the base design • Center points are often an integral part of the base design

• Used to model significant curvature in the response

Introduction to Design of Experiments, Planning and Execution, July ‘05 14

1 A two-level experiment is called a “screening experiment” if its primary purpose is to screening-out the most important factors from a large group of candidate factors. In this scenario, we are interested in identifying significant main effects, rather than interaction effects or curvature effects; the latter assumed to be an order of magnitude less important.

Page 15: Planning for DOE, Lesson 1

Which Response Can You Model With a 2-Level Design?

Response Surface A

X

Y

Res

pons

e

00.20.40.60.81 0

80

Y

Response Surface B

XY

Res

pons

e

-4-2

-100

Response Surface C

XY

Res

pons

e

-2-30

-12

(X 1000)

Introduction to Design of Experiments, Planning and Execution, July ‘05 15

Answer: A only. B and C require a design with at least three levels.

Page 16: Planning for DOE, Lesson 1

The Empirical Model

The software builds a model which is a polynomial approximation for

the true and unknown relationship between each response and the

factors.

Here is the complete model (up to 3rd order) for a three-level

experiment involving three factors, x1, x2, and x3.

0 1 1 2 2 3 3

12 1 2 13 1 3 23 2 3 123 1 2 3

2 2 211 1 22 2 33 3

2 2 2 2 2112 1 2 113 1 3 122 1 2 133 1 3 223 2 3 233 2 3

3 3 3111 1 222 2 333 3

y b b x b x b x

b x x b x x b x x b x x x

b x b x b x

b x x b x x b x x b x x b x x b x x

b x b x b x ε+

= + + +

+ + + +

+ + +

+ + + + + +

+ + + +

2

Model Term

Definition

y

The estimated average response over the experiment space.

0b

Constant.

1 1 2 2 3 3, , b x b x b x

Main effects representing the linear part of the model.

Introduction to Design of Experiments, Planning and Execution, July ‘05 16

Page 17: Planning for DOE, Lesson 1

The Empirical Model cont.

Model Term

Definition

12 1 2 13 1 3 23 2 3, , b x x b x x b x x

The two-factor interactions representing mild curvature in the model.

123 1 2 3b x x x

The three-factor interaction. Usually considered small or not important for many processes.

2 2

11 1 22 2 33 3, , b x b x b x2

Quadratic effects representing significant curvature in the model.

2

112 1 2b x x , , , 2113 1 3b x x 2

122 1 2b x x2

133 1 3b x x , , 2223 2 3b x x 2

233 2 3b x x

Partial cubic effects. Usually considered small or not important for many processes.

3

111 1b x , , 3222 2b x 3

333 3b x

Cubic effects. Usually considered small or not important for many processes.

ε

The error term representing common cause variation not accounted for by the other terms. Assumed to have a normal distribution with mean zero and variance, 2σ .

Note that you can select designs to estimate three-factor interactions, partial cubic, and cubic effects, but they are seldom important and therefore usually not worth the extra expense to estimate.

Introduction to Design of Experiments, Planning and Execution, July ‘05 17

Page 18: Planning for DOE, Lesson 1

Steps for Completing a DOE The font size indicates the relative importance for having a successful DOE outcome.

1. Planning

2. Select Experiment Design

3. Execute Experiment & Collect Data

4. Analyze Data

5. Perform best-setup confirmation run, or validation

Introduction to Design of Experiments, Planning and Execution, July ‘05 18

Page 19: Planning for DOE, Lesson 1

The sequential Nature of DOE

DOE is iterative; rarely does one run a large, comprehensive design

in which a final conclusion is made.

Introduction to Design of Experiments, Planning and Execution, July ‘05 19

Factor A

Direction of better performance

First DOE

Second DOE

Factor B

Page 20: Planning for DOE, Lesson 1

Planning for DOE

Introduction to Design of Experiments, Planning and Execution, July ‘05 20

Page 21: Planning for DOE, Lesson 1

DOE Planning Prior to Selecting an Experiment Design

Introduction to Design of Experiments, Planning and Execution, July ‘05 21

Define the process you want to study

Define experiment objectives

Identify responses and measurement methods (IQ/OQ)

Identify process variables and measurement methods (IQ/OQ)

Start

Are interactions between factors likely?

Is curvature in the response likely?

Will any factor be a nuisance to vary?

End

Make sure process is stable (IQ/OQ)

Page 22: Planning for DOE, Lesson 1

Define the Process You Want To Study

• A process is a sequence of tasks required to accomplish something, or

• A process is a means of transforming inputs into outputs

IPO process model

Introduction to Design of Experiments, Planning and Execution, July ‘05 22

Product Design Influences

Product Performance

Variation in Product Performance

Equipment & Tooling

Procedures & Methods

Operator Influences

Material Properties

Environmental Influences

Process

Controllable process variables (Inputs)

Responses (Outputs)

Uncontrollable process variables (Noise)

Page 23: Planning for DOE, Lesson 1

Define the Experiment Objectives

• Clear statement of what you want to accomplish • Prioritize objectives

• State constraints

• Get team agreement

Poorly Stated Objective

Maximize seal strength.

Better Objective

Learn the process variable settings that maximize seal strength subject to the following constraints: (1) Sterile barrier pouches must pass the SOP leak test; (2) The visual appearance of the seals must pass SOP

requirements; and (3) Throughput must be greater than or equal to 20 pouches per

minute.

Introduction to Design of Experiments, Planning and Execution, July ‘05 23

Page 24: Planning for DOE, Lesson 1

Identify Responses • Select responses that are good predictors of field performance.

• Use the mean of several observations from a run as the official response value for a run.

• Use the standard deviation of several observations from a run as an additional response representing the variation in the process.

• Measure responses as continuous variables instead of categorical variables, whenever possible.

• If unsure about two measurement methods, use both.

• Omitting a key response variable can be a big mistake.

• Choosing responses and their measurement methods are typically the most time-consuming aspects of DOE planning.

Introduction to Design of Experiments, Planning and Execution, July ‘05 24

Page 25: Planning for DOE, Lesson 1

The Sample Mean

If n observations are denoted by , their mean is: 1 2,, , nx x xL

1 21 ( )nx x xnx + + += L .

Or, in more compact notation:

1ix xn= ∑

Suppose that these response data were taken during a particular run:

54 59 35 41 46

The mean response for the run is

1 (54 59 35 41 465

47

1ix xn

= + + + +

=

= ∑

Introduction to Design of Experiments, Planning and Execution, July ‘05 25

)

Page 26: Planning for DOE, Lesson 1

The Sample Standard Deviation

An experiment objective might be to minimize the standard deviation,

or at least make sure that there is no obvious indication that the

standard deviation appears noticeably worse within the

manufacturing window you end up selecting.

If n observations are denoted by , their variance is: 1 2,, , nx x xL

2 2 21 2

1 ( ) ( ) ( )1 ns x x x x xn⎡ ⎤⎣ ⎦= − + − + + −

−L 2x .

Or more compactly:

2 21 ( )1 ii

s xn= −− ∑ x

The standard deviation, s, is the square root of the variance

2s s=

Introduction to Design of Experiments, Planning and Execution, July ‘05 26

Page 27: Planning for DOE, Lesson 1

The Sample Standard Deviation cont.

Suppose these response data were taken during a particular run:

54 59 35 41 46

The sample standard deviation response is

2

2 2

2 2 2 2

1 ( )1

1 (54 47) (59 47) (35 47) (41 47) (46 47)1

93.51

9.67

i

s

s x xn

n

s

⎡ ⎤⎣ ⎦

= −−

= − + − + − + − + −−

=

=

=

∑ 2

Introduction to Design of Experiments, Planning and Execution, July ‘05 27

Page 28: Planning for DOE, Lesson 1

The %CV Like the standard deviation, the coefficient of variation (%CV) may be

used as a response representing variation in performance. An

experiment objective might be to minimize the %CV.

The %CV of n observations 1 2,, , nx x xL is

% sample standard deviation 100 100sample meanCV sx= ⋅ ⋅=

For many processes, the variation increases as the mean increases. In this situation, the %CV is a better response than the standard deviation. See Appendix B for more information.

Introduction to Design of Experiments, Planning and Execution, July ‘05 28

Page 29: Planning for DOE, Lesson 1

Pass/Fail Responses

Consider as a last resort because the sample size requirements for

each run can reach into the hundreds

The percent defective for DOE run i, is

number defective in runˆ 100total number tested in runip = ⋅

Strategies:

• Substitute a continuous response whenever possible • If a Pass/Fail test must be used, redefine it so more test articles

fail

o Change a 10 psig test to a 30 psig test

• If a Pass/Fail test must be used, rate each test article as to the extent of failure

o 0=no failure, 1=25% failure, 2=50%, 3=75%, 4=100%

failure

Introduction to Design of Experiments, Planning and Execution, July ‘05 29

Page 30: Planning for DOE, Lesson 1

More Than One Sample Per Run Usually Required

The software assumes that the response value you provide for a particular run is close to the true (long-run) response for that run. An approach for getting near the true response value for the run is to use the mean of multiple samples processed during the run.

Response (units)

dens

ity

0 3 6 9 12 150

0.1

0.2

0.3

0.4

0.5

Run #1 Run #2

Long-run Performance

Suppose you make one observation per run. . .

You might get lucky and get these values; then the software can

accurately estimate the difference between the runs.

You might get these values; then the software will over estimate the difference between the runs.

You might get these values; then the software will underestimate the difference between the runs.

Introduction to Design of Experiments, Planning and Execution, July ‘05 30

Page 31: Planning for DOE, Lesson 1

What Sample Size (# Observations per Run) Do I Need? Use available feasibility data taken under a single set of process

conditions, and plot the data as shown below.

After about 12 test articles (for this example only) the mean remains

roughly unchanged.

n

mea

ns o

f the

firs

t n te

st a

rticl

es

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

-2

-1

0

1

2

3

4

Note: all 20 test articles were from the same run

Introduction to Design of Experiments, Planning and Execution, July ‘05 31

Page 32: Planning for DOE, Lesson 1

More on Sample Size and Noise

• The greater the variation (noise) in the response, the larger the

sample size needed to get good estimates for the response

values for the runs.

• To reduce noise, consider reporting the official measurement of

a test article as the mean of several measurements made on

that test article

o Use when there is noticeable within-test article variation

o Use when there is noticeable measurement variation

• The process must be stable during the execution of a DOE;

otherwise, process drift or jumps will likely obscure the real

effects of the factors

o Stability means in a state of statistical control. See

references on statistical process control (SPC).

Introduction to Design of Experiments, Planning and Execution, July ‘05 32

• Rule-of-thumb: it usually takes about twice as many samples

(observations) to get as good an estimate of the standard

deviation as it does the mean; however, most experimenters

will go with the sample size required for the mean and glean

whatever information they can about the standard deviation

response. This approach works pretty well unless your primary

experiment objective is to study and reduce process variation.

Page 33: Planning for DOE, Lesson 1

Response Calculation Worksheet (an example)

In this example, there were 10 samples used in Run #1 and 5 measurements per sample. The “Run Mean” and “Run SD” were the response values for Run #1 that were entered into StatGraphics

Introduction to Design of Experiments, Planning and Execution, July ‘05 33

Page 34: Planning for DOE, Lesson 1

Measuring the Response

• Measurement processes include:

o instruments (gages)

o procedures

o fixtures

o software

o people

o environment

o assumptions

• Basic requirements for measurement processes:

o good discrimination

o adequate range

o low bias

o good repeatability

Introduction to Design of Experiments, Planning and Execution, July ‘05 34

Page 35: Planning for DOE, Lesson 1

Measurement Discrimination • Discrimination is the smallest change response that the

measurement process can faithfully detect and indicate

• Discrimination should be no larger than about 1/10 of the

smallest difference in the response you require the experiment to

detect

Response (units)

dens

ity

0 3 6 9 12 150

0.1

0.2

0.3

0.4

0.5

Run #1 Run #2

Difference in Response

Long-run Performance

Introduction to Design of Experiments, Planning and Execution, July ‘05 35

Page 36: Planning for DOE, Lesson 1

Measurement Discrimination cont. • Alternatively, the discrimination should divide the process

variation into 10 parts or more (rule of tens). The process

variation is the long run, ±3σ spread in the response observed

over a single set of process conditions. The ±3σ spread is also

called the natural tolerance of the process.

Introduction to Design of Experiments, Planning and Execution, July ‘05 36

response measured in standard deviations

dens

ity

-4 -3 -2 -1 0 1 2 3 40

0.1

0.2

0.3

0.4

0.5

99.7% of observations

95% of observations

68% of observations

Normal Distribution Model

Page 37: Planning for DOE, Lesson 1

Measurement Discrimination cont.

The process standard deviation, �, may be estimated as follows:

4ˆ s

cσ = ,

where s is the center line from a standard deviation control chart,

and is a constant found in SPC references. 4c

Alternatively,

ˆ sσ = ,

Introduction to Design of Experiments, Planning and Execution, July ‘05 37

where is the sample standard deviation of n observations made over

a single set of process conditions.

s

Page 38: Planning for DOE, Lesson 1

Measurement Range & Bias

• Range refers to the smallest and largest values that a

measurement process can faithfully detect and indicate. Select

a gage will adequate range.

• Bias2 is the difference between the mean of multiple

measurements on the same reference standard and the

accepted value of the reference standard. Bias is a smaller-

the-better characteristic. Bias is usually handled through

calibration.

Introduction to Design of Experiments, Planning and Execution, July ‘05 38

2 Bias is sometimes referred to as “inaccuracy”

Page 39: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 39

Measurement Repeatability Repeatability is that portion of the observed process variation

attributable to the measurement process alone (assumes the same

operator makes all of the measurements).

Repeatability may be thought of as the sample standard deviation of

n measurements made at the same location on the same test article,

by the same person3.

Guidelines:

• Repeatability must be small compared to the smallest difference in the response that you want the experiment to be able to detect

• Alternatively, repeatability must be small compared to the ±3σ

process variation

• Where appropriate, report the official measurement for a test article as the mean of two or more observations made on that test article

3 This working definition may be used in the early stages of the DOE effort, if the results of a formal gage capability study are not be available

Page 40: Planning for DOE, Lesson 1

ction to Design of Experiments, Planning and Execution, July ‘05 40

Response Worksheet

Response Procedure & Spec

Estimated Range of

Data � � � / � Gage

Disc. Gage Range

Calib. ID

Low: High:

Low: High:

Low: High:

Low: High:

Low: High:

Low: High:

Low: High:

Low: High:

� = Smallest change in the response that is worth investigating � = Standard deviation of the experimental error (standard deviation of the means of identical runs)

Introdu

Page 41: Planning for DOE, Lesson 1

Completing the Response Worksheet

• For each line item, think about how you can reduce the overall

noise in the experiment.

• � / � expresses the smallest change in the response that is

worth investigating, as a multiple of the background noise.

• You use � / � in the StatGraphics Power Curve procedure to

calculate the number of extra replicate runs you need to add to

the base design.

• If a transformation is to be performed on a response variable,

then � and � need to be in transformed units.

Estimate � (sigma) as follows: 1. Select one set of run conditions 2. Set up, stabilize and run the process in the same manner as you

will for the DOE 3. Collect n observations, and calculate their mean 4. Completely reset the process 5. Repeat steps 2 through 4 two or more times.

Introduction to Design of Experiments, Planning and Execution, July ‘05 41

6. Calculate the standard deviation of the three or more means.

Page 42: Planning for DOE, Lesson 1

Process Variables Worksheet (example)

Constant, Noise, or

Factor Process Variable

DOE Value

Operator Influences C Process operator Single operator, Peng C Af measurement operator Single operator, Peng

Material Properties

C BioPhysio valve Model 3100 C BioPhysio 2-D flat pattern 193396-029 Rev E, Lot # 0989MG 3011/248-10 C Nitinol flat sheet MetalTex, p/n 193402, Lot #248-10 C Salt Nitrate Salt, p/n 123456001

C Does salt go bad over time or over # parts processed? No

C Quench water Tap water. Room temperature (20°C ± 5°C),

Equipment and Tooling C Shape-setting mandrel, 29 mm p/n 392664 C Salt bath Ajax Hultgren Salt Bath Furnace, model # 3100. C Mandrel rack, 5 up p/n 392827

C Mixer Accumix variable speed controller model MM23401C. ¼ Hp motor. Propeller location and configuration unknown.

C Salt bath temp controller

For gross adjustment only; record setting for each DOE run, but use the chart recorder to setup the DOE runs. Met ID 116229.

C Chart recorder & Temperature probe Met ID 117406

Procedures and Methods C Shape setting SOP 2394 red-lined, attached C Bath stabilization time >= 90 minutes

C Test article position on mandrel rack

Position the test article on the mandrel location marked. Do not load frames on the remaining 4 mandrels (assumption: the missing frames will not invalidate the DOE results with respect to how the process will behave in production).

C Number of mandrels on rack 5 mandrels C Orientation of rack in bath Test article at 6:00 with respect to operator.

C

Depth of salt in bath Start DOE: 21.5 inches

Introduction to Design of Experiments, Planning and Execution, July ‘05 42

Minimum height is 21 – 22 inches from bottom of bath.

C Depth of valve frames in bath Fixed by the geometry of the mandrel rack and by the depth of the salt.

Page 43: Planning for DOE, Lesson 1

Process Variables Worksheet (example) cont.

Constant, Noise, or

Factor Process Variable

DOE Value

Procedures and Methods continued

F

Temperature This is the temperature read from the chart recorder attached to the temperature probe in the vicinity of the shape setting mandrel. The temperature probe is located at approximately the same depth as the parts being shape set. Bath stabilization time after a DOE temperature adjustment, >= 15 minutes 556 – 564 oC

F Time 240 – 300 sec

F

Mixer RPM 20 knob setting = 368 spindle RPM 25 knob setting = 460 spindle RPM 30 knob setting = 552 spindle RPM No stabilization time required after DOE mixer adjustment. 2-unit knob increments. 20 - 30

C Time between lifting the product out of bath and quench. < 5 sec

C Quench time >= 60 sec

Environmental Influences C Salt bath location COE lab, first floor T&D building

Product Design Influences N/A

Introduction to Design of Experiments, Planning and Execution, July ‘05 43

N/A N/A

Page 44: Planning for DOE, Lesson 1

Completing the Process Variables Worksheet

• For each line item, think about how you can reduce the overall noise in the experiment.

• Serves as good documentation.

• Include all variables, even those believed only remotely influential.

• Keep in mind the six process variables categories.

• Brainstorm as the team watches the process operation or the

product build.

• Brainstorm as the team reviews applicable SOPs, process sheets, equipment manuals, and the like.

• Consider measurement discrimination, range, and bias as you

do for responses.

• Consider completing an input variables worksheet for each response measurement process.

Introduction to Design of Experiments, Planning and Execution, July ‘05 44

• Failure to include an important factor in your DOE will likely make the results unusable, especially if the factor is involved in an interaction.

Page 45: Planning for DOE, Lesson 1

Process Variables of the Noise Variety

• Noise variables are not closely controlled because, (1) they

are thought to have at most a small effect on performance,

and (2) they are considered difficult to closely control in long

term production.

• During the execution of the DOE, hold noise variables

constant if you can, and record their values.

Introduction to Design of Experiments, Planning and Execution, July ‘05 45

Page 46: Planning for DOE, Lesson 1

The Importance of Wide Factor Level Ranges

• Wide ranges magnify effects, making it easier for the software

to detect which factors are important and the extent of their

importance.

• Wide ranges minimize the need for extrapolation post-DOE.

Introduction to Design of Experiments, Planning and Execution, July ‘05 46

Mag

nifie

d ef

fect

Response

Small effect

Factor A

Possible DOE factor ranges

Page 47: Planning for DOE, Lesson 1

Wide Factor Ranges, But Not Too Wide

• Typically wider than production ranges.

• Some runs may yield bad product; that’s OK, as long as the

product is recognizable and measurable in terms of the

responses.

• Make sure that all of the runs are doable before executing the

DOE; one missing run can prevent you from drawing useful

conclusions.

• The DOE procedure works only when the response is “smooth”

over the experiment space, so avoid factor ranges that yield

responses of zero or infinity.

Introduction to Design of Experiments, Planning and Execution, July ‘05 47

Page 48: Planning for DOE, Lesson 1

Signal-to-Noise Review DOE operates on a signal-to-noise basis; signals are the effects of

the factors; noise includes the variation in the measurement process,

the inherent variation in the process under study, and the variation

within individual test articles.

Increase signal where you can

• Use wide factor ranges Decrease noise where you can

• Report response values as the means of several test articles per run.

• Where appropriate, report the official measurement for a test

article as the mean of two or more observations made on that test article.

• Make sure the process is stable prior to executing the DOE.

• Thoroughly account for all process variables and control them

during the execution of the DOE.

• During the execution of the DOE, treat each run and each test article identically except for the factor level changes indicated by the experiment design.

• During the execution of the DOE, wait for the process to

stabilize after making factor level changes.

Introduction to Design of Experiments, Planning and Execution, July ‘05 48

Page 49: Planning for DOE, Lesson 1

What about Factors that are a Nuisance to Vary?

Sort the design by the nuisance factor, and perform the runs in the

sorted order; that way you minimize the number of level changes

for the nuisance factor.

• With this approach, lurking variables may influence the

experiment, so make sure that the process conditions are

identical from run to run.

Introduction to Design of Experiments, Planning and Execution, July ‘05 49

• Of course, the best approach for guarding against lurking

variables is to randomize the run order.

Page 50: Planning for DOE, Lesson 1

Factors, Additional Notes If factor levels are indicated by a knob position that does not

correspond to a calibrated scale. . .

• Use an independent gage to learn the true value of each factor level; it may be that knob position does not correspond to a linear scale. For example, knob position #4 may not correspond to twice the factor level of knob position #2. This situation can be a big barrier to analyzing your experiment.

Factors, continuous and categorical

• The levels for continuous factors can be set to (almost) any value along some region of the real number line.

• The levels for categorical factors fall into a finite number of

categories.

• Use continuous factors whenever possible.

Introduction to Design of Experiments, Planning and Execution, July ‘05 50

Page 51: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 51

Executing a DOE

Page 52: Planning for DOE, Lesson 1

Checklist for Executing a DOE

____

Are there enough test articles and materials for the DOE plus extra to cover mishaps and allow for a confirmation experiment?

____

Is the principal investigator available to answer questions? (It’s a good practice for the principle investigator to be present for the first couple of runs to help make sure that there are no misunderstandings about how the DOE should be executed)

Has team practiced/rehearsed all procedures related to the DOE? ____

Will the process be stable for the duration of the DOE execution?

____ Perform runs in the order specified.

____ Note any unusual occurrences during the DOE; record the time, run

#, and test article number. ____

Hold all conditions the same for every run and every test article except for the specified factor level changes.

____

Allow the process to warm up and stabilize after factor level changes.

____

Save all test articles at least until the DOE report is approved.

____

Always record individual data observations, even if means or other statistics will be used as “official” response values.

____

If a run is accidentally skipped, do it at the end of the experiment; the run order is not as important as completing all of the runs.

____

Introduction to Design of Experiments, Planning and Execution, July ‘05 52

Know the minimum number of decimal places for recording the data.

Page 53: Planning for DOE, Lesson 1

Final Thoughts

• Include DOE tasks in your project schedule.

• Begin collecting DOE planning information long before you

need to execute your DOE.

• Process knowledge and engineering/scientific expertise

must be applied throughout the DOE process; the key is to

get good data; statistical aspects are of secondary

importance.

• Team effort

Introduction to Design of Experiments, Planning and Execution, July ‘05 53

Page 54: Planning for DOE, Lesson 1

Appendix A

If your response variables include both the mean and standard

deviation of a performance characteristic, plot the standard deviations

vs their corresponding means (choose Plot… Scatterplots… X-Y

Plot… from the menu bar). If the plot shows the standard deviations

increasing with the means, as shown below, it’s possible that one of

the basic DOE assumptions (homogeneity of variances) has not been

met. Take the natural log of the individual response measurements,

generate the scatter plot again, and see if the transformation corrects

the problem; if the transformation solves the problem, analyze the log

response data just as you would without the transformation.

Introduction to Design of Experiments, Planning and Execution, July ‘05 54

Run Means

Run

Sta

ndar

d D

evia

tions

510

510.

5

511

511.

5

512

512.

5

513

513.

5

(X 0.001)

4.2

6.2

8.2

10.2

12.2

14.2(X 0.0001)

Page 55: Planning for DOE, Lesson 1

References Box, G. E. P., Hunter, W. G., and Hunter, J.S. (1978). Statistics for

Experimenters. Wiley, New York

Schmidt, S. R. and Launsby R. G. (1998). Understanding Industrial

Designed Experiments 4th ed. Air Academy Press, Colorado Springs,

Colorado

Taylor, W. T. (1995). Screening Experiments. Baxter Healthcare

Corp. short course

Hahn, G. J. (1977). Some Things Experimenters Should Know About

Experimental Design. J. Quality Technology. Vol. 9, No. 1

Cusimano J. (1996). Understanding & Using Design of Experiments.

Quality, April 1996

Introduction to Design of Experiments, Planning and Execution, July ‘05 55

Page 56: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 56

Notes

Page 57: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 57

Notes

Page 58: Planning for DOE, Lesson 1

Introduction to Design of Experiments, Planning and Execution, July ‘05 58

Notes