New Results about Randomization and Split-Plotting by James M. Lucas 2003 Quality & Productivity Research Conference Yorktown Heights, New York May 21-23,

New Results about Randomization and Split-Plotting

byJames M. Lucas

2003 Quality & Productivity Research Conference

Yorktown Heights, New YorkMay 21-23, 2003

J. M. Lucas and Associates 2

Contact Information

James M. LucasJ. M. Lucas and Associates 5120 New Kent Road Wilmington, DE 19808 (302) 368-1214 [email protected]


Research Team Huey Ju Jeetu Ganju Frank Anbari Peter Goos

Malcolm Hazel Derek Webb John Borkowski

PRELIMINARIES

How do you run Experiments?


QUESTIONS How many of you are involved with

running experiments? How many of you “randomize” to guard

against trends or other unexpected events?

If the same level of a factor such as temperature is required on successive runs, how many of you set that factor to a neutral level and reset it?


ADDITIONAL QUESTIONS

How many of you have conducted experiments on the same process on which you have implemented a Quality Control Procedure?

What did you find?


COMPARING RESIDUAL STANDARD DEVIATION FROM AN EXPERIMENT WITHRESIDUAL STANDARD DEVIATION FROM AN IN-CONTROL PROCES

MY OBSERVATIONS

EXPERIMENTAL STANDARD DEVIATION IS LARGER. 1.5X TO 3X IS COMMON.


HOW SHOULD EXPERIMENTS BE CONDUCTED?

•“COMPLETE RANDOMIZATION” (and the completely randomized design)

•RANDOMIZED NOT RESET (Also Called Random Run Order (RRO) Experiments) (Often Achieved When Complete Randomization is Assumed)

•SPLIT PLOT BLOCKING (Especially When There are Hard-to-Change Factors)


Randomized Not Reset (RNR) Experiments

A large fraction (perhaps a large majority) of industrial experiments are Randomized not Reset (RNR) experiments

Properties of RNR experiments and a discussion of how experiments should be conducted: “Lk Factorial Experiments with Hard-to-Change and

Easy-to-Change Factors” Ju and Lucas, 2002, JQT 34, 411-421 [studies one H-T-C factor and uses Random Run Order (RRO) rather than RNR]

“Factorial Experiments when Factor Levels Are Not Necessarily Reset” Webb, Lucas and Borkowski, 2003, JQT, to appear [studies >1 HTC Factor]


RNR EXPERIMENTS (Random Run Order Without Resetting Factors)

OFTEN USED BY EXPERIMENTERS NEVER EXPLICITLY RECOMMENDED

ADVANTAGES•Often achieves successful results•Can be cost-effectiveDISADVANTAGES•Often can not be detected after experiment is conducted (Ganju and Lucas 99)•Biased tests of hypothesis (Ganju and Lucas 97, 02)•Can often be improved upon•Can miss significant control factors


Results for Experiments with Hard-to-Change and Easy-to-Change Factors

One H-T-C or E-T-C Factor: use split-plot blocking

Two H-T-C Factors: may split-plot Three or more H-T-C Factors:

consider RNR or Low Cost Options Consider “Diccon’s Rule”: Design

for the H-T-C Factor


New Results Joint work with Peter Goos Builds on the Kiefer-Wolfowitz

Equivalence Theorem Implications about Computer

generated designs (especially when there are Hard-to-Change Factors)


Kiefer-Wolfowitz Equivalence Theorem is the design probability measure M() = X’X/n (kxk matrix for a n point

design) d(x, ) = x’(M())-1x (normalized variance) So called Approximate Theory The following are equivalent: maximizes det M() minimizes d(x, ) Max (d(x, ) = k


Very Important Theorem Helps find Optimum Designs Basis for much computer aided design

work Justifies using |X’X| Criterion

Shows “Classical Designs” are great “Which Response Surface Design is Best”

Technometrics (1976) 16, 411-417 Computer generated designs not

needed for “standard” situations


Optimality Criteria Determinant (D-optimality)

Maximize |X’X| D-efficiency = {|X’X/n|/ |X*’X*/n*|}1/k where

X* is an optimum n* point design Global (G-optimality)

Minimize the maximum variance G-efficiency = k/Max d(x, )

G-efficiency < D-efficiency No bad designs with high G-efficiency


Computer Generated Design Arrays

Different criteria give different “n” point designs

Do not pick a single “n”Some “n” values may achieve an excellent

designCheck other criteria (especially G-)

Lucas (1978) “Discussion of: D-Optimal Fractions of Three Level Factorial Designs”

Borkowski (2003) “Using A Genetic Algorithm to Generate Small Exact Response Surface Designs”


Equivalence Theorem does not hold for Split-Plot Experiments D- and G- criteria converge to different

designs Example: r reps of a 23 Factorial (linear

terms model) Optimum design depends on d =w

2/2 where w is the whole-plot and is the split-plot error

For large values of d: D-optimal design has 4 r blocks with I = A = BC G-optimal design has 8r – 2 blocks (Number of

observations minus number of split-plot terms)


Computer Generated Split-Plot Experiments Useful Research Recent publications:

Trinca and Gilmour (2001) “Multi-stratum Response Surface Designs” Technometrics 43: 25-33

Goos and Vandebroek (2001) “Optimal Split-Plot Designs” JQT 33: 436-450

Goos and Vandebroek (2003) “Outperforming Completely Randomized Designs” JQT to appear

All use |X’X| Criterion


RELATED SPLIT-PLOT FINDINGS

SUPER EFFICIENT EXPERIMENTS (With One or Two Hard-to-Change Factor) SPLIT PLOT BLOCKING GIVES HIGHER PRECISION AND LOWER COSTS THAN COMPLETELY RANDOMIZED EXPERIMENTS


Design Precision: Calculating Maximum Variance Simplifications for 2k factorials Sum Variances of individual terms Whole plot terms:

w2/ number blocks + 2/ 2k

Split plot terms: 2/2k

Completely randomized design has variance: k(w

2+ 2)/ 2k

Blocking Observation to achieve Super Efficiency


26-1 with one or two Hard-to-Change Factors

Main Effects plus interaction Model 22 Terms = (1 + 6 + 15)

Use Resolution V, not VI with I=ABCDEUse four blocks I=A=BCF=ABCF=BCDE=ADEF=DEF

Nest Factor B within each A block giving a split-split-plot with 8 Blocks =B2=AB2=CF2=ACF2=CDE2=ABDEF2=BDEF2

I and A have variance 02/32 + 1

2/4 +22 /8

B, AB and CF have 02/32 + 2

2 /8 Other terms have variance 0

2/32 G-efficiency = 22(0

2+12+2

2)/(2202+161

2+2022 )

>1.0 Drop 2

2 terms for one h-t-c factor results


Observations Does not use Maximum Resolution

or Minimum Abberation Similar results for most 2k

factorials

Super Efficient Experiments are not always Optimal

26-1 Main effects plus 2FI model

G-optimum design has 12 blocks when d gets large


Conclusions Showed K-W Equivalence theorem

does not hold for Split-Plot Experiments

Discussed Implications Exciting research area Much more to do

Documents

New Results about Randomization and Split-Plotting by James M. Lucas 2003 Quality & Productivity Research Conference Yorktown Heights, New York May 21-23,