Design and Analysis of Experiments Lecture 4.1

Diploma in StatisticsDesign and Analysis of Experiments

Design and Analysis of ExperimentsLecture 4.1

1. Review of Lecture 3.2

2. More on Variance Components

3. Measurement System Analysis

Minute Test: How Much

How Much

Minute Test: How Fast

How Fast

Homework 3.2.1

Process improvement study, reduced model:

Y = + B + C + D + BC +

Set up a "design matrix" with columns for the significant effects, headed by the effect coefficients. Calculate a fitted value for each design point by applying the rows of signs to the effect coefficients and adding the overall mean. Crosscheck with the fitted values calculated by Minitab

Homework 3.2.1

Minitab provides estimated effects:

Term Effect

NaOHCon -17.250Speed 9.750Temp 21.750NaOHCon*Speed -7.500

Model?

Y = + B + C + D + BC +

Coef 49.750 -8.625 4.875 10.875 -3.750

Homework 3.2.1

Minitab provides fitted values,

residuals,

estimate via ANOVA

Analysis of Variance for Impurity (coded units)

Source DF SS MS F PMain Effects 3 3462.75 1154.25 381.86 0.0002-Way Interactions 1 225.00 225.00 74.44 0.000Residual Error 11 33.25 3.02Total 15 3721.00

Comparison of fits

All effect estimates are the same; SE's vary.

Lenth: s = 2.25, PSE = 1.125

Reduced: s = 1.74, SE(effect) = 0.87

Projected: s = 1.87, SE(effect) = 0.94

"Projected" model has 3 interactions missing from the "Reduced" model.

Degrees of freedom

"Error" degrees of freedom relevant for t

– check ANOVA table

– count estimated effects

– use replication structure

t5, .05 = 2.57

t8, .05 = 2.31

t11,.05 = 2.20

s = 2.25

s = 1.87

s = 1.74

Ref: EM Notes Ch. 4 pp. 3, 6

Ref: Extra Notes, Models for Experiments and Lab 2 Feedback

Design Point

A B C Y

1 – – – Y1

2 + – – Y2 3 – + – Y3 4 + + – Y4 5 – – + Y5 6 + – + Y6 7 – + + Y7 8 + + + Y8

Review of Lecture 3.2 Introduction to Fractional Factorial Designs

Each row gives design points for a 4-factor experiment

Fourth column estimates D main effect.

Fourth column also estimates ABC interaction effect.

In fact, fourth column estimates D + ABC in 24-1.

Fractional factorial designs

Full 24 requires 16 runs

Half the full 24 requires 8 runs

Saves resources, including time

Sacrifices high order interactions via aliasing

Check D = ABC, design generator

Derive ABC from first principles.

D aliased with ABC

4th column estimates D + ABC, = D if ABC = 0

Alias List

A = BCD

B = ACD

C = ABD

D = ABC

AB = CD

AC = BD

AD = BC

ABCD = Y

Part 2: More on Variance Components

• Identifying sources of variation

• Hierarchical design for variance component estimation

• Hierarchical ANOVA

Values

e = eB + eS + eT

Sources of variation in moisture content

Batchvariation

Samplingvariation

Testingvariation

Components of Variance

Recall basic model:

Y = + eB + eS + eT

Components of variance:2T

Hierarchical Design forVariance Component Estimation

Batch 1 2 3 4 5 Sample 1 2 3 4 5 6 7 8 9 10 Test 40 39 30 30 26 28 25 26 29 28 14 15 30 31 24 24 19 20 17 17 Batch 6 7 8 9 10 Sample 11 12 13 14 15 16 17 18 19 20 Test 33 32 26 24 23 24 32 33 34 34 29 29 27 27 31 31 13 16 27 24 Batch 11 12 13 14 15 Sample 21 22 23 24 25 26 27 28 29 30 Test 25 23 25 27 29 29 31 32 19 20 29 30 23 23 25 25 39 37 26 28

Minitab Nested ANOVA

Analysis of Variance for Test

Source DF SS MS F PBatch 14 1216.2333 86.8738 1.495 0.224Sample 15 871.5000 58.1000 64.556 0.000Error 30 27.0000 0.9000Total 59 2114.7333

Variance Components % ofSource Var Comp. Total StDevBatch 7.193 19.60 2.682Sample 28.600 77.94 5.348Error 0.900 2.45 0.949Total 36.693 6.058

Model for Nested ANOVA

Yijk = m + Bi + Sj(i) + Tk(ij)

SS(TO) = SS(B) + SS(S) + SS(T)

59 = 14 + 15 + 30

)YY()YY()YY(YY .ijijk..i.ij..iijk

Minitab Nested ANOVA

Expected Mean Squares

1 Batch 1.00(3) + 2.00(2) + 4.00(1)2 Sample 1.00(3) + 2.00(2)3 Error 1.00(3)

Translation:

EMS(Batch) =

EMS(Sample) =

EMS(Test) =

Calculation

= EMS(Test)

= ½[EMS(Sample) – EMS(Test)]

= ¼[EMS(Batch) – EMS(Sample)]

Estimation

= MS(Test)

= ½[MS(Sample) – MS(Test)]

= ¼[MS(Batch) – MS(Sample)]

Conclusions fromVariance Components Analysis

Variance Components % ofSource Var Comp. Total StDevBatch 7.193 19.60 2.682Sample 28.600 77.94 5.348Error 0.900 2.45 0.949Total 36.693 6.058

Sampling variation dominates, testing variation is relatively small.

Investigate sampling procedure.

Part 3: Measurement System Analysis

• Accuracy and Precision• Repeatability and Reproducibility• Components of measurement variation• Analysis of Variance• Case study: the MicroMeter

The MicroMeter optical comparator

• Place object on stage of travel table

• Align cross-hair with one edge

• Move and re-align cross-hair with other edge

• Read the change in alignment

• Sources of variation:

– instrument error

– operator error

– parts (manufacturing process) variation

Precise

Biased

Accurate

Characterising measurement variation;Accuracy and Precision

Imprecise

Characterising measurement variation;Accuracy and Precision

Centre and Spread

• Accurate means centre of spread is on target;

• Precise means extent of spread is small;

• Averaging repeated measurements improves precision, SE = /√n

– but not accuracy; seek assignable cause.

Accuracy and Precision: Example

Each of four technicians made six measurements of a standard (the 'true' measurement was 20.1), resulting in the following data:

Technician Data

1 20.2 19.9 20.1 20.4 20.2 20.4

2 19.9 20.2 19.5 20.4 20.6 19.4

3 20.6 20.5 20.7 20.6 20.8 21.0

4 20.1 19.9 20.2 19.9 21.1 20.0

Exercise: Make dotplots of the data. Assess the technicians for accuracy and precision

Accuracy and Precision: Example

Repeatability and Reproducability

Factors affecting measurement accuracy and precision may include:

– instrument

– material

– operator

– environment

– laboratory

– parts (manufacturing)

Repeatability and Reproducibility

Repeatability:

precision achievable under constant conditions:

– same instrument

– same material

– same operator

– same environment

– same laboratory

• How variable is measurement under these conditions

Reproducibility:

precision achievable under varying conditions:

– different instruments

– different material

– different operators

– changing environment

– different laboratories

• How much more variable is measurement under these conditions

Measurement Capability of the MicroMeter

4 operators measured each of 8 parts twice, with random ordering of parts, separately for each operator.

Three sources of variation:

– instrument error

– operator variation

– parts(manufacturing process) variation.

Data follow

Measurement Capability of the MicroMeter

Quantifying the variation

Each measurement incorporates components of variation from

– Operator error

– Parts variation

– Instrument error

and also

– Operator by Parts Interaction

Measurement Differences

Part Operator Repeats Diffs Part Operator Repeats Diffs

1 1 96.3 95.4 0.9 5 1 99.4 99.9 -0.5 2 97.0 96.9 0.1 2 100.1 99.8 0.3 3 98.2 97.4 0.8 3 100.9 99.4 1.5 4 97.4 99.6 -2.2 4 100.0 99.4 0.6

2 1 95.5 95.8 -0.3 6 1 93.8 94.9 -1.1 2 96.1 96.8 -0.7 2 95.9 95.8 0.1 3 97.9 99.4 -1.5 3 96.3 98.5 -2.2 4 97.3 100.0 -2.7 4 94.5 94.5 0

3 1 102.8 100.3 2.5 7 1 86.4 85.4 1 2 101.5 101.4 0.1 2 86.8 86.7 0.1 3 102.6 104.3 -1.7 3 88.2 89.6 -1.4 4 101.9 101.9 0 4 88.6 89.0 -0.4

4 1 94.6 96.2 -1.6 8 1 90.5 90.5 0 2 97.8 95.5 2.3 2 89.1 90.2 -1.1 3 96.0 94.3 1.7 3 92.9 92.1 0.8 4 95.3 94.4 0.9 4 92.1 92.4 -0.3

Graphical Analysis of Measurement Differences

Average measurementsby Operators and Parts

Graphical Analysis of Operators & Parts

876543210

Operator

Interaction Plot

Graphical Analysis of Operators & Ordered Parts

PartOrder

876543210

Operator

Interaction Plot, ordered by Parts

Quantifying the variation

Notation:

E: SD of instrument error variation

P: SD of parts (manufacturing process) variation

O: SD of operator variation

OP: SD of operator by parts interaction variation

T: SD of total measurement variation

Calculating sE

Part Operator Repeats ½(diff)2 Part Operator Repeats ½(diff)2

1 1 96.3 95.4 0.40 5 1 99.4 99.9 0.13 2 97.0 96.9 0.00 2 100.1 99.8 0.04 3 98.2 97.4 0.32 3 100.9 99.4 1.13 4 97.4 99.6 2.42 4 100.0 99.4 0.18

2 1 95.5 95.8 0.04 6 1 93.8 94.9 0.61 2 96.1 96.8 0.25 2 95.9 95.8 0.01 3 97.9 99.4 1.13 3 96.3 98.5 2.42 4 97.3 100.0 3.65 4 94.5 94.5 0.00

3 1 102.8 100.3 3.13 7 1 86.4 85.4 0.50 2 101.5 101.4 0.00 2 86.8 86.7 0.00 3 102.6 104.3 1.45 3 88.2 89.6 0.98 4 101.9 101.9 0.00 4 88.6 89.0 0.08

4 1 94.6 96.2 1.28 8 1 90.5 90.5 0.00 2 97.8 95.5 2.64 2 89.1 90.2 0.61 3 96.0 94.3 1.45 3 92.9 92.1 0.32 4 95.3 94.4 0.40 4 92.1 92.4 0.05

sum = 18.6 sum = 7.0

s2 = (18.6 + 7.0)/32 = 0.8

sE = 0.89

Analysis of Variance

Analysis of Variance for Diameter

Source DF SS MS F P

Operator 3 32.403 10.801 6.34 0.003Part 7 1193.189 170.456 100.02 0.000Operator*Part 21 35.787 1.704 2.13 0.026

Error 32 25.600 0.800

Total 63 1286.979

S = 0.894427

Basis for Random Effects ANOVA

F-ratios in ANOVA are ratios of Mean Squares

Check: F(O) = MS(O) / MS(O*P)

F(P) = MS(P) / MS(O*P)

F(OP) = MS(OP) / MS(E)

MS(O) estimates E2 + 2OP

2 + 16O2

MS(P) estimates E2 + 2OP

2 + 8P2

MS(OP) estimates E2 + 2OP

MS(E) estimates E2

Variance Components

Estimated StandardSource Value Deviation

Operator 0.5686 0.75

Part 21.0939 4.59

Operator*Part 0.4521 0.67

Error 0.8000 0.89

Diagnostic Analysis

Measurement system capability

E P means measurement system cannot distinguish between different parts.

Need E << P .

Define TP = sqrt(E2 + P

Capability ratio = TP / E should exceed 5

Repeatability and Reproducibility

Repeatabilty SD = E

Reproducibility SD = sqrt(O2 + OP

Total R&R = sqrt(O2 + OP

2 + E2)

Reading

EM §5.7, §7.5, §8.2

BHH, Ch. 5, §§6.1 - 6.3, §9.3

Design and Analysis of Experiments Lecture 4.1

Documents

Lecture 4 1:Lecture 4.1: AstrobiologyAstrobiology:

RELS 205 LECTURE 4.1

Lecture 12: 4.1~4.2 Euclidean Vector Space

Lecture 4.1: Relations Basics

ECON4150 - Introductory Econometrics Lecture 17: Experiments · ECON4150 - Introductory Econometrics Lecture 17: Experiments Monique de Haan (moniqued@econ.uio.no) ... Program evaluationis

Lecture 19: Program Evaluation and Quasi-Experiments

Diploma in Statistics Design and Analysis of Experiments Lecture 4.11 Design and Analysis of Experiments Lecture 4.1 Review of Lecture 3.1 Homework 3.1.1

Lecture 4. Experiments and Causal Design

4.1 Lecture 2008

Experimental Techniques Lecture 2 – Signals and experiments

CS2008 WK 5 Experiments I Lecture

Lecture of experiments

Design and Analysis of Experiments Lecture 1.2

1 Chapter 4: Design of Experiments 4.1 Why Experiment? 4.2 Introduction 4.3 Multi-Factor Experiments 4.4 Orthogonality and Blocking 4.5 Business Experiments

Lecture 1 Experiments, Models and Probabilities. Outline

Design and Analysis of Experiments Lecture 3.1

P301 Lecture 11 “Scattering experiments”

Lecture 9 Experiments in Psychoacoustics Martin Giese

Design and Analysis of Experiments Lecture 4 · Design and Analysis of Experiments Lecture 4.1 1. ... • General pattern of increasing rates from Block 1 to ... in a 24 design run

Design and Analysis of Experiments Lecture 2.1