Understanding the Accuracy of Assembly Variation Analysis Methods

ADCATS 2000

Robert Cvetko

June 2000

June 2000 ADCATS 2000 Slide 2

Problem Statement

There are several different analysis methods An engineer will often use one method for

all situations The confidence level of the results is

seldom estimated

June 2000 ADCATS 2000 Slide 3

Outline of Presentation

New metrics to help estimate accuracy Estimating accuracy (one-way clutch)

Monte Carlo (MC)RSS linear (RSS)

Method selection technique to match the error of input information with the analysis

Sample Problem

One-way Clutch Assembly

June 2000 ADCATS 2000 Slide 5

Clutch Assembly Problem

e

c

ba

c

Contact angle important for performance

Known to be quite non-quadratic

Easily represented in explicit and implicit form

June 2000 ADCATS 2000 Slide 6

Details for the Clutch Assembly

Cost of “bad” clutch is $20

Optimum point is the nominal angle

Variable Mean Standard Deviation a - hub radius 27.645 mm 0.01666 mm c - roller radius 11.430 mm 0.00333 mm e - ring radius 50.800 mm 0.00416 mm

Contact Angle Value (degrees)Upper Limit 7.6184

Nominal Angle 7.0184Lower Limit 6.4184

June 2000 ADCATS 2000 Slide 7

Monte Carlo Benchmark

Value (mean)..................................... 7.014953 (Standard Deviation)........... 0.219668 (Skewness)............................ -0.094419 (Kurtosis)............................... 3.023816

2.6814,4062,1666,572

Quality Loss ($/part)..................

Contact Angle for the Clutch

Lower Rejects (ppm).................Upper Rejects (ppm).................Total Rejects (ppm)....................

(One Billion Samples)

June 2000 ADCATS 2000 Slide 8

Monte Carlo - 1,000 RunsRun #1(10,000 Samples) Max/Min Std Dev

7.01111 7.02288/ 7.00846 .002203

2 0.04893 0.05036/0.04598 .000717

-.00086 -.00011/-.00184 .000263

0.00732 0.00788/ 0.00628 .000251

10,000 Sample Monte Carlo

There is significant variability even using Monte Carlo with 10,000 samples.

June 2000 ADCATS 2000 Slide 9

One-Sigma Bound on the Mean

Estimate of the Mean versus Sample Size

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

-3 -2 -1 0 1 2 3

Estimate of the Mean

Pro

bab

ility

De

nsi

ty f

or

the

E

sti

ma

te o

f th

e M

ean 16 samples

= 0.25

1 sample = 1

4 samples = 0.5

June 2000 ADCATS 2000 Slide 10

New Metric: Standard Moment Error

Dimensionless measure of error in a distribution moment

All moments scaled by the standard deviation

i

iiSERi

ˆ

Estimate True

June 2000 ADCATS 2000 Slide 11

SER1 for Monte Carlo

SER1 versus One-Simga Bound

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+00 1.E+02 1.E+04 1.E+06 1.E+08

Sample Size (log scale)

SE

R (

log

sc

ale

)1Simga

SER1a

SER1b

n

n

n

SER

SER

1

1ˆVariance

onDistributi Normal Standardˆ

1

1121

11

June 2000 ADCATS 2000 Slide 12

SER2 for Monte Carlo

SER2 versus One-Simga Bound

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+00 1.E+02 1.E+04 1.E+06 1.E+08

Sample Size (log scale)

SE

R (

log

sc

ale

)

1Simga

SER2a

SER2b

1

2

1

2ˆVariance

1

2ˆVariance

12ˆ1

Variance

12 variance,1mean

:onDistributi Square-Chiˆ1

2

2

2

2

2

2

2

2

2

2

2

n

n

n

nn

nn

n

SER

June 2000 ADCATS 2000 Slide 13

SER3-4 for Monte Carlo

2

43

nSER6

1004

nSER

SER3 versus One-Simga Bound

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+00 1.E+02 1.E+04 1.E+06 1.E+08

Sample Size (log scale)

SE

R (

log

sc

ale

)

1Simga

SER3a

SER3b

SER4 versus One-Simga Bound

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+00 1.E+02 1.E+04 1.E+06 1.E+08

Sample Size (log scale)

SE

R (

log

sca

le)

1Simga

SER4a

SER4b

June 2000 ADCATS 2000 Slide 14

Standard Moment Errors

One-Sigma Bound for SER1-4 versus Sample Size

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E+08

Sample Size (log scale)

SE

R (

log

sca

le)

-

SER4

SER3

SER2

SER1

June 2000 ADCATS 2000 Slide 15

Monte Carlo - 1,000 Runs Est 68%Run #1(10,000 Samples) Max/Min Std Dev Conf Int

7.01111 7.02288/ 7.00846 .002203 ± .002212

2 0.04893 0.05036/0.04598 .000717 ± .000692

-.00086 -.00011/-.00184 .000263 ± .000212

0.00732 0.00788/ 0.00628 .000251 ± .000233

10,000 Sample Monte Carlo

You don’t have to do multiple Monte CarloSimulations to estimate the error!

June 2000 ADCATS 2000 Slide 16

Application: Quality Loss Function

2

2

2

21

2

2

2

12

,

1

2ˆ

1ˆ)ˆ(2

21

nK

nmK

SER

L

SER

LSERSERTotalL

2

min12

min )(

)()(

KK

dfK

dfLL

m

f()

L()

1

June 2000 ADCATS 2000 Slide 17

Estimating Quality Loss with MC%Error in Quality Loss for Monte Carlo

0.001%

0.010%

0.100%

1.000%

10.000%

100.000%

1.E+00 1.E+02 1.E+04 1.E+06 1.E+08 1.E+10

Sample Size (log scale)

%E

rro

r (l

og

sca

le)

%Error

1Sigma

2222

ecba ecba

RSS Linear Analysis

Using First-Order Sensitivities

June 2000 ADCATS 2000 Slide 19

New Metric: Quadratic Ratio

Dimensionless ratio of quadratic to linear effect

Function of derivatives and standard deviation of one input variable

a

aaa

a b

b

a

fa

f

QR

2

2

2

1

June 2000 ADCATS 2000 Slide 20

Calculating the QR

The variables that have the largest %contribution to variance or standard deviations

The hub radius a contributes over 80% of the variance and has the largest standard deviation

a c e1st Derivative -11.91 -23.73 11.822nd Derivative -20.11 -81.05 -20.41Standard Deviation 0.01666 0.00333 0.00416QR (quadratic ratio) -0.0141 -0.0057 -0.0036

Input Variable

June 2000 ADCATS 2000 Slide 21

Linearization Error

42

3

2

60604

863

22

1

QRQRaSER

QRQRaSER

QRaSER

QRaSER

First and second-order moments as function of one variable

Simplified SER estimates for normal input variables

June 2000 ADCATS 2000 Slide 22

Linearization of Clutch

The QR is effective at estimating the reduction in error that could be achieved by using a second-order method

If the accuracy of the linear method is not enough, a more complex model could be used

Quadratic Ratio of a

RSS vs. Method of

System Moments

RSS vs. Benchmark

SER1 0.0141 0.0156 0.0157SER2 -0.0004 -0.0004 -0.0034SER3 0.0844 0.0936 0.0944SER4 -0.0119 -0.0144 -0.0441

Error Estimates Obtained From:

Method Selection

Matching Input and Analysis Errorand Matching Method with Objective

June 2000 ADCATS 2000 Slide 24

Error Matching

“Things should be made as simple as possible, but not any simpler”-Albert Einstein

Method complexity increases with accuracy

Simplicity Reduce computation error Design iteration Presenting results

Input Error

Analysis Error

June 2000 ADCATS 2000 Slide 25

Converting Input Errors to SER2

Incomplete assembly model

Input variable Specification limits Loss constant

n

iiSERiSER

1

2,22 ion%Contribut

XX L

XSER

2

2

KSER K 1

2

June 2000 ADCATS 2000 Slide 26

Design Iteration Efficiency

Design Iteration

Efficiency

Accuracy

MSMDO

E

RSS

MC

June 2000 ADCATS 2000 Slide 27

Conclusions Confidence of analysis method should be

estimated Confidence of model inputs should be

estimated New metrics - SER and QR help to estimate

the error analysis method and input errors Error matching can help keep analysis

models simple and increase efficiency