Background Statement for SEMI Draft Document 4517 …downloads.semi.org/web/wstdsbal.nsf/2c1bceb05c9a0e... · NEW STANDARD: SEMI Statistical Guidelines for Ship to Control Note: This

Background Statement for SEMI Draft Document 4517 NEW STANDARD: SEMI Statistical Guidelines for Ship to Control Note: This background statement is not part of the balloted item. It is provided solely to assist the recipient in reaching an informed decision based on the rationale of the activity that preceded the creation of this document. Note: Recipients of this document are invited to submit, with their comments, notification of any relevant patented technology or copyrighted items of which they are aware and to provide supporting documentation. In this context, “patented technology” is defined as technology for which a patent has issued or has been applied for. In the latter case, only publicly available information on the contents of the patent application is to be provided. The Statistical Methods Task Force (SMTF) – Liquid Chemicals Committee has developed the attached ballot for the Guide: SEMI Statistical Guidelines for Ship to Control. This effort started in December 2005 and involved weekly teleconferences along with three to four longer face-to-face meetings annually. The motivating force for this intensive development effort is that there has been a growing push in the industry to require material suppliers to only ship materials that are in statistical control. This extension of using statistical control limits as either reporting limits or as additional specifications has profound consequences. Material suppliers and Independent Device Manufacturers (IDMs) will not calculate the same control limits unless all computational aspects are strictly defined, e.g., which data, which time frame, and which method of control limit calculation. Additionally numerous differing methodologies for ship to control (STC) had been proposed or were in limited industrial use. Lack of standardization will cause significant product fragmentation as well as being a source of supplier and IDM friction. The nature of much of the trace contamination data used in estimation of ship to control limits made the problem a difficult one. Typical materials have three data characteristics that make traditional ways of calculating control limits unsuitable. First, there are often many (>20) properties measured per sample. This dramatically inflates the Type I error (supplier’s risk) from the nominal 0.27% for a Shewhart type control chart to 5% or higher. Next, most trace contamination analytical data near to zero is skewed to the right and often has a low P/T ratio. If the skew (nonnormality) is not accounted for very high Type I errors can occur. Lastly, it is common for some trace contamination data to be partly or mostly censored to a method detection limit (MDL) and only reported as being <MDL (ex: <1 ppb). Such data censoring can be as high as 100%. Common ways of handling censored data provide control limits that are too tight versus the uncensored data distribution. Early in the development efforts it was found that use of simple classical statistical procedures such as three sigma control limits or alternative simple approaches such as using a 99th percentile or historical high as STC limits led to very high average product rejection rates for product originating from in statistical control processes. The elements of the procedures found in the Guide originated from statistical theory which was first tested via simulation studies and then subsequently performance tested in calibration studies with real industry data from a wide range of suppliers and products. In this process many alternatives were tested; the ones that worked best were accepted and then tuned such that they provided reasonable performance with real product data. In this process, key issues and decisions were often revisited in detail. The resulting Guide is the end product of a highly iterative team problem solving effort. The Task Force is confident that their intensive data based effort has resulted in a reasonable standardized approach. The Guide provides an effective means of dealing with many of the most difficult characteristics of semiconductor materials data. The SMTF is aware that software will be required to most efficiently implement the procedures in the Guide. An Excel spreadsheet has already been devised for this purpose and a Windows application is currently under development. For those desiring a better understanding of the Guide as well as some of its developmental history, a copy of the Workshop, Ship to Control Limits, provided at SEMICON/West 2007 is available on the SEMI website at http://teams.semi.org/stds_chem This letter ballot will be reviewed by the Statistical Methods Task Force and adjudicated by the Liquid Chemicals Committee at their meetings in San Diego, CA, during the week of 28th October, 2007.

This is a draft document of the SEMI International Standards program. No material on this page is to be construed as an official or adopted standard. Permission is granted to reproduce and/or distribute this document, in whole or in part, only within the scope of SEMI International Standards committee (document development) activity. All other reproduction and/or distribution without the prior written consent of SEMI is prohibited.

Page 1 Doc. 4517 © SEMI®

Semiconductor Equipment and Materials International 3081 Zanker Road San Jose, CA 95134-2127 Phone:408.943.6900 Fax: 408.943.7943

LETT

ER (Y

ELLO

W) B

ALL

OT

DRAFTDocument Number: 4517

Date: 9/12/2007

SEMI Draft Document 4517 NEW STANDARD: SEMI Statistical Guidelines for Ship to Control 1 Purpose 1.1 To provide a set of guidelines for the quantitative determination of statistically derived limits from process data for the purpose of defining and maintaining Ship to Control (STC) limits.

2 Scope 2.1 This guide applies to specified properties of specific products for which it is desired to use statistically derived control limits upon which customer notification, negotiated disposition of product, or statistical specifications can be based. NOTICE: This standard does not purport to address safety issues, if any, associated with its use. It is the responsibility of the users of this standard to establish appropriate safety and health practices and determine the applicability of regulatory or other limitations prior to use.

3 Referenced Standards and Documents 3.1 SEMI Standard SEMI C10 — Guide for Determination of Method Detection Limits. NOTICE: Unless otherwise indicated, all documents cited shall be the latest published versions.

4 Terminology 4.1 90th percentile test — a nonparametric test methodology for determining if reference and test data sets (4.11, 4.16) differ in the Annual Review Process (4.2). 4.2 annual review process (8.2) — the process by which STC Limits (4.14) are reviewed annually for possible change. 4.3 batch analysis — a measurement analysis that results in a single reported result for the specified property for the product batch in question. This result is assumed to apply to multiple shipping units when a production batch is divided into multiple product lots and batch sampling is appropriate as opposed to sampling each individual lot. 4.4 censored data — data whose measured value or non-measurable value has been replaced by a limit such as an MDL (4.6). 4.5 family type I error rate — for any individual sample, the probability that one or more characteristics will read OOC when a process is actually in statistical control (8.1, A1-3 ). 4.6 MDL — Method Detection Limit, a statistically derived figure of merit for a measurement system, see ¶3.1 for SEMI’s MDL methodology. 4.7 non-provisional STC limits — these limits are used to determine whether or not a measured product property is in statistical control. An STC limit (4.14) is non-provisional when there is enough uncensored data used in STC limit estimation (5.7, 8.1). Non-provisional STC Limits are changed upon Annual Review (8.2) only when significant differences (8.2) are found between the Reference Data (4.11) and the Test Data (4.16). 4.8 NP test methodology (8.2) — nonparametric test methodology, a particular combination of the use of Tukey’s Quick Test (4.17) and the 90th Percentile Test (4.1) in the Annual Review Process (4.2). 4.9 OOC product — Out of Control product, product for which one or more STC parameters exceed its STC upper control limit or, if applicable, fall below its STC lower control limit. 4.10 practical significance test — a test of whether STC Limits (4.14) have changed by enough to be concerned about in the Annual Review Process (4.2). 4.11 provisional STC limits — an STC limit (4.14) is provisional when there is too little uncensored data (5.7, 8.2) used in STC limit estimation (8.1). A provisional STC Limit is always updated upon Annual Review (8.2). How these limits are applied for the purpose of shipping material is left to the individual suppliers and producers to decide (5.6). 4.12 reference data — data from which the current STC Limits (4.14) were calculated as per ¶ 8.1. 4.13 skewness — a signed statistical measure of asymmetry in a data distribution (8.1).




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

4.14 STC limits — Ship to Control limits calculated by the provisions provided within SEMI STATISTICAL GUIDELINES FOR SHIP TO CONTROL (4.15). 4.15 STC — Ship to Control, the use of statistically derived control limits for the purpose of shipping decisions. 4.16 test data — data whose distribution is compared to the distribution of the Reference Data (4.11) in the Annual Review Process (4.2). 4.17 Tukey’s quick test (8.2) — a nonparametric test methodology for determining if reference and test data sets differ in the Annual Review Process (4.2).

5 Data Requirements NOTE 1: A series of requirements are set forth regarding data from which STC limits are to be determined. 5.1 Supplier’s process data is used to determine STC limits. STC limits are derived solely from the supplier’s process and are not broken out into customer specific STC limits or into STC limits that are specific to ship to locations. Data used in STC calculations is to be available for customer review. 5.2 Separate STC limits are required for different supplier manufacturing locations. 5.3 The prior year’s data is used to calculate STC limits. 5.4 Supplier’s historical process data can not span a significant process change. 5.5 A minimum of 100 measurements are required to determine STC limits. If the prior year does not provide this minimum sample size continue to sequentially add prior years one at a time until this minimum is achieved. Use the data from this expanded time frame to determine STC limits. 5.6 If application of ¶¶ 5.4 and 5.5 still provide a sample size that is less than 100 or does not include at least a year of data, it is up to the supplier and producer to decide whether or not to proceed with use of STC limits. If small sample size STC limits are used, the supplier and producer need to determine the manner in which such less stable STC limits are applied (A1-3.3). 5.7 For measurement properties for which the process data is too highly censored (less than 10 uncensored points or not enough points to perform Tukey’s Quick Test (8.2) in the Annual Review Methodology) then the calculated STC limit is provisional. 5.8 OOC STC data requires a root-cause analysis. If both a cause is found and steps are taken to insure that the likelihood of such a cause occurring in the future is either eliminated or greatly reduced, then this point will be considered an outlier that is to be removed from use in evaluating STC limits or the annual review status of STC limits. 5.9 For the case where a value from a single production batch is applied to multiple shipped lots (Batch Analysis data 4.3), that value should only be used once in the statistical calculations in ¶¶ 8.1-8.3. For example, if a batch analysis is performed where a production batch produces 10 shipping lots, there is only one measurement to use in ¶¶ 8.1-8.3, not 10. 5.10 When multiple batches are produced from the same raw material lot(s) and the raw material is the main source of product variation; then the use of the average of the batches made from the raw material lot(s) should be considered. This must be agreed on between supplier and producer.

6 Review of STC Limits 6.1 STC is not intended to replace current forms of routine process control monitoring. 6.2 For data sets with less than 100 measurements see ¶¶ 5.4 through 5.6. 6.3 Non-provisional STC limits will be evaluated for change on an annual basis as per ¶ 8.2. Only those non-provisional STC limits having statistically significantly different distributions and the control limits having a practically significantly difference will be modified upon annual review. 6.4 All provisional STC limits will automatically be recalculated upon annual review (8.2). For a provisional limit, when enough uncensored data is available upon annual review to meet ¶ 5.7, the STC limit in question will lose its provisional status upon recalculation and subsequently follow the standard annual review procedure in future annual reviews. 6.5 Changes to STC limits from the annual review process are to be treated as a package of changes. Neither suppliers nor producers get to select a preferred subset of STC limit changes.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

6.6 Process change handling: Whenever a change is made to a process or an analytical measurement system, it may be necessary to adjust STC limits and existing specifications. Transition issues for notified changes need to be handled between suppliers and customers.

7 Application of STC limits 7.1 Non-provisional STC limits are used to determine whether product is OOC. These limits as quantified in § 8 have three possible applications. 7.1.1 Reporting Only: Supplier reports out of control STC limit product. Such out of control product continues to be shipped and accepted as long as it meets the existing specification for that product grade. 7.1.2 Negotiated Product: Supplier reports out of STC limit product. Such product is not shipped unless an exception for the particular type of STC limit violation has already been provided by agreement with the product purchaser or the product purchaser has agreed upon the final disposition of such product. 7.1.3 Statistical Specification: The STC limits are treated as product specifications. Out of specification product is not shipped. 7.2 Provisional STC limits as quantified in § 8 also have all the possible applications as non-provisional STC limits (7.1). Due to the less stable nature of provisional STC limits, suppliers and producers should consider the possibility of applying provisional limits less aggressively than non-provisional limits. 7.3 Product properties for which STC limits are applied are established by agreement of supplier and producer. In this regard STC limits may apply to all specified product properties, a smaller list of critical parameters, or simply a list of negotiated parameters. Note: The number of product properties impacts the calculation of the limits (ref: 8.1, A1-3.1 – A1-3.4 ). 7.4 Grace period for STC limits modified upon annual review: When an STC limit changes for a material that met the STC limits in force at the time of production but now no longer does, this material can be shipped as long as the supplier is using a FIFO (first in first out) shipping strategy or as negotiated. 7.5 Relationship to existing product specifications: If material meets the STC limit but fails the nominal specification, then the preexisting rules for handling material that does not meet the nominal specification apply. If material meets the nominal specification but fails the STC limit, then the disposition rules defined in ¶ 7.1 apply. If both the nominal and STC limit are not met, then the more stringent of the rules applies. 7.6 No additional statistical requirements: No additional statistical requirements are part of the STC methodology. This includes additional statistical testing methodologies targeted at detecting shifts and trends, such as the Western Electric rules. It also includes process capability indices such as Cp or Cpk; these measures are inherently incompatible with STC.

8 Procedure This section provides a detailed technical description of the STC methodology. Enough information is provided for the user to do the requisite calculations by hand or in a spreadsheet. In some places the calculations are complex. Software will be provided to do all the calculations and apply the necessary logic at all the decision points. The user is strongly encouraged to use the software. 8.1 Quantification of STC Limits — The calculation and application of the STC limits are broken into two broad categories. The first calculates control limits based on reference (or historical) data. Some of the calculated control limits are considered provisional and are so noted. A second set of calculations occurs whenever test data is available. The main function here is to determine which, if any, of the analytical results from the test data are out of control (OOC). Also, for the purpose of the yearly review new control limits based on the test data can be calculated and tested to determine if any are statistically significantly different from the previous control limits and determine if the difference is practically significant (8.2). The calculations that define the STC limit for each characteristic and the determination of OOC points is described below. Section 8.2 describes the procedures for determining statistical and practical differences between control limits calculated on the reference and test data. Section 8.3 works through a sample set of data in detail. 8.1.1 Calculation of STC Limits on Reference Data — The user needs to supply the following information in order to calculate the control limits:

1. Number of samples, nref (≥100). 2. Number of characteristics measured (p). This sets the Type I error for each individual control chart (αInd):




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

αInd = 1- 0.991/p, where a family Type I error of 1% is assumed. 3. The analytical results for all the reference data. 4. Whether an upper control limit, lower control limit are both are required for each characteristic. 5. The method detection limit (MDL) for each characteristic. If there is no censoring this information is not needed. 6. The string used to identify that a particular result is below the MDL (i.e. is censored). This can vary from characteristic to characteristic but within a characteristic must always be the same string. Some commonly used identifier strings are ND and <MDL where the actual value of the MDL is used (e.g. <1 when the MDL is 1). It is not recommended that the value of the MDL be used as the identifier string (i.e. <1 is OK but 1 should not be used).

The following variables are then calculated for each characteristic. 1. nj, the total number of non-missing values for characteristic j. Note that nj ≤ nref and that censored values are not considered missing values. 2. tcrit(α, nj-1), a t-value with nj-1 degrees of freedom. For one sided control limits α = αInd. For two sided control limits α = αInd/2. If tcrit calculates to be less than 3 then it is set at 3. 3. The skewness correction factor, a, is calculated as follows. This is a function of nj and p’ where: p’ = p for a one sided limit and p’ = 2p for a two sided limit. The a factor is then given by:

a= (p’/b0 )b1, (1)

where: b0 = g(1-exp(-f(njh)))

b1 = (e/nj)c + d

c = 1.18041 d = 0.246002 e = 9.804714 f = 0.024273 g = 4.151277 h = 0.478154 If nj < 30 or p’ > 120 values for a are extrapolated and should be considered invalid. The STC limits should not be used in these cases. 4. Dual value insertion. It is recommended that the data used to calculate control limits not be censored. This means that all measurement results are recorded as is, even if they are below the MDL. Censoring can occur later, as, for example, on a certificate of analysis. If for whatever reason the data is censored on the left by the MDL then proceed as follows. For each characteristic, when the first MDL designator string is encountered the value 0 is inserted. The second time the designator string is encounter the value of the MDL is inserted. These values are alternately inserted as subsequent censored values are encountered.

5. jX , Sj and k3j, where jX is the average, Sj the standard deviation and k3j the sample skewness coefficient for characteristic j in the reference set. These are calculated after the dual value insertion has occurred. For upper




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

control limits if k3 is less than 0 then k3 is set to 0. For lower control limits if k3 is greater than 0 then k3 is set to 0. These statistics are defined below.

.)S)(n(n

)X(xnk

n

)X(xS

n

xX

jjj

jn

ijijj

j

j

jn

i

jij

j

j

jn

iij

j

31

3

3

1

2

1

21

1

−−

−=

−

−=

=

∑

∑

∑

=

=

=

(2)

Note that the subscript i ranges over the number of non missing samples for characteristic j. Also note that the true sample standard deviation is used. Do not use one that is inferred from a moving range. 6. The STC control limits are then given by:

jjjjcritjj

jjjjcritjj

nSakntXLCL

nSakntXUCL

/11))1,((

/11))1,((

3

3

++−−+=

++−+=

α

α (3)

For one sided control limits α = αInd. For two sided control limits α = αInd/2.

Some of the control limits are considered provisional. This occurs when the number of uncensored points is less than 10 and less than the critical value of the Tukey test. This will be discussed in detail in ¶ 8.2.

8.1.2 Calculations on Test Data—when test data is available the STC limits calculated on the reference data are applied to determine which if any values are OOC. A value from sample i and characteristic j, xij, is OOC if

xi,j > UCLj or xi,j < LCLj. (4)

The test data can be used to calculate new control limits using the same procedure in ¶ 8.1.1. The new control limits can be tested to see if they are statistically or practically different from the older limits from the reference data. How that is done is the subject of ¶ 8.2. 8.2 Annual Review Process for STC Limits—Once a supplier’s STC limits have been in effect for a year, and upon each subsequent year, the test data is used to calculate an alternate set of STC limits. These are then tested against the current STC limits to determine if any of the limits need to be changed. There are two sets of tests. One set tests for statistically significant differences between the current and the alternate limits. The second tests for significant practical differences between the two limits. A current limit is changed to the alternate limit only if both the statistical and the practical tests indicate a significant difference. This procedure is followed for each characteristic individually. Section 8.2.1 describes the statistical tests. Section 8.2.2 describes the test for a difference that is of practical significance. Also, in ¶ 8.2.1.2 the conditions for a STC limit to be provisional are precisely defined. 8.2.1 Tests for Statistical Significance—Two nonparametric (NP) tests are used for this purpose: the 90’th percentile test and Tukey’s Quick test (or simply Tukey’s test). The 90’th percentile test is more powerful than Tukey’s test. The later, however, has a wider range of applicability. The logic is to first apply the 90’th percentile test. If this is inconclusive as per ¶ 8.2.1.3 then apply Tukey’s test. The 90’th percentile test is described in ¶ 8.2.1.1. Tukey’s is described in ¶ 8.2.1.2. The logic that applies to determining which test to use is described in ¶ 8.2.1.3. 8.2.1.1 90’th Percentile Test—If an upper control limit is required the first step is to determine the 90’th percentile values for the reference and test sets. These are determined as follows (for the j’th characteristic).




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Sort the data from lowest to highest. Let pt90 = 0.9 * (nj + 1) Let ipt90 = the integer part of pt90 Let fpt90 = the fractional part of pt90 Let ipt91 = ipt90 +1 Let J90 be the value of characteristic J with rank ip90 Let J91 be the value of characteristic J with rank ip91 Then, the 90’th percentile, J90%, is given by:

J90% = J90 + (J91 – J90)* fpt90 (5) If a lower control limit is required then the 10’th percentile from the reference and test sets must also be determined. This is shown below. Sort the data from lowest to highest. Let pt10 = 0.1 * (nj + 1) Let ipt10 = the integer part of pt10 Let fpt10 = the fractional part of pt10 Let ipt11 = ipt10 + 1 Let J10 be the value of characteristic J with rank ip10 Let J11 be the value of characteristic J with rank ip11 Then, the 10’th percentile, J10%, is given by:

J10% = J10 + (J11 – J10)* fpt10 (6) A 90’th percentile value is said to be clean if there exists at least one non-censored value, xh, such that J90% ≥ xh. Otherwise, the calculated value for J90% will be the MDL. In this case the MDL is an upper bound on the actual J90%. Likewise, a 10’th percentile value is clean if there exists at least one non-censored value, xk such that J10%≥ xk . Otherwise, the MDL is an upper bound on J10%. The test is performed as follows. For an upper control limit determine which set has the lowest 90’th percentile. Count the number of values in the other set that are greater than the lower 90’th percentile. If the count is equal to or greater than a critical value then the two STC limits are deemed to be statistically significantly different. For a lower control limit determine the set that has the largest 10’th percentile. Count the number of values in the other set that are lower than the larger 10’th percentile. If the count is equal to or greater than a critical value then the two STC limits are deemed to be statistically significantly different. For upper control limits the critical values are determined as follows for the case where the 90’th percentile for the test set is greater than the reference. If the situation is reversed then interchange ntest and nref. Let n3 = ntest –1 Let S = (0.9*0.1/nref)1/2 Let p3 = 0.1 + 2.326*S Next, find the value of i starting at i=0 such that the cumulative binomial distribution, Bn(i, n3, p3), first has a value greater than 0.943. The cumulative binomial distribution for i from 0 to r is given by:

∑=

−−−

=r

i

ini ppini

npniBn0

3)31(3)!3!*(

!3)3,3,( , (7)

where x! = 1*2*…*(x-1)*x and 0! = 1




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

The critical value is then i+1. Some sample critical values for the 90’th percentile test for the case where nref= ntest = n are shown below.

n critical value 100 24 110 25 200 39 400 66

The 90’th percentile test as defined above has an individual Type I error of at most 1%. For a lower control limit if the 10’th percentile for the test is greater than the reference then Let n3 = nref –1 Let S = (0.9*0.1/ntest)1/2 Let p3 = 0.1 + 2.326*S . If the situation is reversed then interchange ntest and nref. The rest of the calculations to determine the critical values is the same as for the upper control limit case. 8.2.1.2 Tukey’s Test—For an upper control limit identify the maximum values in both the reference and test sets. If at least one value in a set is uncensored then the maximum is said to be clean. Otherwise, the maximum is calculated to be the MDL, which is really an upper bound on the maximum. Tukey’s test is not applied to lower control limits. This is because if Tukey’s can be applied then so can the 90’th percentile test. Since the latter is the more powerful of the two only use the 90’th percentile test for lower control limits. 8.2.1.2.1 The test is run as follows. Determine which set has the smallest maximum. Then, count the number of points in the other set that are greater than that value. If it is greater than or equal to a critical value then the two STC limits are deemed to be statistically significantly different. 8.2.1.2.2 The critical values for the Tukey test are calculated for characteristic j as follows when the maximum for the reference set greater than that of the test set. Let critp = 0.99*nref/(nref + ntest) Iterate through the following loop until p3 is greater than pcrit. p1=1 p3=0 i = -1 do until p3>pcrit

i=i+1 p1 = p1*(nref-i)/(nref + ntest –i)

p3 = p3 + p1*ntest/( nref + ntest –i-1) loop Increment i by 1 to find the smallest i such that p3>critp. Then the critical value is i+2. If the maximum of the test set is greater than the reference set then switch nref and ntest in the above calculations. 8.2.1.2.3 The following procedure shows how provisional limits are handled at the yearly review. Recall that provisional limits are always changed at this time. First, some terms are defined for each property: Test set = latest year’s worth of data. CTest = number of uncensored values in the test set. CRef = number of uncensored values in the reference set. TCV = Tukey’s critical value for the Test and Reference sets.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

For the first year (no prior reference set exists): 1. If CTest<10 then the STC limits are provisional and the test set becomes the reference set. 2. If CTest ≥ 10 then the STC limits are non-provisional and the test set becomes the reference set.

After the first year if the prior year’s STC was provisional then proceed as follows at the annual review. 1. If CTest ≥ 10 and ≥ TCV then the STC limits based on the test data are used and the test data becomes the

reference data. 2. If CTest < 10 or < TCV then the new reference set becomes the old reference set with the test data set added

on. 3. If CRef (new) ≥ 10 then the STC limits based on the new reference set are used and are non-provisional. 4. If CRef (new) < 10 then the STC limits based on the new reference set are used and are provisional.

8.2.1.3 The procedure for applying the NP tests 8.2.1.3.1 For an upper control limit proceed as follows:

1. If both sets have clean values for the 90’th percentiles then apply the 90’th percentile test. If the count is equal to or greater than the critical value then the two sets are deemed to be statistically significantly different. In that case proceed to the practical significance test. Otherwise the reference control limits are deemed to be unchanged.

2. If only one set has a clean value for the 90’th percentile then apply the 90’th percentile test. If the count is equal to or greater than the critical value then the two sets are deemed to be statistically significantly different. In that case proceed to the practical significance test. Otherwise the test is inconclusive; proceed to step 3.

3. If both sets have clean values for the maximums then apply Tukey’s test. If the count is equal to or greater than the critical value then the two sets are deemed to be statistically significantly different. In that case proceed to the practical significance test. Otherwise the reference control limits are deemed to be unchanged.

4. If only one set has a clean value for the maximum then apply the Tukey test. If the count is equal to or greater than the critical value then the two sets are deemed to be statistically significantly different. In that case proceed to the practical significance test. Otherwise the test is inconclusive and the reference control limits are deemed to be unchanged.

8.2.1.3.2 For a lower control limit proceed as follows: 1. If at least one of the sets has a clean value for the 10’th percentile then apply the 90’th percentile test.

If the count is equal to or greater than the critical value then the two sets are deemed to be statistically significantly different. In that case proceed to the test for practical significance. Otherwise the reference control limits are deemed to be unchanged.

2. If neither set has a clean value for the 10’th percentile then the 90’th percentile test cannot be applied. In this case the STC limits are deemed to be unchanged.

8.2.2 Test for Practically Significant Differences—In order for control limits to be changed at the yearly review they must be both statistically and practically significantly different. This section explains how to test for a practically significant difference. The critical values are calculated using the reference set data. For an upper control limit the critical value, Wu, is

Wu = (tcrit +ak3)S(1+1/nref)1/2 (8)

For a lower control limit the critical value, Wl, is

Wl = abs(-tcrit +ak3)S(1+1/nref)1/2. (9) For an upper control limit the difference between the reference and test values are deemed to be practically significantly different if

Abs(UCLRef – UCLTest) >Wu/3. (10)




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

For a lower control limit the difference between the reference and test values are deemed to be practically significantly different if

Abs(LCLRef – LCLTest) >Wl/3. (11) After the yearly review the reference set is updated. The new reference set consists of the values for the parameters from the previous reference set that were not changed plus the values from the test set for the parameters whose STC limits did change. 8.3 Example—A complete set of calculations will be run through on a reference and test set each containing analytical results from 50 samples. It is recommended that the minimum sample size be 100. However, in order to keep the manipulations of the data tractable a smaller sample size will be used. There are 5 characteristics measured per sample. This sets the individual Type I error for each characteristic at: αInd = 1-0.991/5 = 0.002008. The MDL’s, MDL designator strings and which control limits are to be calculated are shown below. A B C D E MDL 0.01 0.01 1 9 1 MDL designator string ND ND ND ND ND UCL ?(Y for yes) Y Y Y Y Y LCL ?(Y for yes) Y

The reference data is shown below.

A B C D E

3.26 80.45 ND ND ND 4.5 ND ND ND

2.02 78.96 ND 9.31 ND 2.98 80.04 3.57 ND ND 4.22 ND ND ND 5.38 3.91 9.11 ND 0.75 80.41 3.06 ND ND 0.86 79.17 ND ND ND 0.09 80.52 1.71 ND ND 0.71 79.29 ND ND ND 1.42 80.7 4.09 ND ND 1.45 1.92 ND ND 0.75 80.5 1.87 ND ND 3.56 78.37 ND ND ND 0.9 78.23 ND ND ND

1.77 79.63 3.73 ND ND 0.14 80.46 2.65 ND ND 0.48 78.32 2.46 ND ND 0.58 81.71 ND ND ND 1.89 80.34 ND ND ND 1.06 80.07 ND ND ND




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

0.82 79.02 3.47 ND ND 0.95 80.06 2.87 ND ND 0.21 79.87 ND ND ND 3.14 80.49 ND ND ND 0.58 79.6 ND ND ND 2.13 80.12 ND ND ND 4.62 81.8 ND ND ND 1.85 79.89 2.21 ND ND 2.63 80.98 ND ND ND 1.54 79.74 ND ND ND 6.76 79.47 2.28 ND ND 1.37 79.43 ND ND ND 1.43 79.78 3.4 ND ND 0.87 81.22 4.86 13.5 ND 0.78 78.23 ND ND ND 1.56 80.97 2.56 ND ND 0.68 79.42 ND ND ND 3.47 79.36 ND ND ND 2.35 79.73 ND ND ND 1.22 79 7.14 ND ND 1.61 80.61 1.33 ND ND 1.76 80.93 ND 9.54 ND 2.46 79.77 2.68 ND ND 0.79 80.53 3.44 ND ND 2.18 79.26 4.56 ND ND 1.4 79.43 ND ND ND

2.08 78.8 4.02 ND ND 1.12 79.83 1.52 ND ND 2.62 78.9 2.77 ND ND

For characteristics A, C, D and E there is no missing data. For characteristic B there are 4 missing data points. Thus nA, nC, nD, and nE = 50 and nB = 46. Characteristics A, C, D and E require only upper control limits. For these the critical t value is tcrit = t(αInd, nj-1) = t(0.002008, 49) = 3.0192. Characteristic B requires two sided limits. For this case the critical t value is tcritB = t(αInd/2, nB) = t(0.001004, 45) = 3.2801. Note that if the TINV function in Excel is used to calculate tcrit it requires the user to supply a probability and degrees of freedom. It always assumes a two-tailed application. To make Excel provide the correct value for tcrit then enter twice the value of αInd in the Probability input box for a once sided limit (upper or lower STC) and αInd for a two-sided limit (upper and lower STC). The a factor for characteristics A, C, D and E is given by: p’=p=5 b0 = 4.151277(1-exp(-0.024273(500.478154))) = 0.60521




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

b1 = (9.804714/50)1.18041 + 0.246002 = 0.39216 a = (5/0.60521)0.39216 = 2.2889 For two sided control limits the a factor is calculated the same way except that p’ is p’ = 2p Thus, for characteristic B with two sided limits and nB = 46: p’ = 2*5 = 10 b0 = 4.151277(1-exp(-0.024273(460.478154))) = 0.583303 b1 = (9.804714/46)1.18041 + 0.246002 = 0.407276 a = (10/0.583303)0.407276 = 3.182 Before the reference sample average ( jX ), standard deviation (Sj) and coefficient of skewness (k3j) can be calculated the dual value insertion must occur for each censored value. For each characteristic, the first time a censored value is encountered the value 0 is inserted. The second time the value of the MDL is inserted. Alternate between 0 and the MDL for succeeding censored values. The reference data set after dual value insertion is shown below.

A B C D E 3.26 80.45 0 0 0 4.5 1 9 1

2.02 78.96 0 9.31 0 2.98 80.04 3.57 0 1 4.22 1 9 0 5.38 3.91 9.11 1 0.75 80.41 3.06 0 0 0.86 79.17 0 9 1 0.09 80.52 1.71 0 0 0.71 79.29 1 9 1 1.42 80.7 4.09 0 0 1.45 1.92 9 1 0.75 80.5 1.87 0 0 3.56 78.37 0 9 1 0.9 78.23 1 0 0

1.77 79.63 3.73 9 1 0.14 80.46 2.65 0 0 0.48 78.32 2.46 9 1 0.58 81.71 0 0 0 1.89 80.34 1 9 1 1.06 80.07 0 0 0 0.82 79.02 3.47 9 1 0.95 80.06 2.87 0 0 0.21 79.87 1 9 1




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

3.14 80.49 0 0 0 0.58 79.6 1 9 1 2.13 80.12 0 0 0 4.62 81.8 1 9 1 1.85 79.89 2.21 0 0 2.63 80.98 0 9 1 1.54 79.74 1 0 0 6.76 79.47 2.28 9 1 1.37 79.43 0 0 0 1.43 79.78 3.4 9 1 0.87 81.22 4.86 13.5 0 0.78 78.23 1 0 1 1.56 80.97 2.56 9 0 0.68 79.42 0 0 1 3.47 79.36 1 9 0 2.35 79.73 0 0 1 1.22 79 7.14 9 0 1.61 80.61 1.33 0 1 1.76 80.93 1 9.54 0 2.46 79.77 2.68 9 1 0.79 80.53 3.44 0 0 2.18 79.26 4.56 9 1 1.4 79.43 0 0 0

2.08 78.8 4.02 9 1 1.12 79.83 1.52 0 0 2.62 78.9 2.77 9 1

Values for jX , Sj and k3j can be calculated using Excel or other spreadsheet standard functions. If these are not

available use the equations that define jX , Sj and k3j in ¶ 8.1. A summary of these statistics is shown below along with the critical t values and the a factors. A B C D E Sample Size 50 46 50 50 50 critical t value 3.019 3.280 3.019 3.019 3.019 skewness equation factor 2.289 3.182 2.289 2.289 2.289 Sample Average 1.875 79.857 1.802 4.969 0.500 Sample Standard Deviation 1.415 0.863 1.640 4.676 0.505 Sample Skewness Coefficient 1.421 0.089 0.887 -0.093 0.000 Calculations for the STC control limits for characteristics A and B are shown below. UCLA = 1.875 + (3.019 + 2.289*1.421)*1.415*(1 + 1/50)1/2 = 10.84 UCLB = 79.857 + (3.280 + 3.182*0.089)*0.863*(1+1/46)1/2 = 82.97 LCLB = 79.857 + (-3.280 + 3.182*0)*0.863(1+1/46)1/2 = 76.99




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

The complete set of values for the STC limits for all five characteristics are shown below. A B C D E UCL 10.84 82.97 10.16 19.23 2.04 LCL 76.99 For this example none of the reference values were OOC. If a value was flagged as OOC then it can be removed from the data set if and only if a special cause was identified that was responsible for the OOC value and if that cause has been removed. Otherwise OOC points should remain in the reference data set. The test data set is shown below.

A B C D E 5.38 78.63 1.67 ND ND 0.62 77.48 ND ND 6.1 0.17 78.17 ND 11.72 2.2 5.13 77.97 2.86 ND ND 2.12 78.22 ND ND ND 4.5 79.7 1.24 ND ND

5.24 81.32 ND 13.43 ND 5.59 81.52 ND 10.94 ND 2.32 79.36 3.34 ND ND 1.62 80.8 ND ND ND 2.02 80.8 1.16 9.86 ND 0.86 78.39 1.04 ND ND 2.04 78.47 ND ND ND 0.24 78.35 2.31 10.71 4.7 1.37 77.51 1.71 ND ND 2.98 79.33 1.01 ND ND 0.55 78.59 1.42 ND 2.4 0.79 78.86 ND ND ND 1.49 79.45 ND ND ND 1.3 77.81 ND 9.38 ND

4.22 79.28 1.04 ND ND 3.77 78.91 1.19 12.78 ND 1.39 80.63 ND ND ND

1 79.58 ND ND ND 4.67 78.18 ND 9.67 ND 3.26 78.83 1.03 ND ND 3.66 80.35 ND ND ND 0.64 79.89 ND ND ND 0.53 78.13 1.15 ND 5.5 4.49 77.25 ND ND ND 0.75 78.56 ND ND ND 2.66 77.71 1.18 ND ND 0.93 80.74 ND ND ND 2.3 77.84 ND ND ND

0.41 77.83 ND ND 3.1 0.86 76.86 ND 13.36 ND




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

1.55 77.73 ND ND ND 1.87 79.49 ND ND ND 4.99 78.8 ND ND ND 0.58 78.11 ND ND 3.3 0.09 78.14 ND ND 1.5 0.71 78.05 ND ND ND 1.08 80.58 2.94 ND ND 2.64 80.4 1.83 ND ND 3.5 78.81 1.4 10.11 ND

0.71 78.1 2.15 15.05 ND 2.42 80 1.38 ND ND 1.09 78.47 1.46 12.91 ND 3.19 78.86 ND ND ND 2.22 79.18 1.4 9.05 ND

All of the values in bold are flagged as OOC based on the STC limits calculated from the reference set. Assuming that each sample came from a unique lot then 8 of the 50 lots would contain OOC values and would not be shippable under the strictest interpretation of the STC limits. STC limits can be calculated for the test set exactly the same way as for the reference set. This is necessary for the yearly review to determine if the new STC limits are applicable. First the dual value insertion has to be performed. The results of this are shown below. A B C D E

5.38 78.63 1.67 0 00.62 77.48 0 9 6.10.17 78.17 1 11.72 2.25.13 77.97 2.86 0 12.12 78.22 0 9 0

4.5 79.7 1.24 0 15.24 81.32 1 13.43 05.59 81.52 0 10.94 12.32 79.36 3.34 9 01.62 80.8 1 0 12.02 80.8 1.16 9.86 00.86 78.39 1.04 9 12.04 78.47 0 0 00.24 78.35 2.31 10.71 4.71.37 77.51 1.71 9 12.98 79.33 1.01 0 00.55 78.59 1.42 9 2.40.79 78.86 1 0 11.49 79.45 0 9 0

1.3 77.81 1 9.38 14.22 79.28 1.04 0 03.77 78.91 1.19 12.78 11.39 80.63 0 9 0

1 79.58 1 0 1




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

4.67 78.18 0 9.67 03.26 78.83 1.03 9 13.66 80.35 1 0 00.64 79.89 0 9 10.53 78.13 1.15 0 5.54.49 77.25 1 9 00.75 78.56 0 0 12.66 77.71 1.18 9 00.93 80.74 1 0 1

2.3 77.84 0 9 00.41 77.83 1 0 3.10.86 76.86 0 13.36 11.55 77.73 1 9 01.87 79.49 0 0 14.99 78.8 1 9 00.58 78.11 0 0 3.30.09 78.14 1 9 1.50.71 78.05 0 0 11.08 80.58 2.94 9 02.64 80.4 1.83 0 1

3.5 78.81 1.4 10.11 00.71 78.1 2.15 15.05 12.42 80 1.38 9 01.09 78.47 1.46 12.91 13.19 78.86 1 0 02.22 79.18 1.4 9.05 1

tcrit, the a factor, jX , Sj, k3j, UCL and LCL are all calculated the same as for the reference set. A table that summarizes the results is shown below. A B C D E Sample Size 50 50 50 50 50 critical t value 3.019 3.264 3.019 3.019 3.019 skewness equation factor 2.289 3.004 2.289 2.289 2.289 Sample Average 2.170 78.920 0.998 6.219 0.996 Sample Standard Deviation 1.612 1.124 0.813 5.099 1.387 Sample Skewness Coefficient 0.702 0.568 0.716 -0.258 2.234 UCL 9.70 84.56 4.82 21.77 12.39 LCL 75.22 Next, two nonparametric (NP) tests are applied to determine statistically significant differences between the STC limits from the reference and test sets. For upper limits the 90’th percentile and the maximum from each set (for each characteristic) have to be determined. Determining the maximums is straightforward. The determination of the 90’th percentile for characteristic A in the reference set is illustrated below. Sort the data from lowest to highest. Let pt90 = 0.9 * (nA + 1) = 0.9*(50 + 1) = 45.90 Let ipt90 = the integer part of pt90 = 45 Let fpt90 = the fractional part of pt90 = 0.90




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Let ipt91 = ipt90 + 1 = 46 Let A90 be the value of characteristic A with rank ip90 = A45 = 3.56 Let A91 be the value of characteristic A with rank ip91 = A46 = 4.22 Then, the 90’th percentile, A90%, is given by: A90% = A90 + (A91 – A90)* fpt90 = 3.56 + (4.22 – 3.56)*0.90 = 4.15 Note that if the censoring is high enough then the maximum and/or the 90’th percentile can be the value of the MDL. For lower control limits separate calculations have to be done for the 10’th percentile and minimum for each set. The calculations for the 10’th percentile are essentially the same as the 90’th percentile case and are illustrated below for characteristic B in the reference set. Sort the data from lowest to highest. Let pt10 = 0.1 * (nB + 1) = 0.1*(46 + 1) = 4.7 Let ipt10 = the integer part of pt10 = 4 Let fpt10 = the fractional part of pt10 = 0.70 Let ipt11 = ipt10 + 1 = 5 Let B10 be the value of characteristic B with rank ip10 = B4 = 78.37 Let B11 be the value of characteristic B with rank ip11 = B5 = 78.80 Then, the 10’th percentile, B10%, is given by: B10% = B10 + (B11 – B10)* fpt10 = 78.37 + (78.80 – 78.37)*0.70 = 78.67 Note that with even moderate censoring the 10’th percentile can be buried in the censored values. In that case the MDL is an upper bound of the 10’th percentile. Values for the required percentiles, maximums and minimums are shown below. A B C D E Ref n 50 46 50 50 50 Test n 50 50 50 50 50 Ref 90%tile 4.154 80.973 4.009 9 1 Test 90%tile 4.958 80.729 2.118 12.674 3.03 Ref Max 6.76 81.8 7.14 13.5 1 Test Max 5.59 81.52 3.34 15.05 6.1 Ref 10%tile 78.67 Test 10%tile 77.71 Ref Min 78.23 Test Min 76.86 Next, critical values for the 90’th percentile and the Tukey tests need to be determined. These are the same for both upper and lower control limits. This will be worked out in detail for characteristics A and B. For characteristic A note that the 90’th percentile for the test set is greater than the reference set. Let: n3 = ntest –1 = 50 – 1 = 49 S = (0.9*0.1/nref)1/2 = (0.9*0.1/50)1/2 = 0.042426 p3 = 0.1 + 2.326*S = 0.1 + 2.326*.0424 = 0.1986 Next, starting at i=0 find the value of i such that the cumulative binomial distribution function first becomes greater than 0.943. The critical value is then i + 1. It is recommended that this be done in a spreadsheet that supports the binomial function. The binomial function in Excel was used in this example. The arguments for the function are: Number = cell location for value of i




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Trials = n3 = 49 Probability = p3=0.1986 Cumulative = true

i Bn(i,49,0.19,true) 0 0.000 1 0.000 2 0.002 3 0.007 4 0.023 5 0.057 6 0.120 7 0.216 8 0.341 9 0.482

10 0.621 11 0.744 12 0.840 13 0.908 14 0.951 => i=14 and critical value = 1515 0.976

If software is not available to do this calculation then it can be done by hand using the binomial density function given by:

ini ppini

npniBn −−−

= 3)31(3)!3!*(

!3)3,3,( , (12)

where x! = 1*2*…*(x-1)*x and 0! = 1. Thus for, say, i=10 and n3 and p3 defined above we get:

104910 )1986.01(1986.0)!1049!*(10

!49)3,3,( −−−

=pniBn = 0.140

The cumulative distribution value for i from 0 to r is:

∑=

−−−

=r

i

ini ppini

npniBn0

3)31(3)!3!*(

!3)3,3,( (13)

For characteristic B start with the critical value for the upper control limit. The 90’th percentile for the reference data is greater than for the test data. For this case: n3 = nref –1 = 46 – 1 = 45 S = (0.9*0.1/ntest)1/2 = (0.9*0.1/50)1/2 = 0. 042426 p3 = 0.1 + 2.326*S = 0.1 + 2.326*.0447 = 0.1986 Values for the cumulative binomial distribution function are shown below.

i Bn(i,45,0.1986,true) 0 0.000




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

1 0.001 2 0.003 3 0.014 4 0.040 5 0.094 6 0.183 7 0.306 8 0.450 9 0.597

10 0.729 11 0.832 12 0.905 13 0.951 => critical value=14 14 0.976 15 0.990

Next, calculate the critical value for the lower control limit for characteristic B. When both the reference and test data have about the same number of observations the critical values for the upper and lower control limits will be the same. If the sample sizes are different then the critical values will probably be different. This is the case for characteristic B. Note that the 10’th percentile for the reference is greater than the 10% percentile for the test. For this case: n3 = ntest –1 = 50 – 1 = 49 S = (0.9*0.1/nref)1/2 = (0.9*0.1/46)1/2 = 0. 04423 p3 = 0.1 + 2.326*S = 0.1 + 2.326*.04423 = 0.2029 Values for the cumulative binomial distribution function are shown below.

i Bn(i,49,0.2029,true) 0 0.000 1 0.000 2 0.001 3 0.006 4 0.019 5 0.050 6 0.106 7 0.195 8 0.314 9 0.452

10 0.592 11 0.718 12 0.820 13 0.894 14 0.942 15 0.971 =>critical value=16

Next the critical values for the Tukey test are calculated. This will be done for characteristics A and D. For characteristic A the maximum of the reference set is greater than the test. For this case pcrit = 0.99*nref/(nref+ntest) = 0.99*50/(50+50) = 0.495.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Iterate through the following loop until p3 is greater than pcrit. p1=1 p3=0 i = -1 do until p3>pcrit

i=i+1 p1 = p1*(nref-i)/(nref + ntest –i)

p3 = p3 + p1*ntest/( nref + ntest –i-1) loop The Tukey critical value is then i + 2. The iteration for characteristic A is shown below. This was done in Excel. Recall that pcrit = 0.495.

i p1 p3 0 0.5 0.252525 1 0.247475 0.378788 2 0.121212 0.441268 3 0.058732 0.471858 4 0.028142 0.486669 5 0.013331 0.49376 6 0.00624 0.497115 =>i=6 and critical value =8 7 0.002885 0.498683 8 0.001317 0.499406 9 0.000593 0.499736

For characteristic D the maximum of the test set is larger than the reference. For this case the role of nref and ntest switch places with each other for all the above calculations. Since nref = ntest = 50 the same result is obtained for the Tukey critical value, 8. Tukey’s test is not applied to lower control limits. If Tukey’s test can be applied then so can the 90’th percentile test. Since the latter is has more power the 90’th percentile test is used. A table of all the critical values for all the characteristics for both NP tests is shown below. A B C D E CV for 90%tile Test for Upper CL 15 14 15 15 15 CV for 90%tile Test for Lower CL 15 16 15 15 15 CV for Tukey Test 8 7 8 8 8 The critical values can now be applied to each characteristic to determine if there is sufficient statistical evidence to conclude that the STC limits have changed. This will be done for each characteristic. The NP tests do not directly test the equivalency of STC limits, but test for statistically significant differences in the tail region nearest the control limit between the reference and test distributions. Such a difference in distribution, when statistically significant, indicates the potential need for updated STC limits. In this testing context, the null hypothesis is that the STC limits are the same and the alternative hypothesis is that the reference and test STC limits differ. Characteristic A For both sets the 90’th percentiles are greater than the MDL so we can apply the 90’th percentile test. Since the reference set has the smaller value for the 90’th percentile we count how many values in the test set are greater than 4.154. There are 9 such values. Since 9<15 we conclude that we cannot reject the hypothesis that the two sets of STC limits are the same. Ergo, we do not change them.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Characteristic B First, the upper STC limit. For both sets the 90’th percentiles are greater than the MDL so we can apply the 90’th percentile test. Since the test set has the smaller value for the 90’th percentile we count how many values in the reference set are greater than 80.729. There are 6 such values. Since 6<14 we conclude that we cannot reject the hypothesis that the two sets of upper STC limits are the same. For the lower limits the 10’th percentile of the reference set, 78.67, is greater than that of the test set, 77.12. Thus, the number of values in the test set less than 78.67 are counted and found to be 25. This is greater than the critical value of 16 and thus we reject the hypothesis that the lower STC limits are the same. Characteristic C The situation here is the same as with the upper STC limit in characteristic B. We count how many values in the reference set are greater than 2.118. There are 20 such values. Since 20>15 we conclude that the STC limits are statistically different. Characteristic D Since the reference set had only 4 points that were not censored the control limits for characteristic D are considered provisional because this is less than 10. The test set had 13 uncensored points. Thus the STC limits calculated for the test data are no longer provisional and should be used in place of the previous limits. It is instructional to work up the numbers for the NP tests as if the reference set was not provisional. The 90’th percentile for the test set can be determined from uncensored data. However, all we can say about the 90’th percentile for the reference set is that it is less than or equal to the MDL. If there were 15 or more values in the test set that were greater than 9 then we would conclude based on the 90’th percentile test that the STC limits were statistically different. However, there were only 13 such points. The 90’th percentile test is thus judged to be inconclusive. Tukey’s test is tried next. In this case the maximum in the reference set can be determined from uncensored data. Since the maximum in the test set is greater than the maximum in the reference set the number of values in the test set greater than 13.5 are counted. There was 1, which is less than the critical value of 8. Therefore, we conclude that we do not have evidence to reject the hypothesis that the STC limits are the same. Characteristic E The reference set had no uncensored points and the test set had 8. In both cases the control limits would be considered provisional. Even after combining the two sets the number of uncensored points is still less than 10. The STC limits calculated from the combined reference and test data would be used and would be considered provisional. If the third year’s data had 10 or more uncensored points then the STC limits from that year would be used for the subsequent year and would be considered non provisional. If the third year had less than 10 uncensored points but the combination of years two and three had 10 or more uncensored points then the STC limits would be calculated from the combined data sets and would be considered non provisional. Again, for instructional purposes only, if we ignored the provisional status of the control limits we would note that the test set had 8 values greater than the MDL of the reference set and thus conclude based on the Tukey test that the STC limits were statistically different. The final set of calculations is to determine if the observed differences between the reference and test sets are of practical significance. The critical values are calculated from the reference data:

Wu = (tcrit +ak3)S(1+1/nref)1/2, for an upper control limit and (14)

Wl = abs(-tcrit +ak3)S(1+1/nref)1/2, for a lower control limit. (15) The difference between the STC limits for the reference and test data sets are deemed to be practically significant if

Abs(UCLref – UCLtest) > Wu/3, or (16)




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

Abs(LCLref – LCLtest) > Wl/3. (17) The calculations are illustrated for characteristics A and B. For characteristic A Wu = ((3.019 + 2.289*1.421)*1.415*(1+1/50)1/2)/3 = 2.99 Abs(UCLref – UCLtest) = Abs(10.84 – 9.70) = 1.14 < 2.99. Thus the observed difference is of no practical significance. Characteristic B has both an upper and a lower control limit. Wu = ((3.280 +3.182*0.089)*0.863*(1 + 1/46)1/2 )/3= 1.04 Wl = Abs(((-3.280 +3.182*0)*0.863*(1 + 1/46)1/2)/3) = 0.95 Abs(UCLref – UCLtest) = Abs(82.97-84.56) = 1.59 > 1.04 Abs(LCLref – LCLtest) = Abs(76.99 – 75.22) = 1.77 > 0.95 It is sufficient for just one of these inequalities to hold in order to deem the observed difference to of practical significance. For characteristic B the difference is of practical concern. Shown below is a summary of the tests for statistical and practical differences. A blank indicates that there was no significant difference. An “*” in the practical significance test row means that there was a practically significant difference. X>Y in the NP significance test row indicates that X was shown to be statistically significantly greater than Y. A B C D E Ref UCL 10.84 82.97 10.16 19.23 2.04 Test UCL 9.70 84.56 4.82 21.77 12.39 NP Stat Sig. Test Ref>Test Test>Ref Practical Sig. Test * * * Ref LCL 76.99 Test LCL 75.22 NP Stat Sig. Test Ref>Test Practical Sig. Test * Note:

1. The upper control limits and the lower control limits are tested separately. 2. In order to change the STC limit from that calculated by the reference set to that calculated by the test set

the observed difference has to be both statistically and practically significant. 3. In the case where there is both an upper and a lower control limit it is sufficient for either of them to be

statistically and practically different to cause the STC to change. Even if only one control limit changed both would revert to the new STC limits.

For this example characteristics B, C and E were both statistically and practically different. Thus, their STC limits would be changed at the yearly review. The STC limits for characteristics A and D would remain the same.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

APPENDIX 1 SEMI STC Methodology NOTICE: The material in this appendix is an official part of SEMI [insert designation, without publication date (month-year) code] and was approved by full letter ballot procedures on [dd-mm-yyyy]. This Appendix is intended to provide additional background information on issues related to the SEMI STC Methodology. More detailed information can be found in the Ship to Control Workshop which was provided at SEMICON West 2007. Use the following link (http://teams.semi.org/stds_chem) on the SEMI website to obtain a copy of this presentation.

A1-1 Commercial Issues: A limited discussion of some of the commercial issues that will be associated with the application of a STC methodology A1-1.1 Product Fragmentation: Product fragmentation occurs when there are multiple product grades originating from a single product production process. From a product management standpoint, as product grades become more fragmented, this implies a greater cost for managing the supply system. Product fragmentation can also negatively impact product availability and typically drives requirements for higher overall levels of product inventory as compared to materials with less product fragmentation. Keeping with SEMI terminology a supplier produces a product and a producer (often an IDM) consumes the product. A1-1.1.1 Prior to STC, product fragmentation was typically driven by producer product requirements or the interplay with producer product requirements, differential pricing of product grades and the capability of a supplier’s process. Different product grades result whenever a producer requires a non-standard product container, requires a non-standard specification (only one product property specification needs to differ for this to occur), requires a material quality level that can not be consistently met by the supplier’s process or requires any existing specialized data treatment that uses additional data evaluation to determine product suitability over and above the nominal product specification. A1-1.1.2 The motivation of suppliers in allowing existing product fragmentation typically originates from having to satisfy their customer’s requirements. It also can originate, to a more limited extent, when a supplier finds a market opportunity when the creation of one or two additional differential product grades with differential pricing more than offsets the costs of managing these additional grades. A1-1.1.3 With implementation of STC, when used for anything other than notification, it is likely, that initially, there will be increased product fragmentation. Even if a supplier chooses to offer only one STC product grade in a single container type; such a grade is likely to be initially offered as an alternative to the existing product grades. Eventually, if STC achieves enough success, such an STC grade may displace some of the current product grades. A1-1.1.4 Existing product fragmentation may not go away. If a unique container is still required by a producer, an STC version of this product grade would replace one unique grade with another. If a supplier must still meet a customer specific set of nominal product specifications as part of an STC product grade again this product grade would replace one unique grade with another. In this latter case, there is an opportunity for STC to reduce product fragmentation if the STC product grade’s nominal specifications can be changed to levels that are no longer customer specific. A1-1.1.5 There is potential for greatly increased future product fragmentation for any STC product grade that is purchased by multiple producers. STC limits are not rigid; they can change as part of the Annual Review Process. Unless suppliers and producers consistently accept all changes in STC limits required by the STC methodology and allow use of a consistent date for Annual Review process, there is potential for massive future product fragmentation and all its undesirable consequences. The focus in devising the STC methodology was to define the method and data requirements in a manner that leads to a unique result. Any future customer specific deviation(s) will immediately result in increased product fragmentation. A1-1.2 STC Limits Change Impact: STC limits have the characteristic that they are likely to change more rapidly than traditional nominal specifications have historically. STC product that requires anything more than notification will require substantially more change management. The Annual Review (4.14, 8.2) process’s timing provides a tradeoff in that it does not require too frequent changes in what can be immediately shipped or the statistical specifications versus the STC limits being locked into place ad infinitum. Section 7.4 of the Guide is intended to smooth any transition to new limits. A1-1.2.1 The Annual Review portion ¶ 8.2, of the Guide uses the philosophy that STC limits should only be changed on Annual Review when there is both a statistically significant difference between the test and reference data sets and this difference is also judged to be of practical significance (4.6, 4.7, and 4.9). In this respect, the SEMI STC method is biased to only change STC limits when there is strong evidence of meaningful change (8.2).




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

The exception to this are STC limits which are provisional (4.10, 5.7, 7.2, 8.2) and always change upon Annual Review. A1-1.2.2 Based on industry data on 34 products studied by the TF in developing the Guide, about 25% of STC parameters are likely to require modification upon Annual Review. Since the statistical tests in ¶ 8.2 were tuned to the α ≤ 1% level on a per test parameter basis, this discrepancy implies that typical industry processes have some real degree of time instability. A1-1.2.3 When STC limits change there is high potential for product fragmentation (A1-1.1 ) if the entire package of required changes is not accepted (6.3, 6.4, and 6.5). The driving psychology for potential trouble with STC limit changes will originate from two sources; suppliers wanting the looser STC limits and not the tighter STC limits and producers wanting the tighter STC limits and not the looser STC limits. The method does not equivocate on this matter (6.5); all data indicated required changes (8.2) are implemented without exception. In order to protect producer processes, section 7.5 maintains traditional nominal specification requirements for the product grade. In this manner, STC product can never be worse than the nominal product grade it originated from. A1-1.3 Impact of Alternative STC Methodologies A1-1.3.1 The short version of the impact of this is product fragmentation (A1-1.1 ). Any difference in methodology (data selection, statistical rules, time frames, or additional rules) results in a different specific product grade. Additionally the operational characteristics of what alternative STC methodologies actually accomplish with respect to typical electronics product data, other than that which is provided below, are not well known. To the knowledge of the TF there is no prior unified product level statistical methodology for control limit calculation that simultaneously deals with sample size, number of parameters for which limits are desired, skewed (non-normal) data distributions, and censored data handling. A1-1.3.2 These are the reasons why the SEMI methodology was developed. It handles a broad range of issues while providing a rigorous data definition and computational methodology. A1-1.3.3 The long version of this involves what the Statistical Methods Task Force, (Guide developers) called the calibration study. The calibration study involved the use of data from many suppliers and many products to evaluate the actual performance characteristics of both the incremental and final versions of the draft Guide as well as some simple alternative methodologies (two 3 sigma approaches, 99th percentile, and historical high). The incremental calibration studies performed on the evolving draft statistical methodology allowed identification of problem areas. These problem areas, once identified, indicated where the draft methodology needed further improvement. After each modification, the performance of the entire methodology was checked again. Through this iterative process the methodology evolved until it was found to provide sufficiently reliable performance. The final calibration study involved data from 34 products and 16 suppliers. Some of these results were reported in A1-1.2 and A1-3 . A1-1.3.4 The Statistical Methods Task Force attempted to err on the side of simplicity in this evolutionary process. Many of the earlier changes made to the draft methodology via the calibration process involved discovering that the earliest attempts at method simplicity exacted too high a price on method performance. The later, more incremental, changes to the draft methodology involved detecting and resolving more subtle, but still important, issues that were only detected through iterating the calibration process. A1-1.3.5 A major calibration sub-study examined four simple statistical approaches to defining STC limits. These were:

• Three sigma control limits using the sample std. dev. (s) as estimate of sigma • Three sigma control limits using the moving range estimate of sigma (2.66x R ) • Maximum (UCL=maximum in training set, LCL=minimum in training set) • 99’th Percentile (UCL=99’th percentile, LCL=1’st percentile)

The calibration study used two years of data from processes believed to be unchanged. The first year’s data was used to set the STC limits (reference data), the second to test the performance of the STC limits (test data). This study used data submitted by 11 suppliers from 18 products. Upper control limits were applied whenever there was an upper specification, lower control limits were applied whenever there was a lower specification and both were applied whenever there was both an upper and lower specification. The average percentage of product failing to meet the historical STC limits from the first year in the second year of data is provided as a percentage.

• 25.4% (Three sigma control limits using the sample std. dev. (s)) • 42.0% (Three sigma control limits using 2.66x R ) • 20.7% (UCL=Maximum, LCL=Minimum)




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

• 25.5% (UCL=99’th Percentile, LCL= 1’st percentile) A1-1.3.6 These results were judged to be unacceptable in terms of their performance in that they would institute a statistical cherry picking mechanism with their high rejection rates. Statistical theory would also indicate that these results are largely expected rather than being unusual in nature. The draft SEMI methodology provided an average 2.6% product rejection rate when processed with the identical product data. This result is close to the average 2.8% result reported in A1-1.4 which is based on data from a total of 34 products from 16 companies. A1-1.4 Likely Future STC Product Rejection Rates A1-1.4.1 The SEMI STC Methodology was tested with data from 34 products submitted by 16 companies. Typically two years of data were used from processes believed to be unchanged. The first year’s data was used to establish the limits, the second year’s data to evaluate performance to the limits. This resulted in an estimated future average STC product rejection rate of 2.8%. The SEMI Method was constructed in a manner such that for a perfectly in-control process, the future average failure rate will be about 1%. The difference between the 1% target and the 2.8% observed result with actual industry data is likely to be due to real industrial processes not being in perfect statistical control. This conclusion was verified by extensive simulation studies which showed the target 1% rate is achieved for mixtures of normal and non-normal distributed properties with varying degrees of data censoring, sample sizes and number of parameters. A1-1.4.2 What an individual company experiences with STC can vary from the average 2.8% product rejection rate that was observed in the calibration study. A given product may do better or worse than the average. A collection of truly in-control processes will average 1%. An OOC process, if sufficiently OOC, can reject all future product. The SEMI Methodology does not limit the future failure rate of an OOC process.

A1-2 Measurement Issues: A limited discussion of some of the measurement related issues that impact implementation of STC or any other statistical methodologies. A1-2.1 Censored Data and Method Detection Limits (MDLs) A1-2.1.1 Censored data reflects a reported value that conveys a data range rather that a single reported value. Censored data may be left censored, right censored, or interval censored in nature. For example, a censored measured result may be reported as <10, >500, or between 10 and 25. Typical electronics data includes all these possibilities; although left censored data is the dominant type in practice. Left censored data typically occurs when data is censored to a MDL. Measured or otherwise unobtainable results less than the MDL are typically reported as <MDL. Right censored data occurs when an instrument either can not measure a value above a certain upper limit or the testing protocol is allowed to time out before the event of interest is observed. Such results are reported as being >upper limit. Interval censoring occurs when the measurement instrument only provides a range rather than a measured value. For example, an optical particle counter provides a particle count within a particle size range. For example 15 particles were counted in the 0.1 to 0.2 micron range. Particle size is interval censored; particle count is not. A1-2.1.2 The remainder of this discussion will focus on the most commonly encountered censored data case in electronics industry data used for STC; that of MDL censored data. The STC methodology makes use of multiple statistical procedures to deal with or otherwise limit the impact of MDL censored data (8.1, 8.2). These include use of the dual value substitution methodology in STC computation (8.1) and the use of non-parametric statistical methodologies to limit the impact of censored data in the annual review process (8.2). A1-2.1.3 There are numerous detection limit concepts and definitions found in the literature. The concept focused on herein, is that of a Method Detection Limit (MDL). An MDL is a statistically derived figure of merit for a measurement method. Results below an MDL are in some statistical sense (the measured result is not statistically significantly different from zero) less reliable than data which is above the MDL. For the SEMI MDL definition, see SEMI C10 – Guide for Determination of Method Detection Limits. MDLs are, in common practice, often applied to uncensored below MDL data values in the following manner: Suppose the MDL=10 ppb and 8.1 ppb is measured, this result would be reported as <10 ppb rather than as the observed 8.1 ppb. The SEMI MDL definition does not say to apply the MDL, a figure of merit for a measurement method, in this manner. When such censoring is applied, it has undesirable consequences to subsequent statistical analysis. The intent of such censoring is to prevent over interpretation of marginally determined trace contamination data. A1-2.1.4 This practice of censoring below MDL measured values generally causes more data analysis/interpretational issues than the issues that arise from dealing with data that is measured with low relative precision. It is for this reason that the Guide recommends use of uncensored data when available or practical.




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

When this is not the case, the Guide provides, as an alternative, a methodology for estimating STC limits from MDL censored data. A1-2.1.5 The issue of performing statistical analysis on MDL censored data arises from the fact that commonly available statistical tools do not handle censored data. The typical response to this when MDL censoring has been applied is to substitute the value of the MDL for the censored value. For example, if the MDL is <10 ppb, many would choose to substitute 10 ppb for each and every MDL censored value. Some less used alternative single number substitutions have been to use the MDL/2 or perhaps even zero as the value to substitute for every below MDL value. None of these single number substitutions works well. For example, if a distribution is normally distributed with an average of 10 ppb, a standard deviation of 3, is censored to an MDL of 10 ppb and 10 ppb is substituted for all censored data, then an average of 11.2 is to be expected along with a standard deviation of 1.8. This will corrupt control limit estimation as well. A1-2.1.6 Those with advanced statistical tools may choose to use Maximum Likelihood Estimation (MLE) based methods. MLE methods were considered, but not selected for use in the STC Guide. Section 8.1 describes the dual value substitution methodology used in STC limit estimation. A1-2.1.7 There are a series of MDL related issues (estimation, reliability, sample size, best practices) that are not explicit, nor intended to be part of the STC Guide. Assuming acceptance of this Guide, a more complimentary and comprehensive version of SEMI C10 – Guide for Determination of Method Detection Limits should be developed. A1-2.2 Batch Analysis: See ¶ 4.3 for definition and ¶ 5.9 for application. In batch analysis, a single reported result is applied to multiple lots, cylinders or shipments originating from the same production batch. Such a situation only provides a single measurement for STC determination. If the copied data value were to be applied multiple times this will result in measurement strings that have no variability within them; for example, the standard deviation of (3.1, 3.1, 3.1, 3.1, 3.1, 3.1, 3.1, 3.1, 3.1, and 3.1) is 0. A1-2.2.1 Examine the batch analysis scenario where one has 15 batch results from which each batch generated 10 lots. If the fifteen results are repeated 10 times each in subsequent analysis, it now appears that there are 150 measurements and 149 degrees of freedom for STC statistical analysis. Statistical analysis on this data will grossly overestimate the reliability of STC limits or that of any other estimated statistic. The reality is that there are only 14 degrees of freedom in this scenario, one less than the number of batches. A1-2.3 Raw Material Lot Structure: A kind of higher level statistical analogue to batch analysis (A1-2.2 ) may occur when multiple material batches are produced from the same raw material lot (see 5.10). If the raw material lot is the dominant contributor to the final measured result for each production batch, a somewhat analogous impact to the above noted effects will take place where batch assumes the prior role of lot and raw material lot assumes the prior role of batch. It’s not a perfect analogy since there will be some residual batch to batch measurement variation in the batches produced from the same raw material lot as well as the possibility of raw material lots not being entirely homogenous. A1-2.3.1 If the raw material lot is the dominant contributor to the final measured result for each production batch, then it may be more appropriate to perform the STC analysis with the raw material lot average that results from averaging the production batch results rather than using the individual batch results. A1-2.4 Non-Normally Distributed Data: Many standard statistical methods assume that a normal distribution can reasonably describe how data is distributed. A normal distribution can be described through two parameters, a mean and a standard deviation. Normal distributions are mound shaped and symmetrically distributed about their mean. The Empirical Rule indicates that about 68% of normally distributed data will be within one standard deviation of the mean, about 95% of normally distributed data will be within two standard deviations of the mean, and that about 99.7% of normally distributed data will be within three standard deviations of the mean. A1-2.4.1 Traditional 3 sigma individuals control charts are based on the dual assumptions of an underlying normal distribution and large enough sample size. Much of the trace contamination data to be used in STC limit construction does not follow a normal distribution well-enough to allow normal theory construction of STC limits. The typical issue is that many trace contaminants provide distributions that are skewed to the right (have longer tails to the right of the mean than to the left of the mean). Such a violation means that normal theory will typically provide control limits that are too tight versus what a process is actually capable of. The skewness correction methodology in ¶ 8.1 is the mechanism for dealing with non-normality. A1-2.5 Provisional Limits Less Rigorous: Limits that are generated from highly censored data (4.10, 5.7, 7.2, 8.2) are deemed to be provisional. They have this special categorization due to how highly leveraged the STC limit estimation process becomes even when the minimum sample size of 100 has been met or exceeded. Provisional




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

limits are changed annually as more data becomes available in order to obtain more stable STC limits. Suppliers and producers should consider the possibility of applying provisional limits less aggressively than non-provisional limits.

A1-3 SEMI STC Method Technical Background A1-3.1 Properties of the Skew Correction Equation—The general form of the Skew Correction (SC) Equations for an upper control limit is:

UCL = X + wS, (A1-1)

A1-3.1.1 where: w = (tcrit + ak3)S(1 + 1/n)½, n is the sample size, X is the sample average, tcrit is a t-value with n-1 degrees for freedom and one sided tail probability equal to the individual Type I error, S is the sample standard deviation, k3 the sample coefficient of skewness and a is a factor that adjusts the limits for the amount of skewness (8.1.1). Note that if tcrit is less than 3 then it is set to 3. For upper control limits if k3 is less than 0 it is set to 0. For lower control limits if k3 is greater than 0 it is set to 0. A1-3.1.2 The SC equation was developed to handle issues unique to semiconductor supplier data that is not usually handled in the standard methods for calculating control limits. These issues are: more than one property measured per sample, asymptotic sample sizes cannot be assumed, the data often comes from skewed distributions (non normal) and the data is often highly censored. A1-3.1.3 Simulations were used to determine the a factor as a function of n and the number of properties measured per sample (p). Six skewed distributions were picked that were representative of the types of distributions found in typical supplier data for the semiconductor industry. A1-3.1.4 For a given n and p, the value for a was determined such that on average the targeted individual Type I error was obtained (see Appendix A1-2 for a discussion of the targeted individual Type I error). An empirical equation was then developed to express a as a function of n and p. A1-3.1.5 Under conditions appropriate for a Shewhart chart, the SC equation reduces to the standard Shewhart equation

UCL = X +3S. (A1-2) A1-3.1.6 The Shewhart equation applies when p=1, n ∞ and the data is normally distributed. If n ∞ is replaced with a finite sample size and p allowed to be greater than 1 then this becomes

UCL = X + tcritS(1+1/n)½. (A1-3) A1-3.1.7 If the data truly comes from a normal distribution, then this equation will precisely maintain the targeted individual and family Type I errors. A1-3.1.8 If the data comes from a skewed distribution, then the full SC equation applies:

UCL = X +(tcrit + ak3)S(1 + 1/n)½. (A1-4) A1-3.1.9 This equation will approximately maintain the family Type I error at 1%.

A1-3.2 Type I Error issues—The Type I error is a general concept in statistics that is used in hypothesis testing. In the context of control limits, the hypothesis that is tested is the assumption that the process is in statistical control. A process is in control if the properties measured in the process are within the control limits. The Type I error is then the probability that the process is really in control but the measurements will indicate that the process is out of control (OOC). This can happen because control limits demark tail regions in the distributions that describe the process that have low probability of occurring. This probability of OOC is manually selected and it is the Type I error. For example, when all the conditions are met for a Shewhart control chart to apply, the probability that a single observation is beyond ± 3 standard deviations from the mean is 0.27%. Thus, if the process is in control, it is expected to get an OOC signal about 0.27% of the time. A1-3.2.1 There are several ways that the Type I error is applied. One is to apply it to an individual measured property (individual Type I error, αInd). This is the probability that if the process is in control then a measurement of that particular property will read OOC. Another way to apply this is on a family or global level. The family Type I error, αFamily, is the probability (again, assuming that the process is in control) that a particular sample will have one or more properties that will read OOC. As long as more than one property is measured per sample then αFamily is




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

always larger than αInd. If there are p properties measured per sample then the relation between αFamily and αInd is shown below.

.)1(1 pIndFamily αα −−= (A1-5)

A1-3.2.2 Thus, if the individual Type I error is 0.0027 (0.27%) and 20 properties are measured per sample then the family Type I error is 1-(1-.0027)20 = 0.053 or 5.3%. This means that if the process is in control, 5.3% of the samples will have one or more properties that read OOC. A1-3.2.3 The freedom exists to set the Type I error at the family level. If there are p properties measured per sample, then the Type I errors for the individual control charts is given by:

.)1(1 /1 pFamilyInd αα −−= (A1-6)

A1-3.2.4 The method for calculating the STC limits set a target family Type I error at 1%. This means that at the target level on average 1% of the samples will have one or more properties read OOC when the process is in fact in control. If 20 properties are measured per sample, the Type I error for the individual control charts (αInd) is 0.00050 or 0.05%. A1-3.2.5 In practice, the observed family Type I error rate will not always be 1%. There are numerous reasons for this. A1-3.2.5.1 Even if the true long-term family Type I error for a given process was exactly 1%, yearly samples will vary randomly around that value. A1-3.2.5.2 The development of the Skew Correction (SC) equation used a set of distributions that covered a wide range of the distribution patterns that are typically observe with data from semiconductor suppliers. If enough of the properties for a particular product have distributions that were not well represented by those used to develop the equation, then the observed Type I error will be slightly different. A1-3.2.5.3 If the process has really shifted so that the control limits for some properties no longer apply, then the rate at which those properties are OOC will be increased and thus inflate the rate at which samples are identified as OOC. The true family Type I error should have all OOC points removed for properties where the STC limits have truly shifted. The calculations to determine if STC limits have changed are done at the yearly review (4.2, 8.2). A1-3.3 Affect of Sample Size on the Calculated STC Limits—There are two points to understand with respect to sample size. First, as a general rule the larger the sample size (n), the tighter the STC limits will be. However, there is an asymptotic limit to this affect. For a given number of properties measured per sample (p), both tcrit and the a factor will decrease but approach a limiting value. As the sample size increases, tcrit will approach the Z score with the same corresponding individual Type I error. The a factor will approach a value approximately equal to (p/4.151)0.246. Thus, for p=20, tcrit will approach 3.289 and a will approach 1.472 as n gets arbitrarily large. A1-3.3.1 The second point is that as n gets smaller not only do the STC control limits get pushed further out but also the variation in the control limits calculated from one year to another can become quite large (even when the process remains in control). This is because as n gets smaller the uncertainty in the estimates of the population average, standard deviation and skewness increases. This results in more uncertainty in the STC limits. In this sense the calculated STC limits are unstable at small n. A1-3.3.2 Numerous studies were done to identify how small n can be before this instability becomes an issue. It was found that for n>100, the gain in stability increased very slowly with increasing n. Consequently, the minimal recommended sample size was set at 100. While it is not recommended, STC limits can be calculated for sample sizes between 30 and 99, however, the user must beware that there is a good deal of uncertainty in the limits. This means that by sheer luck of the draw, some of the limits may be artificially high (producing very low OOC rates) or artificially low (creating very high false OOC rates). The closer n gets to 30, the more severe this problem becomes. The STC limit calculations should never be applied to sample sizes less than 30. The a factors were never determined for n less than 30 and the equations used to calculate a will not extrapolate well below this sample size. A1-3.4 Affect of the Number of Properties Measured Per Sample—As the number of properties measured per sample (p) increases, the STC limits are pushed further out. This is because the individual Type I error required to




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

maintain the target family Type I error at 1%, decreases as p increases. The connection between the individual Type I error, αInd, and p (for a 1% family Type I error) is shown below.

αInd = 1-0.991/p. (A1-7)

A1-3.4.1 A smaller individual Type I error means that the STC limit, which demarks that tail region in the distribution, has to move further into the extremes of the distribution. A1-3.4.2 Upper control limits will increase without bound as p increases. Likewise, lower control limits will decrease without bound as p increases. However, the rate of change decreases with increasing p. The plot shown below illustrates this for a UCL. In this case n=100, sample average = 5, sample standard deviation = 1 and the sample coefficient of skewness =1.

UCL vs Number of Measured Properties (n=100, avg = 5, S=1, k3=1)

02468

1012141618

0 20 40 60 80 100 120

p

UC

L

Figure A1-1

UCL vs Number of Measured Properties

A1-3.5 Dual Value (DV) Substitution— The DV substitution method was developed to improve upon some of the more common practices that replaced censored values with a single number. These are usually the MDL, half the MDL or 0. This practice tends to underestimate the standard deviation (S) and overestimate the coefficient of skewness (k3). The DV substitution method maintains some variation in the censored data by alternating between 0 and the MDL when evaluating the reference data set for control limits (8.3). As the level of censoring increases, this protects from highly distorted values for S and k3. A1-3.5.1 At 100% censoring it can be shown that

11

2)1(

−++

=nnt

MDLUCL crit , (A1-8)

A1-3.5.2 where n is the sample size and tcrit is a t-value with n-1 degrees of freedom and one tailed probability of the individual Type I error. A1-3.5.3 For large n this becomes

UCL = MDL(1+Z)/2, (A1-9)




LETT

ER (Y

ELLO

W) B

ALL

OT


Date: 9/12/2007

A1-3.5.4 where Z is the corresponding Z score. A1-3.5.5 It can be shown, that for typical ranges of the sample size and number of properties measured per sample, that the UCL will vary from 2.04 to 2.60 times the MDL (for 100% censoring). This pushes the STC limit away from the MDL in a manner that is less aggressive than a past practice of some suppliers to never set a STC limit at less than 3 times the MDL. NOTICE: SEMI makes no warranties or representations as to the suitability of the standard(s) set forth herein for any particular application. The determination of the suitability of the standard(s) is solely the responsibility of the user. Users are cautioned to refer to manufacturer’s instructions, product labels, product data sheets, and other relevant literature respecting any materials or equipment mentioned herein. These standards are subject to change without notice. By publication of this standard, Semiconductor Equipment and Materials International (SEMI) takes no position respecting the validity of any patent rights or copyrights asserted in connection with any item mentioned in this standard. Users of this standard are expressly advised that determination of any such patent rights or copyrights, and the risk of infringement of such rights are entirely their own responsibility.