Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
1
DETERMINING OPTIMUM SAMPLE SIZES WITH RESPECT TO COST FOR
MULTIPLE ACCEPTANCE QUALITY CHARACTERISTICS
By
DOOYONG CHO
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2010
2
© 2010 DOOYONG CHO
3
I would like to dedicate this dissertation to my supporting parents, Joonhee Cho and SungOck Shin and to my wife, Hyeseung Chung and to my son, Leo Seungjae Cho.
4
ACKNOWLEDGMENTS
It is a great pleasure for me to thank and acknowledge the many individuals who
assisted me and supported me during the course of my doctorial program. I begin by
expressing my gratitude to Dr. Fazil T. Najafi, my advisory committee chairman, for his
continuing encouragement, patience, and support throughout my studies at the
University of Florida. He has been treating me as his son in my life in United States of
America as well as in my school life. I will always be grateful for lessons learned under
his tutelage.
I am greatly indebted to Mr. Peter A. Kopac, P.E., Senior Research Engineer for
the Federal Highway Administration, who helped me select this research topic and
contribute toward fulfilling some of the FHWA research needs. I would like to thank him
for his invaluable assistance, patience, advice, and critique throughout this research. I
want to express my gratitude to Dr. Nasir G. Gharaibeh, from Texan A&M University for
assisting me on a program that became an excellent starting point for this research. I
would also like to thank the rest of my committee members, Dr. Mang Tia, Dr. Reynaldo
Roque, and Dr. Ian Flood, for their support, guidance, and help in accomplishing my
work. I would have not been able to reach this milestone if not for their advice, guidance,
and support. I would like to thank Eric Chavez of the Colorado Department of
Transportation for providing the data in a form suitable for performing the analyses in
this study. I would also like to extend my thanks to all of my friends for their support in
the progress and completion of my study.
Finally, I express my deepest gratitude to my parents, my wife, and my son for
their love and support and for many sacrifices they have provided me with the
opportunities that enabled me to pursue my higher education at the University of Florida.
5
I will always be grateful for everything they have done and owe them a debt that
can never be repaid.
6
TABLE OF CONTENTS
page
ACKNOWLEDGMENTS .................................................................................................. 4
LIST OF TABLES ............................................................................................................ 8
LIST OF FIGURES .......................................................................................................... 9
ABSTRACT ................................................................................................................... 11
CHAPTER
1 INTRODUCTION .................................................................................................... 13
1.1 Background ....................................................................................................... 13
1.2 Problem Statement ........................................................................................... 14
1.3 Objectives ......................................................................................................... 15
1.4 Scope ................................................................................................................ 16
1.5 Research Approach .......................................................................................... 16
1.5.1 Task 1 ...................................................................................................... 16
1.5.2 Task 2 ...................................................................................................... 16
1.5.3 Task 3 ...................................................................................................... 16
1.5.4 Task 4 ...................................................................................................... 17
1.5.5 Task 5 ...................................................................................................... 17
1.5.6 Task 6 ...................................................................................................... 17
2 LITERATURE REVIEW .......................................................................................... 18
2.1 Sample Size ...................................................................................................... 18
2.1.1 Central Limit Theorem ............................................................................. 18
2.1.2 Statistical Equations ................................................................................ 20
2.1.3 Practical Consideration ............................................................................ 22
2.2 Optimization with respect to Cost...................................................................... 22
2.3 Determining Sample Size for PWL Specification .............................................. 25
2.3.1 Percent Within Limit ................................................................................. 25
2.3.2 Determining Optimum Sample Size......................................................... 27
3 OPTIMUM SAMPLE SIZE ...................................................................................... 28
3.1 Introduction ....................................................................................................... 28
3.2 Optimization Model ........................................................................................... 28
3.3 Discussion of Assumptions ............................................................................... 33
4 PRELIMINARY OPTIMIZATION MODEL AND ANALYSIS .................................... 36
7
4.1 Introduction ....................................................................................................... 36
4.2 Preliminary Analysis .......................................................................................... 37
4.2.1 Single AQC .............................................................................................. 37
4.2.2 Independent Double AQCs ...................................................................... 40
4.2.3 Dependent Double AQCs ........................................................................ 42
5 DATA COLLECTION .............................................................................................. 45
5.1 Introduction ....................................................................................................... 45
5.2 Colorado State Acceptance Plans .................................................................... 45
6 DATA ANALYSIS .................................................................................................... 48
6.1 Acceptance Quality Characteristics Data Analysis............................................ 48
6.1.1 Asphalt Content Data Analysis ................................................................ 48
6.1.2 In-Place Density Data Analysis ............................................................... 52
6.1.3 Multi Characteristics Analysis .................................................................. 53
6.2 Correlation Between Two AQCs ....................................................................... 54
6.3 Pay Adjustments ............................................................................................... 58
6.3.1 CDOT Data with Agency’s Desired Pay Factor ....................................... 59
6.3.2 Contractor’s Expected Pay Factor ........................................................... 62
6.3.3 Pay Adjustment Acceptance Plan ........................................................... 63
6.4 Cost of Sampling and Testing ........................................................................... 68
6.6 Cost of Contractor Reaction .............................................................................. 72
7 CONCLUSIONS RECOMMENDATIONS ............................................................... 75
7.1 Summary and Conclusions ............................................................................... 75
7.2 Recommendations ............................................................................................ 77 APPENDIX
A PWL QUALITY MEASURE ..................................................................................... 79
A.1 Estimating PWL ................................................................................................ 80
A.2 Calculation and Rounding Procedures ............................................................. 80
A.3 Quality Index and PWL ..................................................................................... 81
A.4 Example ........................................................................................................... 90
LIST OF REFERENCES ............................................................................................... 93
BIOGRAPHICAL SKETCH ............................................................................................ 95
8
LIST OF TABLES
Table page 3-1 Acceptable Risk Levels, based on the Criticality of the Inspected Product
(AASHTO 1995). ................................................................................................ 31
3-2 Probability of Accepting Poor Quality Lot Considering Two AQCs ..................... 33
3-3 Inputs for Determining Optimum Sample Sizes for Two AQCs .......................... 34
4-1 Fictitious Data ..................................................................................................... 36
4-2 Category of Fictitious Data ................................................................................. 37
5-1 Formulas for calculating PF based on sample size ............................................ 46
5-2 “W” Factors for Various Elements ....................................................................... 47
6-1 Asphalt Content Proportion ................................................................................ 48
6-2 In-Place Density Proportion ................................................................................ 52
6-3 Classified AQCs PWL Data with Pay Factor....................................................... 60
6-4 Agency’s Desired Pay Factor for A Lot Having any Combination of AQC PWLs .................................................................................................................. 61
6-5 Difference Between Agency’s Desired Pay and Contractor’s Expected Pay for n=3 ................................................................................................................ 64
A-1 Quality Index Values for Estimating PWL I ......................................................... 82
A-1 Quality Index Values for Estimating PWL II ........................................................ 83
A-2 PWL Estimation Table for Sample Size n = 5 ..................................................... 84
A-3 Another PWL Estimation Table for Sample Size n = 5 ..................................... 875
A-4 Areas Under the Standard Normal Distribution................................................... 87
9
LIST OF FIGURES
Figure page 2-1 Relationship between standard error of the mean and sample size ................... 20
2-2 Relationship among costs in sampling adapted from Lapin ................................ 24
2-3 Percent Within Limits. LSL = Lower Specified Limit, USL = Upper Specified Limit, PD = Percent Defected, PWL = Percent Within Limits .............................. 25
2-4 Three Examples of Symmetric Beta Distributions ............................................... 27
3-1 Graphical depiction of the sample size optimization concept. ............................ 29
3-2 Typical OC Curve for Accept/Reject Acceptance Plan ....................................... 31
4-1 AQC1 Histogram ................................................................................................ 37
4-2 AQC2 Histogram ................................................................................................ 37
4-3 Proportion of prior lots of rejectable quality ........................................................ 38
4-4 Venn Diagram for single AQC ............................................................................ 39
4-5 Single AQC model for Optimum Sample Size .................................................... 40
4-6 Venn Diagram for Independent Double AQCs .................................................... 41
4-7 Independent Double AQCs model for Optimum Sample Size ............................ 42
4-8 Venn Diagram for Dependent Double AQCs ...................................................... 43
6-1 Proportion Histogram at each sample size category for Asphalt Contents I ....... 50
6-2 Proportion Histogram at each sample size category for Asphalt Contents II ...... 51
6-3 Proportion Histogram for Asphalt Contents ........................................................ 51
6-4 Proportion Histogram for In-Place Density ......................................................... 53
6-5 Plot of Density and Asphalt Content Lot PWL Pairs ........................................... 56
6-6 Plots of Randomly Assigned Density and Asphalt Content Lot PWL Pairs ......... 57
6-7 Pay Equation Curves for Asphalt Content and Density in SpecRisk software .... 62
6-8 Pay Equation Curves for Asphalt Content and Density with a low threshold ...... 63
10
6-9 Application of optimization model with negative agency acceptance costs ........ 65
6-10 Application of optimization model with negative agency acceptance costs (with Threshold) .................................................................................................. 66
6-11 Comparison of a composite pay equation and its expected pay curve ............... 67
6-12 National Average Cost of Sampling and Testing for unmolded HMA (Binder Content and Gradation together) (2008 Prices) .................................................. 69
6-13 National Average Cost of Sampling and Testing for HMA Density Cores (2008 Prices) ...................................................................................................... 69
6-14 Distribution of CDOT asphalt content PWL estimates for lots with n=3, 9, and 17-20 .................................................................................................................. 74
A-1 Illustration of the Calculation of the Z -statistic ................................................... 89
A-2 Illustration of Positive Quality Index Values ........................................................ 91
A-3 Illustration of a Negative Quality Index Value ..................................................... 92
11
Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy
DETERMINING OPTIMUM SAMPLE SIZES WITH RESPECT TO COST FOR
MULTIPLE ACCEPTANCE QUALITY CHARACTERISTICS
By
Dooyong Cho
August 2010
Chair: Fazil T. Najafi Major: Civil Engineering
There has been little and limited research done to determine the optimum sample
size n for highway construction acceptance plans containing pay adjustment provisions.
To optimize the sample size of Acceptance Quality Characteristics (AQCs) with respect
to the cost to the agency, an understanding is required of costs of testing and of the
cost consequences associated with making a wrong acceptance decision. This study
developed an optimization model to determine the sample size. Actual state agency’s
data were used not only to check the validity of model’s assumptions but also draw the
conclusions. Among the key conclusions were: (1) The optimum sample size is
generally small (n=3) for Percent Within Limits acceptance plans; (2) The degree of
correlation among AQCs has a negligible effect on the optimum sample size; and (3)
The highway agency tends to underpay contractors through use of its pay adjustment
provisions.
Some cautions are presented for state highway agencies planning to use small
sample sizes. Agencies are encouraged to perform their own optimization calculations.
By doing so, they will gain a better understanding of their acceptance plan systems and
12
associated costs, and have greater confidence in applying economic decision analysis
principles to minimize expected costs and optimize statistical acceptance risks.
13
CHAPTER 1 INTRODUCTION
1.1 Background
There has been little research done to establish the optimum sample sizes∗ for
today’s highway construction acceptance plans containing pay adjustment provisions.
One notable recent research project was conducted for the Federal Highway
Administration by the Transtec Group, Inc. The purpose of that research was to
determine whether today’s commonly used sample sizes (i.e., n = 4-7 for most
acceptance quality characteristics [AQCs]**) are too small, too large, or about right. The
larger the sample size, the lower the risk of making wrong decisions about rejecting,
accepting, or assigning a pay factor to a lot. On the other hand, the more sampling and
testing performed, the greater the cost and personnel required. If this sample size is too
small, the cost consequences in the long run of making erroneous acceptance or pay
adjustment decisions would be too high for state highway agencies.
The approach that was taken in that research was one that involved optimization
of sample size with respect to minimizing the cost of acceptance plans to the highway
agencies. Numerous assumptions were made, and the research conclusions/findings
were general in nature. Also, it was anticipated specific questions will need to be
answered through further research before appropriate implementation of findings can be
done by highway agencies.
∗ Within the highway construction community, the term “sample size” refers to the number of samples obtained from a unit of materials or construction. It should not be confused with the physical size or amount of material or construction that is sampled. ** A quality characteristic is an attribute of a unit or product that is actually measured to determine conformance with a given requirement (e.g., asphalt content, density). When the quality characteristic is measured for unit or product acceptance purposes, it is an acceptance quality characteristic.
14
To optimize the sample size of AQC(s) with respect to the cost to the agency, an
understanding is required of costs of testing and the cost consequences associated with
making a wrong acceptance decision. An understanding of specific AQCs is also
required, including the correlation among AQCs in an acceptance system composed of
several individual AQC acceptance plans, as correlation among AQCs may affect
optimum sample sizes in an acceptance system. The research described in this thesis
deals with determining the optimum sample sizes for multiple AQCs in a pay adjustment
acceptance system* and will result in specific recommendations for implementation.
1.2 Problem Statement
The following are some questions that will be addressed:
• What is the optimum sample size n for a selected two-AQC acceptance system
such as to minimize the cost to a state highway agency? Today’s commonly used
sample sizes range between 4 and 7 for most AQCs. This range of sample size is
typically established based on practical considerations such as personnel and time
constraints. While a sample size within this range may be practical, it is unclear if it
is economically optimal.
• What effect does the degree of correlation among AQCs have on the optimum
sample size? If two AQCs are correlated, are their optimum sample sizes greater
than, the same as, or less than if they were independent AQCs? Quality assurance
experts have encouraged state highway agencies to employ independent AQCs,
but many agencies still use some correlated AQCs.
• What effect do pay adjustments have on optimum sample size determination? * A major difference between quality management systems in highway construction and those in other applications is that units of production submitted for acceptance purposes are either rejected, accepted, or accepted with a pay adjustment (either positive or negative).
15
Do pay increases and pay decreases necessarily balance out in the long run such
that the highway agency neither gains nor loses money through use of its pay
adjustment provisions, or is there some gain or loss to the agency? Although most
state highway agencies have pay adjustment provisions that include pay increases
(incentives) as well as decreases (disincentives), for some AQCs only pay
decreases are possible in some states.
• How do the costs of sampling and testing compare against the expected cost of
the consequences to the state highway agency as a result of a wrong acceptance
decision? State highway agencies need an optimization method they can apply on
their own acceptance systems. The use of any optimization method to minimize
costs requires an understanding of economic decision theory and of costs and cost
models.
1.3 Objectives
The objectives of this study are to:
1. Collect real state highway agency data and use the data to determine the
optimum sample sizes for two AQCs commonly employed in current highway
construction acceptance plans for hot mix asphalt (HMA).
2. Evaluate the degree of correlation between the two AQCs.
3. Determine the effect of having correlated versus uncorrelated AQCs on the
determination of optimum sample size.
4. Evaluate pay adjustment system.
5. Compare findings against current practices and make practical suggestions for
implementation.
16
1.4 Scope
The study consists of developing procedures that can be used to determine
optimum sample sizes for AQCs and evaluating correlation between AQCs and its effect
on optimum sample size. The results from this study are to be compared with state
highway agency practice. The overall goal of this study is to minimize, through the use
of optimal sample sizes, the cost of state highway agencies’ acceptance plans.
1.5 Research Approach
The research approach that was followed in order to fulfill the research objectives
mentioned in Subheading 1.3 is described below:
1.5.1 Task 1
Conduct a literature review to (a) identify how AASHTO and state highway agencies
determine sample sizes for their acceptance plans (b) critically evaluate the findings
from the Transtec Group research, and (c) establish the AQCs to be analyzed for
purposes of this study and the degree to which the AQCs are correlated.
1.5.2 Task 2
(1) Drawing on the Transtec Group research, develop an advanced model that will allow
determination of optimum sample sizes for HMA concrete.
(2) Develop a procedure that will be used to determine optimum sample size n for a
single AQC and multiple (at least two) AQCs.
1.5.3 Task 3
Obtain the necessary state highway agency data needed to apply the advanced
optimization model developed in Task 1 in keeping with the procedure developed in
Task 2.
17
1.5.4 Task 4
Analyze the state highway agency data to determine the degree of correlation that is
present between the AQCs selected in Task 1.
1.5.5 Task 5
Determine the optimum sample sizes for the state highway agency’s single and multiple
AQC acceptance plans.
1.5.6 Task 6
Draw conclusions and develop recommendations for how to minimize the cost to state
highway agency through the use of optimum sample sizes.
18
CHAPTER 2 2 LITERATURE REVIEW
2.1 Sample Size
Sample size refers to the number of samples obtained from a unit of materials or
construction. State highway agencies commonly use a sample size between 3 and 7
units per lot (Russell et al. 2001, Mahoney and Backus 2000). This range of sample size
is typically established based on practical considerations such as personnel and time
constraints. While a sample size within this range may be practical, it is unclear if it is
economically optimal. When highway construction QA specifications were first being
developed in the 60’s and 70’s, the issue of acceptance sample size was approached
several ways.
2.1.1 Central Limit Theorem
In probability theory, the central limit theorem states conditions under which the
mean of a sufficiently large number of independent random variables, each with finite
mean and variance, will be approximately normally distributed. The central limit theorem
also justifies the approximation of large-sample statistics to the normal distribution in
controlled experiments. Equation 1 shown in FHWA report “Cost Effectiveness of
Sampling & Testing Programs”, FHWA/RD-85/030 explained that if a population has a
normal distribution with mean μ and standard deviation σ, then the distribution of the
means x of samples of size n from that population approaches a normal distribution
with mean μ and standard deviation n/σ as the sample size n increases. The term
n/σ is also called the standard error of the mean ( Mσ ).
19
n
xZσµ−
= ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅1
Where,
Z = standardized statistic with a mean of zero and standard deviation of one
x = sample mean
μ = population mean
σ = population standard deviation
n = number of samples
Equation 1 can be rewritten as:
nM /σσ = ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅2
The assumption of a normal distribution for sample means from a normal parent
population does not hold for small n; these obey a distribution called a “Student’s t-
distribution.” Small n here might be considered to be n less than 20 shown in Figure 2-1.
The methodology does make this distinction in actual practice, using the t-statistic rather
than z-statistic.
20
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 5 10 15 20 25 30 35 40Sample Size, n
Stan
dard
Err
or o
f the
Mea
n (N
orm
aliz
ed)
Figure 2-1. Relationship between standard error of the mean and sample size
2.1.2 Statistical Equations
Many statistic text books present Equation 3. The sample size (per lot) required to
meet allowable tolerance in the mean (typically stated in the construction
specifications), considering both the seller’s risk (α) and buyer’s risk (β) can be
computed as follows:
( ) 2
+=
eZZ
nσβα ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅3
Where
=n sample size
=αZ standard normal distribution value for the required seller’s risk
21
=βZ standard normal distribution value for the required buyer’s risk
=σ standard deviation of the population
=e tolerable error allowed by the specifications
ASTM E122-09 “Standard Practice for Calculating Sample Size to Estimate, With
Specified Precision, the Average for a Characteristic of a Lot or Process” is intended for
use in determining the sample size required to estimate, with specified precision, a
measure of quality of a lot or process. The practice applies when quality is expressed as
either the lot average for a given property, or as the lot fraction not conforming to
prescribed standards. The level of a characteristic may often be taken as an indication
of the quality of a material. If so, an estimate of the average value of that characteristic
or of the fraction of the observed values that do not conform to a specification for that
characteristic becomes a measure of quality with respect to that characteristic. ASTM
E122 present equation of the following form for the required sample size, n:
23
=
En σ ……………………………………….…………………………………………4
Where:
=n sample size σ = the estimate of the lot standard deviation E = the maximum acceptable difference between the true average and the sample
average. The multiplier 3 is a factor corresponding to a low probability that the difference
between the sample estimate and the result of measuring (by the same methods) all the
units in the lot or process is greater than E. The value 3 is recommended for general
use. With the multiplier 3, and with a lot or process standard deviation equal to the
advance estimate, it is practically certain that the sampling error will not exceed E.
22
These equations have the following limitations to apply to Highway construction
Quality Assurance:
1. It does not consider cost.
2. In its current form, it is meant only accept/reject acceptance plan.
3. It applies to estimates of the average, and is not meant for Percent Within Limit
(PWL) estimates.
4. It is for a single acceptance plan (one AQC) and not for an acceptance system
(more than one AQC). As the number of AQCs increase, the probability of rejecting the
lot increases. Thus, as the number of AQCs increases, the probability of rejecting good
construction (α risk) increases, and the probability of accepting poor construction (β
risk) decreases.
2.1.3 Practical Consideration
Today’s’ commonly used sample size is typically established based on practical
considerations such as personnel and time constraints. From a practical standpoint, a
reasonable acceptance sample size is the number of tests the technician(s) can perform
in one day (assuming the lot represents one-day’s production). This guidance typically
results in n = 5 for many AQCs. While a sample size may be practical, it is unclear if it is
economically optimal.
2.2 Optimization with respect to Cost
Lapin addressed that whether a sample size is appropriate depends upon (1) the
disadvantage of erroneous estimates, and (2) the costs of obtaining the sample. Larger
samples are more reliable, but they are also more costly. The costs of obtaining the
sample must therefore be balanced against any potential damage from error, as
reliability and economy are competing ends.
23
The risks of error can be considered as having two components: the probability of
making them and the consequences they cause. In a sense, implicit costs are incurred
when an estimate is erroneous if it leads to consequences that may be damaging.
(Lapin, 1975)
In highway construction, it is also very difficult to place cost on the results of an
erroneous estimate. Also, the seriousness of an error will vary. A more reliable sample
will yield even smaller chances of major error.
Ideally, one should then select a sample size that achieves the most desirable
balance between the chances of making errors their costs, and the costs of sampling.
Figure 2-1 illustrates the concepts involved in finding the optimal sample size,
which minimizes the total cost of sampling. The costs of collecting the sample data
increase with the value of n. But larger samples are more reliable, so that the risks of
loss from chance sampling error decline. The total cost of sampling-the sum of
collection costs and error costs-will achieve a minimum value for some optimal n. This is
the sample size that should be used.
24
n
Total cost of sampling
Cost of collecting sample
Sampling error costs
Optimalsample size
Possible sample size
Cost with 100%sampling
Minimum costwith sampling
Maximum savingswith sampling
Figure 2-2. Relationship among costs in sampling adapted from Lapin
Because of the difficulties associated with finding the costs of sampling error, the
above procedure is not usually used to determine the required sample size in traditional
applications. Instead, the focus has been upon a single number that delineates
insignificant errors from decidedly undesirable ones, i.e., e in equation 3. This is called
the tolerable-error level. In determining this level, one acknowledges that all error is
undesirable, but that potential error must be accepted as the price for using a sample n
instead of the entire population N. There are no guarantees that the tolerable error will
not be exceeded. However, large errors may be controlled by keeping the chances of
their occurrence small. Furthermore, as indicated earlier, reducing the chance of error
increases reliability. Thus, one speaks of reliability in terms of the probability that the
estimate will differ from the parameter’s true value by no more than the tolerable error.
25
2.3 Determining Sample Size for PWL Specification
As indicated earlier, previous approaches to determine required sample size are
based on average as the measure of quality rather than the PWL measure or any other
measure of quality. PWL is considered a preferred measure of quality because it
considers both the central tendency and variability in a statistically sound way and
Federal Highway Administration (FHWA) also promoted.
2.3.1 Percent Within Limit
The PWL is the percentage of the lot falling above the lower specified limit (LSL),
below the upper specified limit (USL), or between the specified limits, as seen in Figure
2-2. PWL may refer to either the population value or the sample estimate of the
population value. The PWL quality measure uses the mean and standard deviation in a
normally distributed curve to estimate the percentage of population in each lot that is
within the specified limit (TRB, 2005).
Figure 2-3. Percent Within Limits. LSL = Lower Specified Limit, USL = Upper Specified
Limit, PD = Percent Defected, PWL = Percent Within Limits
26
In practice, it has been found that statistical estimates of quality are reasonably
accurate provided the sampled population is at least approximately normal (i.e., bell
shaped and not bimodal or highly skewed). The PWL is calculated using the following
equation:
( ) ( )( ) 10012
5.0,1100 ×
−−−=−=
NNQBPDPWL
iLi βα …………………………...5
Where
PD = percent defective B(α,β) = beta distribution with parameters α and β (α,β) = shape parameters of the distribution
iLQ = lower quality index for an AQC N = number of samples per lot
Unlike the normal distribution, which is a single distribution that uses the z-statistic
parameter to calculate areas below the distribution, the beta distribution is a family of
distributions with four parameters alpha (α) and beta (β). The PWL calculation uses the
symmetrical beta distribution. For symmetric distributions, the alpha and beta are the
same. Figure 2-3 shows three examples of a symmetric beta distribution. As α and β
values increase the distributions become more peaked. The uniform distribution has
alpha and beta both equal to one. This does not have a well-defined mode because
every point has the same probability. Distributions with alpha and beta less than one are
bathtub shaped curves and generally not useful for statistical modeling (Ramanathan,
1993)
27
Figure 2-4. Three Examples of Symmetric Beta Distributions
2.3.2 Determining Optimum Sample Size
Recent research (Gharaibeh, 2009) addressed the optimum sample size n for pay
adjustment acceptance plans and acceptance systems (two or more AQCs) rather than
accept/reject acceptance plans and single acceptance plans (one AQC) in previous
research. And PWL quality measure was used in this research to determine the
optimum sample size for hot-mix asphalt concrete (HMAC) pavement. PWL-based (or
PD-based) plans are considered to be advantageous because they take product
variability into account and thus promote uniform quality.
28
CHAPTER 3 3 OPTIMUM SAMPLE SIZE
3.1 Introduction
In this Chapter, recent research by Gharaibeh et al. was used to understand how
to determine optimum sample sizes with respect to minimizing the cost of acceptance
plans to the highway agencies. At first, his methodology was thoroughly verified. And
then, through his optimization model and underlying assumptions, the findings and
many questions raised were closely examined. This thesis summarizes that research.
3.2 Optimization Model
Generally a sample size is determined depending upon the disadvantage (cost) of
erroneous estimates and the costs of obtaining the sample. The model used by
Gharaibeh was come from this concept. To optimize sample size with respect to State
Agency’s cost, a quantitative relationship must be established between sample size and
this cost. Figure 3-1 illustrates the State Agency’s total cost of accepting a construction
or material lot consisting of two components (Expected cost of erroneous acceptance
decision and Cost of sampling & testing).
29
Sample Size, n
Expe
cted
Cos
t, $
Optimum Sample Size
Cost of Sampling & Testing
Expected Cost of Erroneous Acceptance Decision
Total Cost of Lot Acceptance
Figure 3-1. Graphical depiction of the sample size optimization concept.
From this model, total cost of lot acceptance can be formulated follows:
Total Cost of Lot Acceptance = CT + CD ……….…………………………......6
Where, Cost of sampling and testing (CT) Expected cost of erroneous acceptance decisions (CD) The cost of sampling and testing increases with the value of sample size. But
larger samples are more reliable, so that the expected cost of erroneous acceptance
decision decline. The total cost of lot acceptance achieves an optimum sample size n.
As the cost of sampling and testing increases (i.e. the slope of this cost increases),
the curve of total cost of lot acceptance changes and then the optimum sample size
would move to the left resulting in a smaller optimum n. Similarly, if the expected cost of
erroneous acceptance decision were greater, the curve of total cost of lot acceptance
would move to the right resulting in a larger optimum n.
In his research, the cost of sampling and testing is relatively straightforward and
can be computed as follows:
30
CT = n × m × c .........................................................................................................7
Where n is sample size, m is number of replicates, and c is unit cost of testing.
The relationship between n and cost of sampling and testing may not be exactly linear
in some cases; in those cases where a linearity assumption is not deemed to be
appropriate, a discrete function can be applied.
It is also important to understand that while an agency can apply the general
model to either a single AQC (a plan) or to multiple AQCs (a system of plans), true
optimization can occur only when the agency applies it to the whole system rather than
just to a component of the system. Most, if not all, highway agencies have multi-AQC
systems. The risks associated with lot acceptance of multi-AQCs are different from
those of a single AQC. The more AQCs, the greater the contractor’s risk (α) of having
good material rejected and the smaller the agency’s risk (β) of accepting poor material.
Operating characteristic (OC) curves should be constructed for each quality
characteristic to determine if the buyer’s and seller’s risks associated with the sampling
plan are acceptable to the agency and the contractor. The seller’s risk (α) is the risk of
erroneously rejecting or assigning a payment decrease to a lot that indeed should be
accepted or assigned a pay increase. The buyer's risk (β) is the risk of erroneously
accepting or not assigning a payment decrease to a lot that indeed should be rejected
or assigned a pay decrease. The seller’s risk represents the contractor’s risk and the
buyer’s risk represents the highway agency’s risk. The graphical representation of an
OC curve for accept/reject acceptance plans is shown in Figure 3-2.
31
Figure 3-2. Typical OC Curve for Accept/Reject Acceptance Plan
AASHTO R9-05 (AASHTO 2005) suggests that highway agencies should design
acceptance plans that control these risks at suitable levels. AASHTO R-9 does not
provide specific recommendations for acceptable risk levels, but it states “The more
critical the application, the lower should be the buyer’s risk. But only under rare
circumstances should the buyer’s risk be lower than the seller’s risk.” AASHTO R9-90
(AASHTO 1995), however, indicates that these risks can be set based on the criticality
of the measured property as it affects safety, performance, or durability as shown in
Table 3-1.
Table 3-1 Acceptable Risk Levels, based on the Criticality of the Inspected Product (AASHTO 1995).
32
A conventional OC curve represents the relationship between PWL (or PD) and
probability of acceptance. The statistical procedure for developing OC curves is well-
documented in the statistics and probability literature (see for example Duncan 1986).
Perhaps one of the most established software tools in the highway construction arena
for developing OC curves is the OCPLOT simulation software (Weed 1996).
For acceptance plans with a single AQC, a lot is considered of poor quality if that
particular AQC is rejectable [i.e., PWL is at (or below) the rejectable quality level (RQL)].
Thus, the agency’s expected cost due to erroneous acceptance decisions is computed
as follows:
CD = βRQL × PRQL × B × S ..................................................................................8
Where
βRQL = buyer’s risk (i.e., probability of erroneously accepting a lot that has a true PWL equal to or less than the RQL, computed at midpoint between zero and RQL). PRQL = proportion of prior lots of rejectable quality throughout the state. This prior distribution of quality can be obtained from the agency’s construction quality databases or paper records of past construction projects. B = unit bid price S = lot size For an acceptance plan that considers multiple AQCs, a lot is considered of poor
quality if at least one of the AQCs is rejectable (i.e., PWL is at or below RQL). Thus, the
agency’s expected cost due to erroneous acceptance decisions is computed as follows:
CD = [ ] SBPPPj
l
jkkiiii ××∑ ×××
=++
111 .... βββ ………………………………………….…9
where l is the number of combinations of βP for all AQCs, where e=βRQL and P =
PRQL for at least one AQC. Combinations that do not have all AQCs at the RQL are
influenced by the acceptable quality level (AQL). Suppose an acceptance plan includes
33
two AQCs, a rejectable-quality lot can occur under any one of the five possible
scenarios shown in Table 3-2.
Table 3-2 Probability of Accepting Poor Quality Lot Considering Two AQCs
The acceptance plans that have been developed for State Highway Agencies are
used to determine accept/reject/pay for all lots that are delivered by the contractor to the
state. To optimize n, we need information on how the quality of submitted lots varies
within a state. Some states have construction quality databases that can be used to
determine the proportions of lots submitted at various estimated quality levels. From
these construction quality databases, indications are that few lots are estimated to be
RQL, thus few lots are rejected; most lots are estimated to be close to or above the AQL
3.3 Discussion of Assumptions
Gharaibeh assumed some items to balance the complexity of the developed
method with the level of accuracy needed to achieve the study’s objectives. In this
chapter, his assumptions were introduced and further discussion in details will be
considered in Chapter 5.
34
Gharaibeh used three different anticipated lot-quality categories in his calculations
shown in Table 3-3. With these lot-quality assumptions, Gharaibeh found optimum n to
be very small (3 or less) for all but a few cases under the “regular” and “poor” quality
categories for all but a few cases under the “regular” and “poor” quality categories.
These categories were based on likely-conservative assumptions because of the
absence of specific data (real state agency’s data).
Table 3-3 Inputs for Determining Optimum Sample Sizes for Two AQCs
He assumed that erroneous acceptance decision is defined as accepting a lot that
should have been rejected. It is assumed that erroneous decisions associated with
assigning the wrong pay adjustment cancel each other out (i.e., the cost to the state
DOT of paying more for a lot than it is worth is offset by the gain to the state DOT of
paying less for a lot than it is worth). He essentially treated pay-adjustment acceptance
plans as if they were accept/reject acceptance plans. The validity check that the
assumption of zero cost of erroneous pay decisions is appropriate will be performed in
Chapter 6.
He assumed that AQCs are independent or weakly dependent. This assumption is
not unrealistic since contractors tend to pay attention to individual AQCs, rather than
combined measures of quality. In other words, if one AQC has poor quality, it does not
35
necessarily indicate that other AQCs also have poor quality. In Chapter 6, the effect of
correlated AQCs on optimum n will be investigated.
He assumed a linear relationship between the cost of sampling and testing and
sample size n. This cost is hard to model because there are many different possible
cost scenarios. The assumption of linearity tends to “average” all the possible cost
scenarios that exist within the state and/or within the acceptance plan system.
He also assumed that the agency’s expected cost due to erroneous acceptance
decisions is based on bid price, and does not consider other costs such as user costs or
future maintenance and rehabilitation costs.
An important third cost element (CR) was missed from the Gharaibeh model. The
CR can be called as the cost of contractor reaction. This cost can be separate from the
two elements: cost of sampling and testing (CT) and expected cost of erroneous
acceptance decisions (CD). If an agency switches to a smaller n than the one it currently
uses, this cost can affect lot cost and quality.
36
CHAPTER 4 4 PRELIMINARY OPTIMIZATION MODEL AND ANALYSIS
4.1 Introduction
To optimize sample size with respect to the cost to the agency, an understanding
is required of costs of testing and the cost consequences associated with making a
wrong acceptance decision. An understanding of AQCs is also required, and particularly
of the correlation among AQCs in an acceptance system composed of several individual
AQC acceptance plans. Correlation among AQCs may affect optimum sample sizes in
an acceptance system. Due to absence of state agency’s real database preliminarily,
fictitious data was created and used as examples of a modified Gharaibeh approach..
These data are presented in Table 4-1.
Table 4-1 Fictitious Data AQC
Lot Number AQC1 (PWL) AQC2 (PWL)
1 70 75 2 100 98 3 90 88 4 100 100 5 95 75 6 80 60 7 100 97 8 91 100 9 100 100
10 80 45 11 48 93 12 99 95
37
4.2 Preliminary Analysis
4.2.1 Single AQC
The data are categorized to 3 parts of proportion for 2 AQCs respectively since we
will use them as input value to compute proportion of prior lots of rejectable quality
shown in Table 4-2 and Figure 4-1 and 2 as histograms.
Table 4-2 Category of Fictitious Data AQC1 AQC2
Proportion Class Proportion Class
1/6 (17%) RQL (0~70 PWL) 1/6 (17%) RQL (0~70 PWL)
1/3 (33%) AQL (71-91 PWL) 1/4 (25%) AQL (71-91 PWL)
1/2 (50%) 95 PWL (above 91 PWL) 7/12 (68%) 95 PWL (above 91 PWL)
0
17 %
50 70 91 10 0
RQL33 %
50 %AQL95 PWL
Figure 4-1. AQC1 Histogram
0
17 %
50 70 91 10 0
RQL 25 %
68 %AQL 95 PWL
Figure 4-2. AQC2 Histogram
38
To optimize sample size with respect to State Agency’s cost, a quantitative
relationship must be established between sample size and these costs. State Agency’s
total cost of accepting a construction or material lot consists of two components
introduced in Equation 6.
For acceptance plans with a single AQC, State Agency’s expected cost due to
erroneous acceptance decisions is computed as follows:
CD = SizeLot PriceUnit xxβ poor) is (AQC rP ××× …...10
Where,
Pr = proportion of prior lots of rejectable quality xxβ = probability of accepting a lot having xx PWL
Proportion of prior lots of rejectable quality can be found in Figure 4-3.
B Area A AreaA Area poor) isquality estimated (AQC1 rP
+=
Figure 4-3. Proportion of prior lots of rejectable quality
State Agency has an acceptance plan (AQC1, asphalt binder content) in use,
and the database shows that 17 percent of the submitted asphalt content lots are in
RQL class, 33 percent are in AQL class, and 50 percent are 95 PWL class. The
assumption is made here that there is only one event that can lead to State making an
RQL
Say 60 PWL PWL estimates
Area A
Area B ( not shaded)
90 95 10 0
39
acceptance decision error that has significant negative consequences. That event
occurs when the state accepts an RQL lot. FIGURE 5 represents the proportion of lots
(0.17) that are RQL.
A17%
AQC is not RQL
AQC is RQL
Figure 4-4. Venn Diagram for single AQC
Equation 10 can be computed for cost of making erroneous decisions as follow:
CD = sizeLot price Unit β 0.17 50 ××× …….. 11
In order to obtain probability of making erroneous decision β , OC PLOT (AASHTO
R-9, 1990) is used with input values (AASHTO Quality Assurance Guide Specifications,
1996) as follows:
Acceptable Quality Level (AQL) = 90%
Rejectable Quality Level (RQL) = 60%
Acceptance Limit (M) = 70%
Sampling and testing costs (CT) for asphalt binder content was approximately
estimated from an anonymous contractor. Total cost of lot accepting was computed by
Equation 6. These procedures have been programmed in an Excel spreadsheet for
rapid computation.
As can be seen from Figure 4-5, optimum sample size n for AQC1 is 19.
40
Sample Size vs. Cost
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
3 8 13 18 23 28
Sample Size
Cos
t
Expected cost of wrong decisionCost of Sampling and TestingExpected total cost
Figure 4-5. Single AQC model for Optimum Sample Size
4.2.2 Independent Double AQCs
State Agency’s expected cost due to erroneous acceptance decisions for multiple
AQCs consist of two parts. If multiple AQCs are not correlated (i.e., independent), cost
equation is computed as follows:
CD = SizeLot PriceUnit )AQCeach xx β AQCeach r(P ×××Σ …………………..12
Where,
AQCeach r P = proportion of prior lots of rejectable quality for each AQC
AQCeach xxβ = probability of accepting a lot having xx PWL for each AQC
State Agency has an acceptance plan system (asphalt binder content and air void,
AQC1 and AQC2 respectively) in use, and the database shows that 17 percent of the
submitted asphalt binder content lots are in RQL class, 33 percent are in AQL class,
and 50 percent are 95 PWL class for AQC1 and 17 percent of the submitted air void lots
41
are in RQL class, 25 percent are in AQL class, and 68 percent are 95 PWL class. All
events that can occur are shown in Figure 4-6 as Venn Diagram.
0 50 % 10 0 %
17%
17%
42%
10 0 %
A
D
E F G
I
CB
H
Figure 4-6. Venn Diagram for Independent Double AQCs
All events that can lead to any consequential acceptance decision error by the
state (i.e., acceptance of a lot that should be rejected) are as follows:
Event A: Both AQCs come from RQL class
Event B: AQC1 is from AQL class, AQC2 is from RQL class
Event C: AQC1 is from 95 PWL class, AQC2 is from RQL class
Event D: AQC1 is from RQL class, AQC2 is from AQL class
Event E: AQC1 is from RQL class, AQC2 is from 95 PWL class
Note that events F, G, H, and I are inconsequential to us.
We have assumed that the only way the State agency can make an acceptance
decision error is by accepting RQL class material. The State cannot make an
42
acceptance decision error when event F, G, H, or I occur. The State can, however,
make an acceptance decision error when both AQCs are RQL (event A), when only
AQC1 is RQL (events D and E) or when only AQC2 is RQL (events B and C)
For this double AQC case, cost of making erroneous decisions is computed as
follows:
CD = [(1/6)(1/6)β50β50 + (1/6)(1/4)β50β90 + (1/6)(7/12)β50β95 + (1/3)(1/6)β90β50 +
(1/2)(1/6)β95β50] sizeLot price Unit ×× ………………………………………………13
As can be seen from Figure 8, optimum sample size n for independent double
AQCs is 15. This is the total n for both AQC1 and AQC2 and is considerably smaller
than the optimum n of about 19 determined in the single AQC1 example.
Sample Size vs. Cost
$0$2,000$4,000$6,000$8,000
$10,000$12,000$14,000$16,000$18,000$20,000
3 8 13 18 23 28
Sample Size
Cos
t
Expected cost of wrong decisionCost of Sampling and TestingExpected total cost
Figure 4-7. Independent Double AQCs model for Optimum Sample Size
4.2.3 Dependent Double AQCs
If multiple AQCs are correlated (i.e., dependent), cost equation is computed as
follows:
43
CD = SizeLot PriceUnit )AQCeach xx β AQC_giveneach r(P ×××Σ …………14
Where,
AQC_giveneach r P = proportion of prior lots of rejectable quality for AQC given the
quality level proportions of other correlated AQCs AQCeach xxβ = probability of accepting a lot having xx PWL for each AQC
Conditional probabilities from fictitious dada are used to make Venn diagram. The
database shows that 0 percent of the air void lots are in RQL class, 50 percent are in
AQL class, and 50 percent are 95 PWL class for AQC2 when asphalt binder content lots
(AQC1) are in RQL class and 0 percent of the submitted asphalt binder content lots are
in RQL class, 100 percent are in AQL class, and 0 percent are 95 PWL class when air
void lots (AQC2) are in RQL class.
0 50 % 10 0 %
17%
17%
50 %
10 0 %
D
E F GI
B
H
Figure 4-8. Venn Diagram for Dependent Double AQCs
44
Events A and C do not occur in fictitious data. Events F, G, H, and I are
inconsequential to us. Acceptance decision error can not be made when neither AQC is
RQL.
For this double AQC case, cost of making erroneous decisions is computed as follows:
CD = [(1/6)(1/2)β50β90 + (1/6)(1/2)β50β95 + (1/6)(1)β90β50] sizeLot price Unit ×× …15
As can be seen from Figure 4-9, optimum sample size n for independent double
AQCs is 21. This size is larger than 15 for independent.
Sample Size vs. Cost
$0
$5,000
$10,000
$15,000
$20,000
$25,000
$30,000
3 8 13 18 23 28
Sample Size
Cost
Expected cost of wrong decision
Cost of Sampling and Testing
Expected total cost
Figure 4-9. Dependent Double AQCs model for Optimum Sample Size
45
CHAPTER 5 5 DATA COLLECTION
5.1 Introduction
A data collection effort was required to obtain information necessary to apply the
advanced optimization model that will allow determination of sample sizes for HMA
concrete. For the collection of data, Colorado State Agency was selected because of
following reasons. First its database was well organized lots of construction project data
from as early as 2000. Second, it was the PWL base acceptance plan as Federal
Highway Administration (FHWA) promoted to use PWL base. Third, it allowed additional
research to better understand Quality Assurance issues. For example, a couple of
AQCs were employed in current state highway construction to evaluate correlation
between the multiple AQCs and determine the effect of having correlated versus
uncorrelated AQCs on the determination of optimum sample size.
5.2 Colorado State Acceptance Plans
As a part of Colorado Department of Transportation’s (CDOT) Standard
Specifications for Road and Bridge Construction, subsection “Conformity to the Contract
of Hot Mix Asphalt” governs the Quality Control (QC)/Quality Assurance (QA)
calculations. Colorado State’s hot mix asphalt gradation acceptance final report has
asphalt content, gradation, in-place density, and joint density acceptance quality
characteristics. This thesis was focused on the asphalt content and the in-place density
characteristics with test sample sizes and pay factors (PF).
A few years ago, a standard provision revises or modifies CDOT’s Standard
Specifications for Road and Bridge Construction. That provision has gone through a
formal review and approval process and has been issued by CDOT’s Project their own
46
quality of hot bituminous pavement. CDOT use this standard special provision on
projects with 5000 or more metric tons (5000 or more tons) of hot bituminous pavement
when acceptance is based on gradation, asphalt content and in-place density.
In this standard provision, sample size (n) in each process determines the final
pay factor. PF is computed using the formulas designated in Table 5-1 (CDOT’s own
table). Appendix A describes PWL quality measure calculation.
Table 5-1 Formulas for calculating PF based on sample size
n (sample size) Formulas for calculating PF Maximum PF
3 0.31177 + 1.57878 (QL/100) - 0.84862 (QL/100)2 1.025
4 0.27890 + 1.51471 (QL/100) - 0.73553 (QL/100)2 1.030
5 0.25529 + 1.48268 (QL/100) - 0.67759 (QL/100)2 1.030
6 0.19468 + 1.56729 (QL/100) - 0.70239 (QL/100)2 1.035
7 0.16709 + 1.58245 (QL/100) - 0.68705 (QL/100)2 1.035
8 0.16394 + 1.55070 (QL/100) - 0.65270 (QL/100)2 1.040
9 0.11412 + 1.63532 (QL/100) - 0.68786 (QL/100)2 1.040
10 to 11 0.15344 +1.50104 (QL/100) - 0.58896 (QL/100)2 1.045
12 to 14 0.07278 + 1.64285 (QL/100) - 0.65033 (QL/100)2 1.045
15 to 18 0.07826 + 1.55649 (QL/100) - 0.56616 (QL/100)2 1.050
19 to 25 0.09907 + 1.43088 (QL/100) - 0.45550 (QL/100)2 1.050
26 to 37 0.07373 + 1.41851 (QL/100) - 0.41777 (QL/100)2 1.055
38 to 69 0.10586 + 1.26473 (QL/100) - 0.29660 (QL/100)2 1.055
70 to 200 0.21611 + 0.86111 (QL/100) 1.060
> 201 0.15221 + 0.92171 (QL/100) 1.060
If the PF is less than 0.75, the Engineer may:
1. Require complete removal and replacement with specification material at the
Contractor’s expense; or
47
2. Where the finished product is found to be capable of performing the intended
purpose and the value of the finished product is not affected, permit the Contractor to
leave the material in place.
CDOT uses weight factors in its composite pay equation. Table 3-2 shows the
weights CDOT uses for four AQCs. Because only two of four CDOT’s AQCs were
analyzed in this thesis, weights in composite pay equation had to be adjusted.
Table 5-2 “W” Factors for Various Elements HOT BITUMINOUS PAVEMENT
ELEMENT W FACTOR
Gradation 15
Asphalt Content 25
In-place Density 45
Joint Density 15
Since this thesis was focused on the asphalt contents and the in-place density, W
factors were recalculated at 0.357 for asphalt content and 0.643 for density to be
applied to data analysis.
48
CHAPTER 6 6 DATA ANALYSIS
6.1 Acceptance Quality Characteristics Data Analysis
6.1.1 Asphalt Content Data Analysis
CDOT’s report for asphalt content has start date, grading, region number,
project number, sample size, lot size, price per ton, quality level, pay factor,
Incentive/Disincentive Payment (I/DP), standard deviation, etc. From this report,
asphalt content data was resorted by sample sizes in 2000 year through 2007 year
to check quality levels in actual practice. All data were classified by 25 classes
(quality levels) and distribution of quality PWLs were calculated at each class
shown in Table 6-1.
Table 6-1 Asphalt Content Proportion
Class (PWL) Frequency Lot Numbers
Proportion (Percent)
0-4 1 0.16 4-8 0 0.00
8-12 0 0.00 12-16 0 0.00 16-20 0 0.00 20-24 2 0.32 24-28 0 0.00 28-32 1 0.16 32-36 2 0.32 36-40 1 0.16 40-44 3 0.48 44-48 4 0.65 48-52 7 1.13 52-56 7 1.13 56-60 7 1.13 60-64 8 1.29 64-68 13 2.10 68-72 21 3.39 72-76 20 3.23
49
Class (PWL) Frequency Lot Numbers
Proportion (Percent)
76-80 29 4.68 80-84 40 6.45 84-88 56 9.03 88-92 65 10.48 92-96 93 15.00 96-100 240 38.71
Total 620 100.00
Table 6-1 Continued Asphalt Content Proportion
From analysis of CDOT database, at least 50 percent of lots were estimated
to be better than AQL (about 85 PWL depending on pay equation). Based on the
CDOT data and supported by discussions with FHWA official, there is sufficient
reason to believe that most states have lot-quality distributions that are much better
than the distribution Gharaibeh assumed for his “good” category. The CDOT
database contains lots with n ranging from 3 to well over 100. The comparison
consisted of grouping lots by sample size and comparing PWL estimates shown in
Figure 6-1 and 6-2. (Kopac, 2010)
50
Figure 6-1. Proportion Histogram at each sample size category for Asphalt Contents I
Histogram (n=3)
0
10
20
30
40
50
60
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=4)
0
10
20
30
40
50
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=5)
0
10
20
30
40
50
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=6)
05
1015
2025
3035
4045
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=7)
0
5
10
15
20
25
30
35
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=8)
0
5
10
15
20
25
30
35
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
nHistogram (n=9)
0
510
1520
25
3035
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=10)
0
5
10
15
20
25
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=11)
0
5
10
15
20
25
30
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=12)
0
10
20
30
40
50
60
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (n=13)
0
5
10
15
20
25
30
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
Histogram (14<=n<=16)
0
5
10
15
20
25
30
35
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
51
Figure 6-2. Proportion Histogram at each sample size category for Asphalt Contents II
As sample sizes decrease, the spread of PWL estimates increases; thus, the
smaller n distributions contain not only more low-quality estimates but also more high-
quality estimates. This does not mean that the true quality is different, only that the
quality estimates have different distributions.
Figure 6-3 shows all data proportion versus quality level for asphalt contents. The
38.71 percent of lots range between 96 and 100 PWL.
Histogram (All Data)
0
5
10
15
20
25
30
35
40
45
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Pro
porti
on
Figure 6-3. Proportion Histogram for Asphalt Contents
Histogram (17<=n<=20)
0
5
10
15
20
25
30
35
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
nHistogram (n>20)
0
5
10
15
20
25
30
35
40
45
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Prop
ortio
n
52
6.1.2 In-Place Density Data Analysis
From CDOT database, In-place density of asphalt data was also categorized by 25
quality levels in actual practice. The distribution of quality PWLs were calculated at each
class shown in Table 6-2 and Figure 6-4.
Table 6-2 In-Place Density Proportion
Class (PWL)
Frequency (Lot
Numbers)
Proportion (Percent)
0-4 0 0.00 4-8 0 0.00 8-12 0 0.00
12-16 0 0.00 16-20 0 0.00 20-24 0 0.00 24-28 0 0.00 28-32 0 0.00 32-36 2 0.33 36-40 1 0.16 40-44 0 0.00 44-48 0 0.00 48-52 1 0.16 52-56 2 0.33 56-60 4 0.65 60-64 4 0.65 64-68 5 0.82 68-72 10 1.64 72-76 13 2.13 76-80 28 4.58 80-84 37 6.06 84-88 53 8.67 88-92 86 14.08 92-96 111 18.17
96-100 254 41.57
Total 611 100.00
The distribution of In-Place density data had a similar tendency as the one of
asphalt contents. The 82.49 percent of lots were estimated to be better than AQL (85
PWL).
53
The anticipated quality distribution of incoming lots is an important variable in
determining optimum n since the highway agency is using its acceptance system to
make acceptance decisions on all incoming lots, not just on a lot of a given quality level
(not on a typical lot, for example). If incoming lots are anticipated to be generally high
quality (i.e., good means, low standard deviations), less testing is needed. This is
because the joint probability of a contractor delivering and the agency accepting a poor-
quality lot is smaller when most incoming lots are of high quality; the expected cost of
erroneous acceptance decisions is smaller for high-quality lots as shown in Figure 3-1
model.
Histogram (All Data)
0
5
10
15
20
25
30
35
40
45
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Pro
porti
on
Figure 6-4. Proportion Histogram for In-Place Density
6.1.3 Multi Characteristics Analysis
It is important to understand that while an agency can apply the general model
either a single AQC (a plan) or to multiple AQCs (a system of plans), true optimization
can occur only when the agency applies it to the whole system rather than just to a
54
component of the system. Most, if not all, highway agencies have multi-AQC systems.
CDOT has also multi-AQCs such as asphalt contents, in-place density, gradation, joint
density, etc. in the HMA. For this study, asphalt contents and in-place density AQCs
were chosen to determine the required sample sizes to correspond to the desirable
level of risk. These AQCs were also efficiency to explain correlation as a view of AQC
lot quality measures (e.g., lot PWLs). Based on CDOT database, for these AQCs, only
lots (528 lots) that had same mix design number were selected to concern muti-AQC
system risks.
6.2 Correlation Between Two AQCs
In previous research, optimization model assumed AQCs were independent or
weakly dependent. The assumption seemed logical considering highway agencies are
generally discouraged from using correlated AQCs (Burati, 2005). Even when agencies
do use correlated AQCs, contractors tend to pay attention to individual AQC quality
rather than to combined measures of quality; if one AQC is poor-quality, it does not
necessarily indicate other AQCs are also poor-quality (Gharaibeh, 2010).
In this study, the effect of correlated AQCs on optimum n was investigated. The
study showed first, that for purposes of the optimization, the issue is not whether the
AQC test results are correlated but whether the AQC lot quality measures (e.g., lot
PWLs) are correlated. From the CDOT data, asphalt content and in-place density
AQCs were organized with same physical lots and then used to determine the degree of
correlation that might be expected between AQC lot PWLs, and the effect of AQC lot
PWL correlation on optimum n.
The plot of density lot PWLs versus corresponding asphalt content lot PWLs were
shown in Figure 6-5. Each data point represents a pair of PWLs that came from the
55
same physical lot. In-Place density and asphalt content were sampled at different
frequencies, but the average sample size (average n = 24 for density and average n =
13 for asphalt content) is large enough that the PWL estimates can be considered to be
fair estimates of the population PWLs.
Figure 6-5 is typical of the plots one obtains with the various combinations of
CDOT AQC lot PWLs. Data points are clustered at the upper right, indicating very high
lot quality; and they also tend toward either axis, indicating a lack of correlation. Where
R2 is the correlation coefficient, y is the trendline of the correlation. Because
conventional correlation techniques should not be used to determine the degree of
correlation between non-normally distributed variables whose values contain testing
error and are “top-heavy,” a procedure to calculate a new measure of correlation, the
“relative correlation ratio,” was developed as an aside in this study specifically for use in
determining the degree of correlation between AQC lot PWLs. This procedure
compares the plot from actual PWL data (e.g., Figure 6-5) against a plot using the same
PWL data but where the PWL pairs are assigned randomly, thus representing
independence (i.e., Figure 6-6). Because data points are clustered at the upper right,
one can count data points which are less or equal than 80 PWL. In figure 6-6, 21 data
points were at less than 80 PWL, on the other hands, in Figure 6-6, 9 data points were
counted for both 2 figures. The relative correlation ratio was developed as 2.333 (21/9)
for both cases. As this ratio is closer to 1, one can say two AQC lots are more strongly
correlated.
56
y = 0.0915x + 83.492
R2 = 0.0222
0
20
40
60
80
100
0 20 40 60 80 100
Asphalt Content PWL
Den
sity
PW
L
Figure 6-5. Plot of Density and Asphalt Content Lot PWL Pairs
57
y = -0.0398x + 95.036
R2 = 0.0042
0
20
40
60
80
100
0 20 40 60 80 100
Asphalt Content PWL
Ran
dom
Den
sity
PW
L
y = -0.0354x + 94.656
R2 = 0.0033
0
20
40
60
80
100
0 20 40 60 80 100
Random Asphalt Content PWL
Den
sity
PW
L
Figure 6-6. Plots of Randomly Assigned Density and Asphalt Content Lot PWL Pairs
58
The degree of correlation was found to have a negligible effect on the total
optimum n. For the degree of correlation in Figure 6-5 (where the plot exhibited mild
correlation using the relative correlation ratio), the optimization procedure arrived at the
same optimum n = 3 for each AQC (total n = 6) as that arrived under an assumption of
independence. Even if the density and asphalt content lot PWLs were strongly
correlated, the optimization procedure still arrives at an optimum n = 3 for each AQC.
However, according to the optimization procedure (and logic), if the total optimum n for
two perfectly correlated AQCs is 6, the lowest cost to the agency occurs when n = 6 for
the AQC having the lower sampling and testing and n = 0 for the AQC having the higher
sampling and testing cost. Thus, unless the agency “needs” to use both AQCs, as it
might when each AQC controls different key distresses, the recommended course of
action to achieve optimization is for the agency to drop the higher cost AQC from its
acceptance system. The optimization procedure thus provides an economic argument
against the use of highly correlated AQCs.
6.3 Pay Adjustments
The theory behind acceptance plan development that was initially presented to the
highway community in the 60’s dealt strictly with accept/reject acceptance plans
(Military Standard 414, 1957, Development of Guidelines for Practical and Realistic
Construction Specifications, 1967). In the intervening years, the theory had to be
expanded as state highway agencies adopted first pay decreases (penalties) then also
pay increases (bonuses) associated with different levels of estimated quality. Although
the vast majority of acceptance plan systems now contain pay adjustment provisions,
there are still gaps in our understanding of pay adjustment development and function.
59
In the previous research, Gharaibeh assumed that the cost consequences of
erroneous pay decisions were such that, in the long run, the positive cost consequences
incurred when the agency underestimates quality and pays less than it should cancel
out the negative cost consequences incurred when it overestimates quality and pays
more than it should; in other words, he essentially treated pay-adjustment acceptance
plans as if they were accept/reject acceptance plans.
6.3.1 CDOT Data with Agency’s Desired Pay Factor
To calculate the cost of erroneous pay decisions in the this study, it was assumed
that the agency’s desired pay factor for a lot having any combination of AQC PWLs is,
by definition, whatever the agency wants it to be for that particular AQC PWL
combination. Obviously, what the agency wants it to be is spelled out in the agency pay
equations. The CDOT pay equations (which are a function of sample size, see Table 5-
1) were thus used to determine the agency’s desired multi-AQC pay. The SpecRisk
software program was used to determine the contractor’s multi-AQC expected pay.
The difference between the contractor’s expected pay and the agency’s desired pay is
the cost of erroneous pay decisions. Table 6-3 shows classified AQCs PWL CDOT data
with pay factors. The agency’s desired multi-AQC pay factor is calculated in Table 6-4.
60
Table 6-3 Classified AQCs PWL Data with Pay Factor
Class Representative PWL AC. Proportion Den. Proportion
Pay Factor (n=12~14),
1.045
Pay Factor (n=15~18),
1.050
Pay Factor (n=19~25),
1.050
Pay Factor (n=3), 1.025
Class 1 30 PWL 0-45 PWL 2.08% 0-45 PWL 0.19% 0.50711 0.49425 0.48734 0.70903 Class 2 58 PWL 45-65 PWL 5.68% 45-65 PWL 1.70% 0.80686 0.79057 0.77575 0.94199 Class 3 75 PWL 65-80 PWL 12.88% 65-80 PWL 8.52% 0.93911 0.92716 0.91601 1.01851 Class 4 87 PWL 80-90 PWL 20.83% 80-90 PWL 22.92% 1.00982 1.00388 0.99917 1.04299 Class 5 97 PWL 90-100 PWL 58.52% 90-100 PWL 66.67% 1.05445 1.05536 1.05844 1.04472
61
Table 6-4 Agency’s Desired Pay Factor for A Lot Having any Combination of AQC PWLs
Possible Events
Desired Pay Decision
reject or pay PF, n=3 Maximum PF (1.025) PF, n=15~18 Maximum PF
(1.050)
Combined PF (AQC1:n=12~14, AQC2:n=19~25)
Maximum PF (1.048215)
AQC1 in class 1 and AQC2 in class 1 reject or pay 0.70903 0.70903 0.49425 0.49425 0.49440 0.49440 AQC1 in class 1 and AQC2 in class 2 reject or pay 0.85882 0.85882 0.68478 0.68478 0.67984 0.67984 AQC1 in class 1 and AQC2 in class 3 reject or pay 0.90802 0.90802 0.77261 0.77261 0.77003 0.77003 AQC1 in class 1 and AQC2 in class 4 reject or pay 0.92376 0.92376 0.82194 0.82194 0.82350 0.82350 AQC1 in class 1 and AQC2 in class 5 reject or pay 0.92488 0.92488 0.85504 0.85504 0.86162 0.86162 AQC1 in class 2 and AQC2 in class 1 reject or pay 0.79219 0.79219 0.60004 0.60004 0.60141 0.60141 AQC1 in class 2 and AQC2 in class 2 pay 0.94199 0.94199 0.79057 0.79057 0.78686 0.78686 AQC1 in class 2 and AQC2 in class 3 pay 0.99119 0.99119 0.87840 0.87840 0.87704 0.87704 AQC1 in class 2 and AQC2 in class 4 pay 1.00693 1.00693 0.92773 0.92773 0.93051 0.93051 AQC1 in class 2 and AQC2 in class 5 pay 1.00804 1.00804 0.96083 0.96083 0.96863 0.96863 AQC1 in class 3 and AQC2 in class 1 reject or pay 0.81951 0.81951 0.64880 0.64880 0.64862 0.64862 AQC1 in class 3 and AQC2 in class 2 pay 0.96930 0.96930 0.83933 0.83933 0.83407 0.83407 AQC1 in class 3 and AQC2 in class 3 pay 1.01851 1.01851 0.92716 0.92716 0.92426 0.92426 AQC1 in class 3 and AQC2 in class 4 pay 1.03425 1.02500 0.97649 0.97649 0.97773 0.97773 AQC1 in class 3 and AQC2 in class 5 pay 1.03536 1.02500 1.00959 1.00959 1.01584 1.01584 AQC1 in class 4 and AQC2 in class 1 reject or pay 0.82825 0.82825 0.67619 0.67619 0.67387 0.67387 AQC1 in class 4 and AQC2 in class 2 pay 0.97804 0.97804 0.86672 0.86672 0.85931 0.85931 AQC1 in class 4 and AQC2 in class 3 pay 1.02725 1.02500 0.95455 0.95455 0.94950 0.94950 AQC1 in class 4 and AQC2 in class 4 pay 1.04299 1.02500 1.00388 1.00388 1.00297 1.00297 AQC1 in class 4 and AQC2 in class 5 pay 1.04410 1.02500 1.03698 1.03698 1.04109 1.04109 AQC1 in class 5 and AQC2 in class 1 reject or pay 0.82887 0.82887 0.69457 0.69457 0.68980 0.68980 AQC1 in class 5 and AQC2 in class 2 pay 0.97866 0.97866 0.88510 0.88510 0.87525 0.87525 AQC1 in class 5 and AQC2 in class 3 pay 1.02786 1.02500 0.97293 0.97293 0.96543 0.96543 AQC1 in class 5 and AQC2 in class 4 pay 1.04361 1.02500 1.02226 1.02226 1.01890 1.01890 AQC1 in class 5 and AQC2 in class 5 pay 1.04472 1.02500 1.05536 1.05000 1.05702 1.04822
62
6.3.2 Contractor’s Expected Pay Factor
The SpecRisk software program was used to determine the contractor’s multi-
AQC expected pay. Figure 6-7 shows pay equation curves for asphalt content and in –
place density as graphical input values.
Figure 6-7. Pay Equation Curves for Asphalt Content and Density in SpecRisk software
63
Figure 6-8. Pay Equation Curves for Asphalt Content and Density with a low threshold
To compare the estimated PWL pay curve and the expected pay curve, pay
equation curve with low thresholds were considered shown in Figure 6-8. These pay
curves both have a minimum acceptable estimated quality level of 45 PWL.
6.3.3 Pay Adjustment Acceptance Plan
In this study, the previous assumption (Gharaibeh, 2010) of zero cost of erroneous
pay decisions was found to be incorrect as is shown in Table 6-5.
64
Table 6-5 Difference Between Agency’s Desired Pay and Contractor’s Expected Pay for n=3
Colorado A.C PWL
Colorado Den. PWL Expected Pay Desired Pay
Difference between
E.P and D.P 30 30 0.46375 0.49440 -0.03065 30 58 0.63796 0.67984 -0.04189 30 75 0.74035 0.77003 -0.02968 30 87 0.80059 0.82350 -0.02291 30 97 0.83396 0.86162 -0.02765 58 30 0.56520 0.60141 -0.03621 58 58 0.74728 0.78686 -0.03958 58 75 0.84679 0.87704 -0.03026 58 87 0.89846 0.93051 -0.03206 58 97 0.94456 0.96863 -0.02407 75 30 0.61977 0.64862 -0.02885 75 58 0.80878 0.83407 -0.02529 75 75 0.89234 0.92426 -0.03192 75 87 0.94434 0.97773 -0.03338 75 97 0.98748 1.01584 -0.02836 87 30 0.64651 0.67387 -0.02736 87 58 0.82445 0.85931 -0.03486 87 75 0.91494 0.94950 -0.03456 87 87 0.97156 1.00297 -0.03141 87 97 1.01572 1.04109 -0.02537 97 30 0.66922 0.68980 -0.02058 97 58 0.84108 0.87525 -0.03416 97 75 0.93830 0.96543 -0.02714 97 87 0.99119 1.01890 -0.02772 97 97 1.03339 1.04822 -0.01483
As pay adjustment acceptance plans were not designed to make erroneous pay
decision cost consequences cancel each other, it would be coincidental for that to
happen.
This study showed that the net consequences of erroneous pay decisions not only
could be considerable, but that they tended to favor the highway agency, i.e., to
underpay contractors. When the agency uses an acceptance plan system that in the
65
long run underpays contractors, the agency’s expected cost of erroneous acceptance
decision (hence, the agency’s total cost of lot acceptance) is decreased.
Figure 6-9 and 6-10, developed from CDOT data, can be used as examples to
illustrate such a situation. In Figure 6-9, the expected cost of erroneous acceptance
decisions is negative throughout the range of n; the smaller the n, the greater the
probability of erroneous acceptance decisions, thus the more negative the expected
cost, i.e., the more the contractor is underpaid. These negative costs bring the
optimization model’s “total cost of lot acceptance” curve below the “cost of sampling and
testing” curve. With respect to optimum n, it is n = 3, which is the lowest possible n for
the PWL quality measure.
($15)
($10)
($5)
$0
$5
$10
$15
3 8 13 18 23
Sample Size, n
Expe
cted
Cos
t/Lo
t (Th
ousa
nd U
SD)
Expected Cost of Erroneous Acceptance Dec is ion
Cost of Sampling and Testing
Total Cost of Lot Acceptance
Figure 6-9. Application of optimization model with negative agency acceptance costs
66
Sample Size vs . Cost (With Threshold)
($15)
($10)
($5)
$0
$5
$10
$15
3 8 13 18 23
Sample Size, n
Expe
cted
Cos
t (Th
ousa
nd U
SD)
Expected Cost of Erroneous Acceptance Dec is ion
Cost of Sampling and Testing
Total Cost of Lot Acceptance (With Threshold)
Figure 6-10. Application of optimization model with negative agency acceptance costs (with Threshold)
Two variables come into play for the situation depicted in Figure 6-9 and 6-10 to
occur—the lot-quality distribution and the pay adjustment provisions. Figure 6-11 can
provide a better understanding. It compares the composite pay equation with its
expected pay curve for a two-AQC acceptance system (with the X-axis identifying the
same PWL for both AQCs (Asphalt Contents and In-Place Density)). Note that the
composite pay equation has both a minimum acceptable estimated quality level (45
PWL) and a maximum pay factor cap (102.50 PWL). When the lot-quality distribution is
such that delivered lots rarely fall below 45 PWL, the portion of the graph to the right of
45 PWL carries a much greater weight. One can thus see that under such conditions,
unless one or two specific contractors are consistently delivering the few instances of
67
below 45 PWL lots, contractors as a whole are in the long run underpaid in comparison
to agency-desired pay.
Of the two variables, the lot-quality distribution appears to be the most influential.
Application of the optimization model shows that the better the lot-quality distribution,
the greater the likelihood of negative costs and associated long-run underpayment to
the contractors. Further, the lot-quality levels need not be all that high for the long-run
underpayment to occur. It can occur with Gharaibeh’s “regular” and “good” historical
quality distribution, whether or not the pay provisions include a maximum pay cap
and/or a minimum acceptable estimated quality level (below which lots may be
rejected), although use of the latter decreases the long-run underpayment.
0.00
20.00
40.00
60.00
80.00
100.00
120.00
0 10 20 30 40 50 60 70 80 90 100
Estimated and True PWL
Com
posi
te P
ay F
acto
r
Pay Eq. (Estimated PWL)
Expected Pay Eq. (True PWL)
Figure 6-11. Comparison of a composite pay equation and its expected pay curve
68
It is also important to note that sample size is not causing the underpayment.
Sample size simply affects the risks associated with contractor being assessed the
correct payment. For high-quality lots, the risk of an occasional incorrect, low-quality
estimate is greater with small sample sizes, thus resulting in the pay equation bias that
leads to a greater expected underpayment. The underpayment should not be an issue,
provided the agency has developed, is satisfied with, and has made available to
contractors, the expected pay equation.
6.4 Cost of Sampling and Testing
The cost of sampling and testing is difficult to model because there are many
different possible cost scenarios: a technician could perform two or more AQC tests
simultaneously; a technician could perform another function while waiting for test
results; one sample could yield several AQC test results; multiple technicians could be
employed and be more (or less) efficient than a single technician; etc. In this study, a
linear relationship (Gharaibeh, 2010) between the cost of sampling and testing and
sample size n was used. This example explains how Figures 6-12 and 6-13 were
developed based on the cost assumptions presented earlier. The cost of sampling and
testing of 5 HMA cores for density is computed as follows:
Transportation between lab and project site ..................................................$115.1
Coring ($99.6/core).........................................................................................$498.1
Max. Theoretical Specific Gravity test ($52.3/core) .......................................$261.5
Density test ($26.2/core).................................................................................$130.8
Total (for 5 cores) ........................................................................................$1,005.5
69
Figure 6-12. National Average Cost of Sampling and Testing for unmolded HMA (Binder Content and Gradation together) (2008 Prices)
Figure 6-13. National Average Cost of Sampling and Testing for HMA Density Cores (2008 Prices)
As these examples, If a single assumption has to be made about the cost of
sampling and testing element in a generalized optimization model, it would be hard to
argue that linearity should not be the assumption.
The assumption of linearity tends to “average” all the possible cost scenarios that
exist within the state and/or within the acceptance plan system. However, if an agency
70
believes the cost of sampling and testing element to be other than linear for its
optimization purposes, the agency can substitute its own specific model in performing
the necessary calculations.
In any case, agencies should also consider stepped functions. A stepped cost of
sampling and testing function is appropriate in a situation where the cost of performing
additional tests is viewed as negligible up to a certain n, at which point it suddenly
increases (Kopac, 2010). One such example is the cost of nuclear density testing.
Under certain situations, the cost of nuclear density testing could stay about the same
whether say, 3 tests or 10 tests, are performed in one day or by one technician. The
cost could then noticeably increase if n = 11 tests required two technicians or extended
the time on the job for one technician from say, 1 day to 2 days. With a stepped
function, the best optimization solution is for the agency to accept n = 10 as the
optimum in this example rather than the calculated n = 3.
Another assumption made by Gharaibeh dealt with the unit costs of sampling and
testing. As indicated earlier, the higher the unit costs, the larger the optimum n. The
unit costs Gharaibeh used represent national averages and are deemed to be
sufficiently conservative. In this study, unit bid price was calculated by only CDOT data.
State highway agencies that suspect their unit costs are higher can easily input their
own costs.
6.5 Post-Construction Costs
Previous optimization modeling (Gharaibeh, 2010) assumed the agency’s
expected cost due to erroneous acceptance decisions is based solely on bid price and
not any other costs such as user costs or maintenance and rehabilitation costs. This
71
assumption may be valid for accept/reject acceptance plan systems for which there is a
clear line between rejectable and acceptable lots and no need to distinguish among
various levels of acceptable quality. This assumption, however, has no place in pay
adjustment acceptance plan systems.
In this study, optimization modeling by its nature distinguishes among various
levels of acceptable quality (and also among various levels of unacceptable quality). It
indirectly considers user costs and maintenance and rehabilitation costs. These post-
construction costs relate to the expected underpayment/overpayment to contractors as
a result of erroneous pay decisions.
As stated earlier, this study found that the high-quality (above AQL) lots typically
delivered to state highway agencies along with the caps placed on the maximum pay
factor are responsible for the long-run underpayment to the contractors. Including user
costs and maintenance and rehabilitation costs in the modeling significantly raises the
performance-related pay increase the contractor “deserves” to have for above AQL lots.
PaveSpec performance-related specifications (PRS) software shows that the inclusion
of post-construction costs, even when using only 5 percent of the theoretical user cost,
can result in very high “deserved” pay factors (FHWA-RD-00-131, 2000). That is partly
why 100 percent of user costs is not used in PRS development, and why a cap is
placed on the maximum PRS pay factor (Kopac, 2010). Both have the effect of
underpaying the contractors, i.e., of creating negative costs to the agency that in turn
decrease the calculated optimum n (below that optimum n calculated without post-
construction costs).
72
6.6 Cost of Contractor Reaction
This study considered a third cost element separate from the two elements
(Expected Cost of Erroneous Acceptance Decision and Cost of Sampling and Testing)
identified in Figure 3-1. It addressed the impact on lot quality and cost if an agency
switches to a smaller n than the one it currently uses.
Preliminary indications are that with proper precautions there should be little if any
difference in delivered (and accepted) quality. Contractors have already delivered many
lots to various state highway agencies knowing that the acceptance sample size will be
as small as n = 3 or 4 (Kopac, 2010).
CDOT’s database was used to make a comparison of hot mix asphalt quality
delivered when the contractor knows n will be small. The CDOT data base contains lots
with n ranging from 3 to well over 100. The comparison consisted of grouping lots by
sample size and comparing PWL estimates. To allow a fair comparison, those lots that
were meant to have higher n but had been prematurely discontinued due to
unscheduled mix design changes were eliminated from the data.
An example comparison, using the n = 3, 9, and 17-20 groups, is shown in Figure
6-14. As one would expect, the spread of PWL estimates decreases as n increases;
thus, the smaller n distributions contain not only more low-quality estimates but also
more high-quality estimates. This does not mean that the true quality is different, only
that the quality estimates have different distributions. The average PWL of the
distribution is the best measure of true quality in this case, and the three average PWLs
are about the same—91.72, 90.23, and 91.87 for n =3, 9, and 17-20 respectively. The
nonparametric Mann-Whitney U test shows no difference among the three population
means at α = .05 significance level.
73
An argument can even be made that, for some AQCs, quality increases with
decreased n. A frequently cited example involves concrete compressive strength under
an acceptance plan that uses the average as the quality measure (Kopac, 2010). With
smaller n, a contractor would need to target a higher average compressive strength in
order to meet the minimum average estimated strength requirement with the same
probability as for larger n. This argument holds not only for compressive strength but
also other one-sided AQCs such as thickness, density and smoothness; and other
quality measures such as PWL.
In Figure 6-14, it is the spread of the distributions (the error associated with the
estimated PWLs) that would motivate a contractor to increase quality for either one-
sided or two-sided AQCs. For two-sided AQCs, rather than increasing (improving) the
lot average, the contractor would want to decrease variability. Either way, the
contractor’s costs theoretically would increase, but the post construction costs would
probably decrease.
Histogram (n=3)
0
10
20
30
40
50
60
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Pro
porti
on
N=46 Avg. PWL = 91.72
74
Histogram (n=9)
0
5
10
15
20
25
30
35
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Pro
porti
on
Histogram (17<=n<=20)
0
5
10
15
20
25
30
35
40
0 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96
PWL
Pro
porti
on
Figure 6-14. Distribution of CDOT asphalt content PWL estimates for lots with n=3, 9, and 17-20
N=32 Avg. PWL = 90.23
N=45 Avg. PWL = 91.87
75
CHAPTER 7 7 CONCLUSIONS RECOMMENDATIONS
7.1 Summary and Conclusions
This study concluded that previous optimization model (Gharaibeh, 2010)
assumptions were indeed conservative. With modified assumptions, based where
possible on actual data, the optimum acceptance sample size n is even smaller than
that calculated by Gharaibeh. However sample size n cannot be less than 3 for PWL
acceptance plans as PWL estimates cannot be made with less than 3 test results.
For acceptance plan systems related to pavements, optimum n is primarily a
function of incoming lot quality—the higher the quality the lower the n. To illustrate with
an extreme example, if an agency was certain that incoming lots will be of such high
quality that none should be rejected, there would be no need for the agency to do any
testing under an accept/reject acceptance system. The term “acceptance system”
however, has become somewhat of a misnomer. For many highway agencies, it is
more a “pay adjustment system,” i.e., more for the purpose of making pay adjustment
decisions rather than accept/reject decisions. This study examined pay adjustment
acceptance systems and concluded for them too, optimum n is primarily a function of
incoming lot quality. When incoming lot quality is anticipated to be high, the expected
cost associated with erroneous pay decisions is frequently negative (i.e., contractors are
underpaid in the long run), especially in situations where the agency has no minimum
acceptable estimated quality level provision (or it has a provision, but with a low
minimum).
For acceptance plan systems related to other than pavements (e.g., bridge decks),
it should be noted that optimum n may also be influenced considerably by the cost
76
associated with erroneous accept/reject decisions. If the consequences of erroneous
accept/reject decisions are catastrophic and could result in loss of lives, the optimum n
could be much higher than 3. Both previous research and this study reported here
considered only pavement acceptance plan systems.
Optimum n is of course also a function of the AQCs being measured—what are
they? how many are there? are they correlated? and what is the cost to sample and test
them? The number of AQCs is important as they work as a unit to estimate lot quality
which then determines the composite pay factor. Because the expected cost of
erroneous pay decisions is spread out among the AQCs, each AQC’s contribution to the
expected cost decreases as the number of AQCs increases, resulting in a smaller
optimum n for each AQC. This study investigated systems with only two AQCs, and
already optimum n was below 3. Correlated AQCs were not found to change the
optimum n = 3, conclusion for PWL, provided the agency needed the correlated AQCs
within the acceptance system in order to control different key distresses. The unit costs
of sampling and testing derived by previous research (Gharaibeh, 2010) were deemed
to be conservative and were used in this study as well.
This study concluded statewide lot quality (and therefore lot performance) is not
likely to suffer if agencies that switch to smaller acceptance sample sizes take proper
precautions. Some recommendations are provided below in the recommendations
section. There is also reason to believe quality might actually increase for some one-
sided AQCs, as statistical-risk-aware contractors tend to raise target quality to account
for the increased variability of estimates from small sample sizes. In such cases, to
keep from having a corresponding increase in the cost of lots, agencies that expect
77
contractors to raise their quality levels might consider simply lowering the specified
quality level. However, it is doubtful that any increase in costs due to such increases in
quality would change the optimum n. For agencies that believe quality levels will
decrease, the simple solution would be to raise the specified quality level.
7.2 Recommendations
It is strongly recommended agencies do their own optimum n determinations. In
doing so, they will gain a better understanding of their acceptance plan systems and
associated costs, and just as important, of the various underlying assumptions and their
effect on optimum n. Having this understanding, they will be in a better position to draw
their own conclusions from the performed economic decision analysis, which identifies
the best decision in the long run and not necessarily the best decision with respect to a
specific project or contractor or submitted quality level. The understanding will also
provide the agencies greater confidence in applying economic decision analysis
principles to minimize expected costs and optimize statistical risks.
Once an agency has followed the optimization procedure and determined optimum
n for its AQCs, the agency will have simultaneously identified the optimum buyer’s and
seller’s risks (as these statistical risks are a function of n). It is also possible for the
agency to determine the theoretical optimum lot size since lot size affects the expected
cost of erroneous acceptance decisions (an element of the optimization model).
Assuming the agency is considering switching to a lower n based on its
optimization, the following recommendations are offered:
• In going to less acceptance testing, the agency should consider (a) placing more
emphasis on the contractors’ quality control programs, (b) placing more
emphasis on inspection to identify isolated instances of poor quality, and/or (c)
78
replacing programs that use contractor test results for acceptance purposes with
programs that use only the agency’s test results.
• The agency should include a strong retest provision in each acceptance plan to
further test lots that yield borderline test results.
• If the agency can identify specific contractor(s) with a record of having submitted
low quality levels in the past, the agency should consider taking appropriate
action. The agency has many options, one of which is the use of an acceptance
plan system with same low n but greater pay reductions for the specific
contractor(s).
• The agency should monitor how the lower n is working statewide with respect to
quality and cost. Here too, the agency has many options that allow it to control
overall quality and cost; simply increasing/decreasing specified quality is one
such option.
79
APPENDIX A A PWL QUALITY MEASURE
If the quality characteristic is to be used for payment determination, the quality
measure to be related to the payment must be decided upon. There are several quality
measures that can be used. In past acceptance plans, the average, or the average
deviation from a target value, was often used as the quality measure. However, the use
of the average alone provides no measure of variability, and it is now recognized that
variability is often an important predictor of performance.
Several quality measures, including percent defective (PD) and percent within
limits (PWL), have been preferred in recent years because they simultaneously
measure both the average level and the variability in a statistically efficient way. In this
appendix, only the PWL quality measure was described for the purpose of this study.
The Transportation Research Board (TRB) glossary (TRC-E-C074, 2005) includes
the following definition (where LSL and USL represent lower and upper specification
limits, respectively):
• PWL-also called percent conforming.
The percentage of the lot falling above the LSL, beneath the USL, or between the
USL and LSL (PWL may refer to either the population value or the sample estimate
of the population value. PWL = 100 - PD.)
This quality measure uses the sample mean and the sample standard deviation to
estimate the percentage of the population (lot) that is within the specification limits. This
is called the PWL method, and is similar in concept to determining the area under the
normal curve.
80
In theory, the use of the PWL (or PD) method assumes that the population being
sampled is normally distributed. In practice, it has been found that statistical estimates
of quality are reasonably accurate provided the sampled population is at least
approximately normal, i.e., reasonably bell shaped and not bimodal or highly skewed.
A.1 Estimating PWL
Conceptually, the PWL procedure is based on the normal distribution. The area
under the normal curve can be calculated to determine the percentage of the population
that is within certain limits. Similarly, the percentage of the lot that is within the
specification limits can be estimated. Instead of using the Z-value and the standard
normal curve, a similar statistic, the quality index, Q, is used to estimate PWL. The Q-
value is used with a PWL table to determine the estimated PWL for the lot.
A sample PWL table is shown in Table A-1. A different format for a table relating Q
values with the appropriate PWL estimate is shown for a sample size of n = 5 in Table
A-2. A more complete set of PWL tables in this format, for sample sizes from n = 3 to n
= 30, is available. Another way of relating Q and PWL values is presented in Table A-3.
In this table a range of Q values is associated with each PWL value. This table was
developed by an agency such that any estimated PWL is rounded up to the next integer
PWL value. Other possible rounding rules could be used to develop similar tables. The
rounding rule in Table A-3 is the one that is most favorable to the contractor since it
rounds any PWL number up to the next whole number. For example, 89.01 is rounded
up to 90.00 in Table A-3.
A.2 Calculation and Rounding Procedures
As the previous paragraph illustrates, the calculation procedures and rounding
rules can influence the estimated PWL value that is obtained. This can become a point
81
of contention, particularly if the payment determination is based on the estimated PWL
value. It is therefore important that the agency stipulate the specific calculation process,
including number of decimal places to be carried in the calculations, as well as the exact
manner in which the PWL table is to be used.
For example, in Table A-1, is the PWL value to be selected by rounding up,
rounding down, or by linear interpolation. Each of these will result in a different
estimated PWL value. For instance, if the sample size is n = 5, and the calculated Q
value is 1.18, the estimated PWL values for rounding up, rounding down, and
interpolating would be 89, 88, and 88.5, respectively.
A.3 Quality Index and PWL
The Z-statistic that is used with the standard normal table, an example of which is
shown in Table A-4, uses the population mean as the point of reference from which the
area under the curve is measured:
Where:
Z = the Z-statistic to be used with a standard normal table (such as Table A-4).
X = the point within which the area under the curve is desired.
µ = the population mean.
s = the population standard deviation.
The statistic Z, therefore, measures distance above or below the mean, µ, using
the number of standard deviation units, s, as the measurement scale. This is illustrated
in Figure A-1.
82
Table A-1 Quality Index Values for Estimating PWL I
PWL n = 3 n = 4 n = 5 n = 6 n = 7 n = 8 n = 9 n = 10 to 11
100 1.16 1.5 1.79 2.03 2.23 2.39 2.53 2.65
99 - 1.47 1.67 1.8 1.89 1.95 2 2.04
98 1.15 1.44 1.6 1.7 1.76 1.81 1.84 1.86
97 - 1.41 1.54 1.62 1.67 1.7 1.72 1.74
96 1.14 1.38 1.49 1.55 1.59 1.61 1.63 1.65
95 - 1.35 1.44 1.49 1.52 1.54 1.55 1.56
94 1.13 1.32 1.39 1.43 1.46 1.47 1.48 1.49
93 - 1.29 1.35 1.38 1.4 1.41 1.42 1.43
92 1.12 1.26 1.31 1.33 1.35 1.36 1.36 1.37
91 1.11 1.23 1.27 1.29 1.3 1.3 1.31 1.31
90 1.1 1.2 1.23 1.24 1.25 1.25 1.26 1.26
89 1.09 1.17 1.19 1.2 1.2 1.21 1.21 1.21
88 1.07 1.14 1.15 1.16 1.16 1.16 1.16 1.17
87 1.06 1.11 1.12 1.12 1.12 1.12 1.12 1.12
86 1.04 1.08 1.08 1.08 1.08 1.08 1.08 1.08
85 1.03 1.05 1.05 1.04 1.04 1.04 1.04 1.04
84 1.01 1.02 1.01 1.01 1 1 1 1
83 1 0.99 0.98 0.97 0.97 0.96 0.96 0.96
82 0.97 0.96 0.95 0.94 0.93 0.93 0.93 0.92
81 0.96 0.93 0.91 0.9 0.9 0.89 0.89 0.89
80 0.93 0.9 0.88 0.87 0.86 0.86 0.86 0.85
79 0.91 0.87 0.85 0.84 0.83 0.82 0.82 0.82
78 0.89 0.84 0.82 0.8 0.8 0.79 0.79 0.79
77 0.87 0.81 0.78 0.77 0.76 0.76 0.76 0.75
76 0.84 0.78 0.75 0.74 0.73 0.73 0.72 0.72
75 0.82 0.75 0.72 0.71 0.7 0.7 0.69 0.69
74 0.79 0.72 0.69 0.68 0.67 0.66 0.66 0.66
73 0.76 0.69 0.66 0.65 0.64 0.63 0.63 0.63
72 0.74 0.66 0.63 0.62 0.61 0.6 0.6 0.6
71 0.71 0.63 0.6 0.59 0.58 0.57 0.57 0.57
70 0.68 0.6 0.57 0.56 0.55 0.55 0.54 0.54
69 0.65 0.57 0.54 0.53 0.52 0.52 0.51 0.51
68 0.62 0.54 0.51 0.5 0.49 0.49 0.48 0.48
67 0.59 0.51 0.47 0.47 0.46 0.46 0.46 0.45
83
PWL n = 3 n = 4 n = 5 n = 6 n = 7 n = 8 n = 9 n = 10 to 11
65 0.52 0.45 0.43 0.41 0.41 0.4 0.4 0.4
64 0.49 0.42 0.4 0.39 0.38 0.38 0.37 0.37
63 0.46 0.39 0.37 0.36 0.35 0.35 0.35 0.34
62 0.43 0.36 0.34 0.33 0.32 0.32 0.32 0.32
61 0.39 0.33 0.31 0.3 0.3 0.29 0.29 0.29
60 0.36 0.3 0.28 0.27 0.27 0.27 0.26 0.26
59 0.32 0.27 0.25 0.25 0.24 0.24 0.24 0.24
58 0.29 0.24 0.23 0.22 0.21 0.21 0.21 0.21
56 0.22 0.18 0.17 0.16 0.16 0.16 0.16 0.16
55 0.18 0.15 0.14 0.14 0.13 0.13 0.13 0.13
54 0.14 0.12 0.11 0.11 0.11 0.11 0.1 0.1
53 0.11 0.09 0.08 0.08 0.08 0.08 0.08 0.08
52 0.07 0.06 0.06 0.05 0.05 0.05 0.05 0.05
51 0.04 0.03 0.03 0.03 0.03 0.03 0.03 0.03
50 0 0 0 0 0 0 0 0 Table A-1 Quality Index Values for Estimating PWL I (continued)
Table A-1 Quality Index Values for Estimating PWL II PWL n = 12
to 14 n = 15 to 18
n = 19 to 25
n = 26 to 37
n = 38 to 69
n = 70 to 200
n = 201 to ∞
72 0.59 0.59 0.59 0.59 0.59 0.58 0.58
71 0.57 0.56 0.56 0.56 0.56 0.55 0.55
70 0.54 0.53 0.53 0.53 0.53 0.53 0.52
69 0.51 0.5 0.5 0.5 0.5 0.5 0.5
68 0.48 0.48 0.47 0.47 0.47 0.47 0.47
67 0.45 0.45 0.45 0.44 0.44 0.44 0.44
66 0.42 0.42 0.42 0.42 0.41 0.41 0.41
65 0.4 0.39 0.39 0.39 0.39 0.39 0.39
64 0.37 0.36 0.36 0.36 0.36 0.36 0.36
63 0.34 0.34 0.34 0.34 0.33 0.33 0.33
62 0.31 0.31 0.31 0.31 0.31 0.31 0.31
61 0.29 0.29 0.28 0.28 0.28 0.28 0.28
60 0.26 0.26 0.26 0.26 0.26 0.25 0.25
59 0.23 0.23 0.23 0.23 0.23 0.23 0.23
58 0.21 0.21 0.2 0.2 0.2 0.2 0.2
57 0.18 0.18 0.18 0.18 0.18 0.18 0.18
84
PWL n = 12 to 14
n = 15 to 18
n = 19 to 25
n = 26 to 37
n = 38 to 69
n = 70 to 200
n = 201 to ∞
56 0.16 0.15 0.15 0.15 0.15 0.15 0.15
55 0.13 0.13 0.13 0.13 0.13 0.13 0.13
54 0.1 0.1 0.1 0.1 0.1 0.1 0.1
53 0.08 0.08 0.08 0.08 0.08 0.08 0.08
52 0.05 0.05 0.05 0.05 0.05 0.05 0.05
51 0.03 0.03 0.03 0.03 0.03 0.03 0.02
50 0 0 0 0 0 0 0 Table A-1 Quality Index Values for Estimating PWL II (continued)
Table A-2 PWL Estimation Table for Sample Size n = 5 Q 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0 50 50.36 50.71 51.07 51.42 51.78 52.13 52.49 52.85 53.2
0.1 53.56 53.91 54.27 54.62 54.98 55.33 55.69 56.04 56.39 56.75
0.2 57.1 57.46 57.81 58.16 58.52 58.87 59.22 59.57 59.92 60.28
0.3 60.63 60.98 61.33 61.68 62.03 62.38 62.72 63.07 63.42 63.77
0.4 64.12 64.46 64.81 65.15 65.5 65.84 66.19 66.53 66.87 67.22
0.5 67.56 67.9 68.24 68.58 68.92 69.26 69.6 69.94 70.27 70.61
0.6 70.95 71.28 71.61 71.95 72.28 72.61 72.94 73.27 73.6 73.93
0.7 74.26 74.59 74.91 75.24 75.56 75.89 76.21 76.53 76.85 77.17
0.8 77.49 77.81 78.13 78.44 78.76 79.07 79.38 79.69 80 80.31
0.9 80.62 80.93 81.23 81.54 81.84 82.14 82.45 82.74 83.04 83.34
1 83.64 83.93 84.22 84.52 84.81 85.09 85.38 85.67 85.95 86.24
1.1 86.52 86.8 87.07 87.35 87.63 87.9 88.17 88.44 88.71 88.98
1.2 89.24 89.5 89.77 90.03 90.28 90.54 90.79 91.04 91.29 91.54
1.3 91.79 92.03 92.27 92.51 92.75 92.98 93.21 93.44 93.67 93.9
1.4 94.12 94.34 94.56 94.77 94.98 95.19 95.4 95.61 95.81 96.01
1.5 96.2 96.39 96.58 96.77 96.95 97.13 97.31 97.48 97.65 97.81
1.6 97.97 98.13 98.28 98.43 98.58 98.72 98.85 98.98 99.11 99.23
1.7 99.34 99.45 99.55 99.64 99.73 99.81 99.88 99.94 99.98 100
85
Table A-3 Another PWL Estimation Table for Sample Size n = 5 QL or QU PWLL or PWLU QL or QU PWLL or PWLU
1.671 or More 100 -0.029 to 0.000 50
1.601 to 1.670 99 -0.059 to -0.030 49
1.541 to 1.600 98 -0.079 to -0.060 48
1.491 to 1.540 97 -0.109 to -0.080 47
1.441 to 1.490 96 -0.139 to -0.110 46
1.391 to 1.440 95 -0.169 to -0.140 45
1.351 to 1.390 94 -0.199 to -0.170 44
1.311 to 1.350 93 -0.229 to -0.200 43
1.271 to 1.310 92 -0.249 to -0.230 42
1.231 to 1.270 91 -0.279 to -0.250 41
1.191 to 1.230 90 -0.309 to -0.280 40
1.151 to 1.190 89 -0.339 to -0.310 39
1.121 to 1.150 88 -0.369 to -0.340 38
1.081 to 1.120 87 -0.399 to -0.370 37
1.051 to 1.080 86 -0.429 to -0.400 36
1.011 to 1.050 85 -0.449 to -0.430 35
0.981 to 1.010 84 -0.469 to -0.450 34
0.951 to 0.980 83 -0.509 to -0.470 33
0.911 to 0.950 82 -0.539 to -0.510 32
0.881 to 0.910 81 -0.569 to -0.540 31
0.851 to 0.880 80 -0.599 to -0.570 30
0.821 to 0.850 79 -0.629 to -0.600 29
0.781 to 0.820 78 -0.659 to -0.630 28
86
QL or QU PWLL or PWLU QL or QU PWLL or PWLU
0.751 to 0.780 77 -0.689 to -0.660 27
0.721 to 0.750 76 -0.719 to -0.690 26
0.691 to 0.720 75 -0.749 to -0.720 25
0.661 to 0.690 74 -0.779 to -0.750 24
0.631 to 0.660 73 -0.819 to -0.780 23
0.601 to 0.630 72 -0.849 to -0.820 22
0.571 to 0.600 71 -0.879 to -0.850 21
0.541 to 0.570 70 -0.909 to -0.880 20
0.511 to 0.540 69 -0.949 to -0.910 19
0.471 to 0.510 68 -0.979 to -0.950 18
0.451 to 0.470 67 -1.009 to -0.980 17
0.431 to 0.450 66 -1.049 to -1.010 16
0.401 to 0.430 65 -1.079 to -1.050 15
0.371 to 0.400 64 -1.119 to -1.080 14
0.341 to 0.370 63 -1.149 to -1.120 13
0.311 to 0.340 62 -1.189 to -1.150 12
0.281 to 0.310 61 -1.229 to -1.190 11
0.251 to 0.280 60 -1.269 to -1.230 10
0.231 to 0.250 59 -1.309 to -1.270 9
0.201 to 0.230 58 -1.349 to -1.310 8
0.171 to 0.200 57 -1.389 to -1.350 7
0.141 to 0.170 56 -1.439 to -1.390 6
0.111 to 0.140 55 -1.489 to -1.440 5
87
QL or QU PWLL or PWLU QL or QU PWLL or PWLU
0.081 to 0.110 54 -1.539 to -1.490 4
0.061 to 0.080 53 -1.599 to -1.540 3
0.031 to 0.060 52 -1.669 to -1.600 2
0.001 to 0.030 51 -1.789 to -1.670 1
-0.029 to 0.000 50 -1.790 or Less 0
Table A-3 Another PWL Estimation Table for Sample Size n = 5
Table A-4 Areas Under the Standard Normal Distribution
Z
or
- Z
Z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
.0 .0000 .0040 .0080 .0120 .0160 .0199 .0239 .0279 .0319 .0359
.1 .0398 .0438 .0478 .0517 .0557 .0596 .0636 .0675 .0714 .0753
.2 .0793 .0832 .0871 .0910 .0948 .0987 .1026 .1064 .1103 .1141
.3 .1179 .1217 .1255 .1293 .1331 .1368 .1406 .1443 .1480 .1517
.4 .1554 .1591 .1628 .1664 .1700 .1736 .1772 .1808 .1844 .1879
.5 .1915 .1950 .1985 .2019 .2054 .2088 .2123 .2157 .2190 .2224
.6 .2257 .2291 .2324 .2357 .2389 .2422 .2454 .2486 .2517 .2549
.7 .2580 .2611 .2642 .2673 .2704 .2734 .2764 .2794 .2823 .2852
88
Z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
.8 .2881 .2910 .2939 .2967 .2995 .3023 .3051 .3078 .3106 .3183
.9 .3159 .3186 .3212 .3238 .3264 .3289 .3315 .3340 .3365 .3389
1.0 .3413 .3438 .3461 .3485 .3508 .3531 .3554 .3577 .3599 .3621
1.1 .3643 .3665 .3686 .3708 .3729 .3749 .3770 .3790 .3810 .3830
1.2 .3849 .3869 .3888 .3907 .3925 .3944 .3962 .3980 .3997 .4015
1.3 .4032 .4049 .4066 .4082 .4099 .4115 .4131 .4147 .4162 .4177
1.4 .4192 .4207 .4222 .4236 .4251 .4265 .4279 .4292 .4306 .4319
1.5 .4332 .4345 .4357 .4370 .4382 .4394 .4406 .4418 .4429 .4441
1.6 .4452 .4463 .4474 .4484 .4495 .4505 .4515 .4525 .4535 .4545
1.7 .4554 .4564 .4573 .4582 .4591 .4599 .4608 .4616 .4625 .4633
1.8 .4641 .4649 .4656 .4664 .4671 .4678 .4686 .4693 .4699 .4706
1.9 .4713 .4719 .4726 .4732 .4738 .4744 .4750 .4756 .4761 .4767
2.0 .4772 .4778 .4783 .4788 .4793 .4798 .4803 .4808 .4812 .4817
2.1 .4821 .4826 .4830 .4834 .4838 .4842 .4846 .4850 .4854 .4857
2.2 .4861 .4864 .4868 .4871 .4875 .4878 .4881 .4884 .4887 .4890
2.3 .4893 .4896 .4898 .4901 .4904 .4906 .4909 .4911 .4913 .4916
2.4 .4918 .4920 .4922 .4925 .4927 .4929 .4931 .4932 .4934 .4936
2.5 .4938 .4940 .4941 .4943 .4945 .4946 .4948 .4949 .4951 .4952
2.6 .4953 .4955 .4956 .4957 .4959 .4960 .4961 .4962 .4963 .4964
2.7 .4965 .4966 .4967 .4968 .4969 .4970 .4971 .4972 .4973 .4974
2.8 .4974 .4975 .4976 .4977 .4977 .4978 .4979 .4979 .4980 .4981
2.9 .4981 .4982 .4982 .4983 .4984 .4984 .4985 .4985 .4986 .4986
3.0 .4987 .4987 .4987 .4988 .4988 .4989 .4989 .4989 .4990 .4990
3.1 .4990 .4991 .4991 .4991 .4992 .4992 .4992 .4992 .4993 .4993
3.2 .4993 .4993 .4994 .4994 .4994 .4994 .4994 .4995 .4995 .4995
3.3 .4995 .4995 .4995 .4996 .4996 .4996 .4996 .4996 .4996 .4997
3.4 .4997 .4997 .4997 .4997 .4997 .4997 .4997 .4997 .4997 .4998 Table A-4 Areas Under the Standard Normal Distribution (continued)
89
Figure A-1. Illustration of the Calculation of the Z -statistic
Conceptually, the Q-statistic, or quality index, performs identically the same
function as the Z-statistic except that now the reference point is the mean of an
individual sample, , instead of the population mean, µ, and the points of interest with
regard to areas under the curve are the specification limits.
and
Where:
QL = quality index for the lower specification limit.
QU = quality index for the upper specification limit.
LSL = lower specification limit.
USL = upper specification limit.
90
= the sample mean for the lot.
s = the sample standard deviation for the lot.
The Q-statistic, therefore, represents the distance in sample standard deviation
units that the sample mean is offset from the specification limit. A positive Q-statistic
represents the number of sample standard deviation units that the sample mean falls
inside the specification limit. Conversely, a negative Q-statistic represents the number
of sample standard deviation units that the sample mean falls outside the specification
limit. These cases are illustrated in Figures A-2 and A-3.
QL is used when there is a one-sided lower specification limit, while QU is used
when there is a one-sided upper specification limit. For two-sided specification limits,
the PWL value is estimated as:
PWLT = PWLU + PWLL - 100
Where:
PWLU = percent below the upper specification limit (based on QU).
PWLL = percent above the lower specification limit (based on QL).
PWLT = percent within the upper and lower specification limits.
A.4 Example
An example using a simplified Portland cement concrete specification can be used
to explain the PWL concept. In this example, the minimum specification limit for strength
is 21,000 kPa. One requirement of the PWL procedure is that the sample size must be
greater than n = 2 since both the sample mean and sample standard deviation are
necessary to estimate PWL. For this specification, the sample size has been chosen as
91
n = 4. Furthermore, the specification requires that at least 95 percent of the lot exceed
the minimum strength (i.e., PWL > 95). Table A-1 shows that the minimum Q value is
1.35 for 95 PWL and a sample size of n = 4. Whenever the mean is 1.35s above the
specification limit, the lot is accepted. However, as used most frequently, the Q value
will be used to compute the PWL and that will, in turn, be used to determine a payment
factor.
For example, suppose that the acceptance tests for a lot have a sample mean of
25,000 kPa and a sample standard deviation of 3,400 kPa. Does this lot meet the
specification requirement? The quality index value is calculated as:
Using this Q-value, n = 4, and Table A-1, the estimated PWL for the lot is between
89 and 90. This is less than the required 95, so the lot does not meet the specified
strength requirement.
Figure A-2. Illustration of Positive Quality Index Values
92
Figure A-3. Illustration of a Negative Quality Index Value
Intuitively, PWL is a good measure of quality since it is reasonable to assume that
the more of the material that is within the specification limits, the better the quality of the
product should be. A detailed discussion and analysis of the PWL measure of quality is
presented in the technical report for the project.
93
LIST OF REFERENCES
1. American Association of State Highway and Transportation Officials (AASHTO), “Standard Recommended Practice for Acceptance Sampling Plans for Highway Construction,” AASHTO Designation R9-90, Washington, D.C., 1995.
2. American Association of State Highway and Transportation Officials (AASHTO), “Quality
Assurance Guide Specification,” Washington, D.C, 1996. 3. Transportation Research Board, “Glossary of Highway Quality Assurance Terms,”
Transportation Research Board, Electronic Circular E-C074, Third Update, Washington, D.C., 2005
4. Vidalis, Sofia M., “Relation Between Cost, Qulity, and Risk in Portland Cement Concrete
Pavement Construction,” University of Florida, Ph.D dissertation, 2005. 5. Duncan, A. J., “Quality Control and Industrial Statistics,” R.D. Irwin, ISBN 0256035350,
Fifth Edition, 1986. 6. Burati, J.L., Weed, R. M., Hughes, C. S., Hill, H. S., “Optimal Procedures for Quality
Assurance Specifications,” Federal Highway Administration, Publication No. FHWA-RD-02-095, McLean, VA., 2004.
7. Velivelli, K.L., Gharaibeh, N.G., Nazarian, S., “Sample Size Requirements for Seismic and
Traditional Testing of Concrete Pavement Strength,” Transportation Research Board, Transportation Research Record, No. 1946, pp. 33-38, 2006.
8. Weed, R.M., “Quality Assurance Software for the Personal Computer,” Federal Highway
Administration, Publication No. FHWA-SA-96-026, McLean, VA., 1996. 9. Kopac, Peter, “The Development of Operating Characteristic Curves for PENNDOT’s
Restricted performance Bituminous Concrete Specifications,” The Pennsylvania Transportation Institute, Project No. 74-27, Research Report No.3, 1976.
10. Tagaras, George, “Economic Acceptance Sampling by Variables with Quadratic Quality
Costs”, IIE Transactions, Vol. 26, No. 6, November 1994. 11. Von Quintus, H.L., Rauhut, J.B., Kennedy, T.W., and Jordahl, P.R., “Cost Effectiveness of
Sampling and Testing Programs,” Federal Highway Administration, Publication No. FHWA/RD-85/030, McLean, VA, 1985.
12. Gharaibeh, N.G., Garber, S.I., and Liu, L., “Determining Optimum Sample Sizes for Percent
Within Limits Specifications,” Transportation Research Board, Transportation Research Record ____, 2010, pp. __-__.
94
13. South, J.B., “Selecting an Acceptance Sampling Plan That Minimizes Expected Error and Sampling Cost,” Pittsburg State University, Quality Progress, Vol. 15, No. 10, pp. 18-22, 1982.
14. Lapin, L. Statistics—Meaning and Method. Harcourt Brace Jovanovich, Inc. New York, NY,
1975. 15. Fabrycky, W.J. and Thuesen, G.J., Economic Decision Analysis. Prentice-Hall, Inc.,
Englewood Cliffs, NJ, 1974. 16. “Military Standard 414, Sampling Procedures and Tables for Inspection by Variables for
Percent Defective,” Superintendent of Documents, Government Printing Office, Washington, DC, 1957.
17. Miller-Warden Associates, “Development of Guidelines for Practical and Realistic
Construction Specifications,” Highway Research Board, National Cooperative Highway Research Program Report 17, 1965.
18. National Highway Institute, “SpecRisk Quality Assurance Specification Development and
Validation Course,” Web-based course number FHWA-NHI-134070, www.nhi.fhwa.dot.gov.
19. Burati, J.L., “Risks with Multiple Pay Factor Acceptance Plans,” Transportation Research
Board, Transportation Research Record No. 1907, Washington, DC, 2005, pp. 37-42.
20. Hoerner, T.E. and Darter, M.I., “Improved Prediction Models for PCC Performance-Related Specifications, Volume II,” Federal Highway Administration, Publication No. FHWA-RD-00-131, McLean, VA, 2000.
21. J. L. Burati, R. M. Weed, C. S. Hughes, H. S. Hill, “Optimal Procedures for Quality Assurance Specifications,” Federal Highway Administration, Publication No. FHWA-RD-02-95, McLean, VA, 1985.
95
BIOGRAPHICAL SKETCH
Dooyong Cho was born in Seoul, Republic of Korea. He started his college career
at SungKyunKwan University in Seoul. He earned Bachelor of Science and Master of
Science degrees, majoring in civil engineering in 2002. He came over United States of
America to study abroad in 2002. At Pennsylvania State University, he earned Master of
Engineering degree in the Department of Civil and Environmental engineering. He
transferred to and entered in a Ph.D program in the Civil and Coastal Engineering
Department at the University of Florida in 2005 in Ph.D program specializing in
infrastructural engineering and public works.