Embed Size (px)
Citation preview
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 1
Statistical Design Methods
For Engineers
Lecture 3: Probability Density FunctionsLecture 3: Probability Density Functions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 2
Objectives
1.1. Introduce concept of probability, probability density functions Introduce concept of probability, probability density functions (pdf) and cumulative probability distributions (CDF).(pdf) and cumulative probability distributions (CDF).
2.2. Discuss central limit theorem & origin of common pdfs: uniform, Discuss central limit theorem & origin of common pdfs: uniform, normal, Weibull. normal, Weibull.
3.3. Introduce JFIT tool to generate pdfs from simulation or Introduce JFIT tool to generate pdfs from simulation or experimental data.experimental data.
4.4. UseUse a fitted pdf to calculate a probability of non compliance a fitted pdf to calculate a probability of non compliance (PNC).(PNC).
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 3
Probability of Events: Multiplication Rule
o Let A = an event of interesto Let B = another event of interesto Let C = Event A occurring and event B occurring.
– What is the probability of compound event C occurring? – P(C)=P(A and B)=P(A)P(B|A)
P(A and B)=P(A∩B)=probability of A occurring multiplied by the probability of B occurring given that event A has occurred.
– P(B|A) is called a conditional probability
– P(C)=P(B and A)=P(B)P(A|B)– If events A and B are independent of one another then P(B|A)=P(B) and
similarly P(A|B)=P(A) so P(C)=P(A)P(B).– If events A and B are mutually exclusive then P(A|B)=0=P(B|A)
o Therefore P(A and B)=P(A)P(B|A)=P(B)P(A|B) which implies P(A|B)=P(A)P(B|A)/P(B) (Bayes’ Theorem)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 4
Probability of Events: Addition Rule
o Let A = an event of interesto Let B = another event of interesto Let D = Event A occurring or event B occurring.
– What is the probability of compound event D occurring? – P(D)=P(A or B)=P(A)+P(B)-P(A and B)
P(A or B ) is defined as an “inclusive OR” which means = probability of event A or event B or both events A and B
– If events A and B are independent of one another then P(D)=P(A) +P(B)-P(A)P(B).
– If events A and B are mutually exclusive then P(D)=P(A)+P(B)
A B
A∩B
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 5
Example 1
o Have two shirts in closet, say one red and the other blue.o Have 3 pants in closet, say one brown, one blue and one
green.o What is probability that randomly picking a shirt and pants that
one chooses a red shirt (event A) and blue pants (event B)?o P(A and B)=P(A)P(B)=(1/2)(1/3)=1/6 (independent events)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 6
Example 2
o Have two parts in series in a circuit and both parts have to operate in order for circuit to operate. (series reliability)
o R1=probability of part 1 operating, R2 = probability of part 2 operating.
o If the reliability of part 1 is independent of the reliability of part 2 then P(part 1 AND part 2 operating)=R1*R2
R1 R2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 7
Example 3
o What if two parts are in parallel in the circuit and at least one of the two parts must operate for the circuit to operate. (parallel reliability)
o P(part1 operating OR part2 operating)=R1 +R2 – R1*R2 if the reliability of part1 is independent of the reliability of part2.– How likely is this independence to be true?– Might the reliability of one of the parts depend on whether or not the
other part is operating?
R1
R2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 8
Defining a PDF
Probability Density Function, f(x)Probability Density Function, f(x) Definition - a mathematical function, f(x), such that
f(x)dx = the probability of occurrence of a random variable X within the range x to x+dx. i.e.
f(x)dx = Pr{x< X < x+dx} A typical Probability Density may appear as -
f(x)
X
Fre
q. O
f O
ccu
rren
ce
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 9
Properties of a PDF
Probability of OccurrenceProbability of Occurrence - is the area under a pdf bounded by any specified lower and upper values of the random variable X.
f(x) for all values of x Total area under f(x) = 1.0
( ) ( )U
L
x
L U xP x x x f x dx
f(x)=freq of occurrence per unit of x
xxL xU
)( UL xXxP
0.1)( xP
Note That The Value of f(x) Can Be > 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 10
Properties of a CDF
Cumulative DistributionCumulative Distribution Function, F(x) Function, F(x) - a function, called a CDF, that returns the cumulative probability that a random variable, X, will have a value less than x. Maximum value = 1.0
X
Pro
bab
ilit
y
x
x)XP(F(x) 1.0
xxduufxXPxF for )()()(
Cumulative Distribution
Function, CDF
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 11
Expectation
Expected value of a random variable X.– If X has only discrete value {xi, i=1,2,…,n} and pi is the probability of having the
value X=xi. The expected value of X, written E[X], is determined by
– If X has a continuous distribution of possible values then f(x)dx = the probability that X is between x and x+dx. The expected value of X is then given by weighting each value of X by its probability of occurrence f(x)dx.
1
i
1 21 2
[ ]
If p =1/n for all i = 1,2, ,n.
i.e. all n possible values have same probability of being chosen.
1 1 1[ ] arithmetic average
n
i ii
nn
E X p x
x x xE X x x x
n n n n
0
[ ] ( )
1. . ( ) , [ ]x x
E X xf x dx
e g f x e E X x e dx
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 12
Measures of Central Tendency
Mean
Median (middle value)
Mode (most probable)
x=+
X1x=-
1Mean= = xf(x)dx, Sample Mean = x =
n
n
kk
x
m
-
F(median)=0.5= f(y)dy=Pr(X m)
X=mode
df0, mode m
dx
Positive Skewness
0
0.005
0.01
0.015
0.02
0.025
0.03
0 20 40 60 80 100
x
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 13
Measures of Dispersion
Variance
Standard Deviation
Range
Mean Absolute Deviation
x=+
22 2 2X X
1x=-
1Variance (x- ) f(x)dx, Sample Variance = s
1
n
X kk
x xn
2
X1
1Standard Deviation= ,Sample Stdev = s
1
n
X kk
Variance x xn
Range R Max x Min x
x=+
X1x=-
1|x- |f(x)dx, Sample MAD
n
n
kk
MAD x x
3 2 1 1 2 3
Normal pdf
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 14
Variance
Discrete distribution
Continuous distribution
2 2 2 2 2
1 1
x-12
[ ] [ ] [ [ ]] [ ] [ [ ]]
1 1-p. . geometric distribution f(x) = (1-p) p, E[X]= , Var[X]=
p p
n n
i i i iVar X p x E X p x E X E X E X
e g
22 2
2- x
2 2
[ ] ( [ ]) ( ) ( ) [ ]
2 1 1e.g. exponential distribution f(x)= e , [ ]
Var X x E X f x dx x f x dx E X
Var X
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 15
Measure of Asymmetry
Skewness (3rd Central Moment)
x=+3 3
X
x=-
3
x=+33
X3 31x=-
Skewness (x- ) f(x)dx [( ) ],
SkewnessCoefficient of Skewness
StandardDeviation
1 1(x- ) f(x)dx, Sample Sk
( 1)( 2)
X
n
kk
E x
Sk
Sk x xs n n
Positive Skewness
0
0.005
0.01
0.015
0.02
0.025
0.03
0 20 40 60 80 100
x
Sk>0, positive Skewness
Mean > Median > Mode
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 16
Measure of “Peakedness”
Kurtosis (4th Central Moment)
x=+4 4
X X
x=-
4
x=+4
X4x=-
42
1
Kurtosis (x- ) f(x)dx =E[(x- ) ],
KurtosisCoefficient of Kurtosis
StandardDeviation
1(x- ) f(x)dx,
( 2 3) (2 3)( 1)Sample Ku 3
( 1)( 2)( 3) ( 2)(
nk
k
Ku
Ku
x xn n n n
n n n s n n n
3)Ku>3, more peaked than Normal distribution
Ku < 3, less peaked than Normal distribution
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 17
Probability Distribution
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
random variable, X
f(x)
, pd
f
0
0.25
0.5
0.75
1
-1.2
5
-1.0
0
-0.7
5
-0.5
0
-0.2
5
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
2.25
2.50
2.75
3.00
3.25
3.50
3.75
Z
CD
F
pdf CDF
Mean = 25
Median = 20.1
Mode = 7.6
stdev = = 20
1
( )x
Weibull
xf x e
Weibull distribution
Z≡(X-) /
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 18
Useful Properties
Expected value of sum of variables is sum of expected values
Variance of a constant, c, times a random variable x is c2 times the variance of x.
Variance of sum of two variables is sum of variances of each variable IF the variables are independent of one another.
1 2 1 2[ ] [ ] [ ] [ ]n nE x x x E x E x E x
2 2 2
2 2 2 2 2 2 2
[ ] [( [ ]) ] [ ] ( [ ])
[ ] [ ] ( [ ]) [ ] ( [ ]) [ ]
Var x E x E x E x E x
Var cx E c x E cx c E x cE x c Var x
If x is independent of y, then cov[x,y]=0 (converse i
[ ] [ ] [ ] cov[ , ]
cov[ , ] [( [ ])( [ ])] [ ] [ ] [ ]
[ ] [ ] [ ]
s not true, Why?)
Var x y Var x Var y x y
x y E x E x y E y E xy E x E y
Var x y Var x Var y
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 19
Types of Distributions
Discrete Distributions Bernoulli Distributions, f(x)=px(1-p)1-x
Binomial Distributions, f(x)=(n!/[(n-x)!x!])px(1-p)n-x
Poisson, f(x)=(x/x!) exp(-), Continuous Distributions
Uniform f(t) = 1/(b-a) fpr a<t<b, zero otherwise Weibull Distributions,
f(t)=()(t/)1exp(-(t/))– Exponential Distribution
f(t)=exp(-t) Logistic Distribution, f(z)=z/[b(1+z)2], z=exp[ (x-a) /b] Raleigh Distribution, f(r)=(r/2)exp(-½(r/)2) Normal Distribution,
f(x)=(1/212)exp(-1/2x2
Lognormal Distribution• f(x)=(1/x212)exp(-1/2lnx)2
Central Limit Theorem–Law of Large Numbers (LLN)
(not discussed here)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 20
Another Way to Pick Distributions: Distributions can be determined using maximum entropy arguments.
Entropy is a measure of disorder or statistical uncertainty.
The density function which maximizes the entropy* expresses the largest uncertainty for a given set of constraints, e.g.
– If no parameters of the data are known f(x) is a Uniform Distribution (max uncertainty says all values equally
probable)– If only the mean value of the data is known
f(x) is an Exponential Distribution– If the mean and variance are known
f(x) is a Normal Distribution (Maxwellian distribution if dealing with molecular velocities)
– If the mean, variance and range are known f(x) is a Beta Distribution
– If the mean occurrence rate between independent events is known: (mean time between failures)
f(x) is a Poisson Distribution
* Ref Reliability Based Design in Civil Engineering, Milton E Harr, Dover Publications 1987,1996.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 21
Discrete Distributions
Bernoulli, binomial
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 22
Bernoulli Trial & binomial distribution
o What if perform test and outcome can either be a pass or a fail (binary outcome). This is called a Bernoulli trial or test.o Let p = probability of a failure
1
Bernoulli distribution
( ) { } ( ) , 0,1
{ 1} probability of failure in te
1
st,
{ 0} probability of passed test
[ ] , [ ] (1 )
1
x xp
p
f x P X x x
P X
P X
E X p Var X
p
p p
p
Since many experiments are of the pass/fail variety, the basic building block for such statistical analyses is the Bernoulli distribution. The random variable is x and it takes on a value of 1 for failure and 0 for passed.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 23
Multiple Bernoulli Trials
o Perform say 3 Bernoulli trials and want to know the probability of 2 successes. R=1-p where p = probability of failure.– P(s=2|3,R) = P(first trial is success and second is success and third is
failure OR first success and second failure and third success OR first fails and second is success and third is success)
– R*R*(1-R) + R*(1-R)*R + (1-R)*R*R = 3R2(1-R)
– General formula P(s|n,R)=nCs * Rs * (1-R)n-s (binomial distribution)
General formulation of binomial distribution
where
( ) ( ) 1n xxn
f x P X x p px
!
! !
n n
x n x x
2 2 2
3 1
4 2
[ ]
[ ] ( [ ]) (1 )
(1 )
1 63
E x np
E x E x npq np p
np p npq
q p
npq
pq
npq
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 24
Probability Mass Function, b(s|n,p)
o Graph of f(x) for various values of n. (p=0.02)
0
0.1
0.2
0.3
0.4
0.5
0.6
0 1 2 3 4 5 6 7 8 9 10 12 14 16 18 20
f(x|n=20,p)
f(x|n=100,p)
f(x|n=500,p)
begins to look like a normal distribution with mean = n*p
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 25
Example with binomial distribution
Suppose we performed pass / fail test (Bernoulli trial) on a system x=1 if fails x=0 if it passes . Perform this test on n systems, the resulting estimate of the probability of non compliance = sum of x values / number of trials, i.e. <PNC> = (x1+x2+…+xn) / n = # noncompliant / # tests.
2 2
/ estimate of PNC
[ ] , [ ] (1 )
[ / ] /
[ / ] [ ] / (1 ) / (1 ) /
[ / ] (1 ) / (1 ) /
binomial
p x n
E x np Var x np p
E x n np n p
Var x n Var x n np p n p p n
Stdev x n p p n p p n
This information will be used later
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 26
0 5
0.2
0.4
Continuous Distributions: Plots for Common PDFs
Uniform
Normal
Log-Normal
Rayleigh
Weibull
Exponential
Gamma
Beta
0 2 4 6
0.1
0.2
0.3
0 5 10
0.2
0.4
0 1 2 3
0.5
1
0 1 2 3
1
2
0 1 2 3
1
2
0 2 4
0.5
1
0 0.5 1
2
4
6
10 0.5
1
2
See following charts for details →
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 27
Uniform & Normal Distributions
Normal distribution (Gaussian distribution)
21
21( )
2
x
f x e
2
3 1
4 2
mean
variance
std dev
0 coef. of skewness
3 coef, of kurtosis
Standard Normal distribution (Unit Normal distribution)
define z = (x - ) /
21
21( )
2
zf z e z
2
3 1
4 2
mean=0
variance=1
std dev=1
0 coef. of skewness
3 coef, of kurtosis
Uniform distributionpdf
CDF
1 ,
b-a( )0 ,otherwise
a x bf x
0 ,x<a
x-a( ) ,a x bb-a
1 ,x>b
F x
22
3 1
4 2
1mean=
21
variance=12
b-astd dev=
2 3
0 coef. of skewness
9 coef. of kurtosis
5
a b
b a
Both the uniform and normal distributions are used in RAVE
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 28
Central Limit Theorem and Normal Distribution
The Central Limit Theorem says:– If X= x1+x2+x3+…+xn and if each variable, xj, comes from a its own
distribution whose mean is j, stdev j, then the variable Z, defined below obeys a unit normal distribution, signified by N(0,1).
– This is true as long as no single xj value overwhelms the rest of the sum
What does this mean? Using a simpler form for Z this means
1 1
2
1
~ (0,1)
n n
j jj j
n
jj
x
Z N
1
1
~ (0,1)
n
jj
xn x
Z Nn
n
The pdf of the mean value, , of a variable is a normal distribution independent of what
distribution characterized the variable itself.
x
-3s/√ -2s/√n -1s/√n +1√n +2s/√n +3/√n
Mean of x, <x>
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 29
Generating PDFs from Data
Commercial codes Crystal Ball
http://www.decisioneering.com/ EasyFit version 4.3 (Professional)
http://www.mathwave.com/ Need some goodness of fit (gof)
test. Kolmogorov-Smirnov (K-S) Anderson-Darling (A-D) Cramer-von Mises (C-vM) Chi squared (2) others
Codes developed by J.L. Alderman
JFIT (quick)– Uses Johnson family of
distributions.– 4-parameter
distributions.– Parameters chosen to
match first 4 central moments of data.
GAFIT (slower, but more accurate fit)– Uses Johnson family– Parameters chosen
using Rapid Tool.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 30
2.82124992
Methods Of PDF Fitting
The ‘Crystal Ball’ Tool and the ‘EasyFit’ Tool both Fit the Actual Data To Each of The Various Density Functions In their Respective Libraries
Temperature
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 31
2.82124992
Temperature
Methods Of PDF Fitting
The PDF That Produces The Best Match, According To User Selectable Criteria (Chi Squared, Kolmogorov-Smirnoff(KS), Anderson-Darling (AD) ) Is Selected To Represent the Data.
The ‘Crystal Ball’ Tool and the ‘EasyFit’ Tool both Fit the Actual Data To Each of The Various Density Functions In their Respective Libraries
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 32
2.82124992
Temperature
N. L. Johnson Method Of PDF Determination
The Johnson family of distributions are fitted in the excel tool, JFIT and commercial tool EasyFit™ by matching the first four standardized central moments of the data to the expressions for the moments of the distributions. GAFIT uses GA for optimized fit.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 33
The Johnson family of distributions is composed of four density functions (normal, lognormal, unbounded and bounded)
These members are all related to the unit normal distribution by a mathematical transformation of variables
Computations are simplified because they can be done using the unit normal variable distribution f(Z)~N(0,1).
N. L. Johnson Method Of PDF Determination
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 34
Calculation of Moments from Data
Standardized Central MomentsMOMENT NAME SYMBOL EXCEL FUNCTION DIRECT CALCULATION (Unbiased)
1st (noncentral) MEAN AVERAGE(Range)*
2nd (central) VARIANCE VAR(Range)*
3rd (central)/s3 SKEWNESS SKEW(Range)*
4th (central)/s4 KURTOSIS* KURT(Range)+3
4th (central)/s4 KURTOSIS* [(n2-2n+3)/(n(n+1))]*KURT + 3(n-1)/(n+1)*
*Unbiased values
x2s
Sk
Ku
1
1 n
ii
xn
2
1
1
1
n
ii
x xn
3
13( 1)( 2)
n
ii
n x x
n n s
42
14
( 2 3)(2 3)( 1)
3( 2)( 3)( 1)( 2)( 3)
n
ii
n n x xn n
n n nn n n s
*Ku
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 35
Regions of Johnson Distributions
Regions for Johnson Distributions
0
1
2
3
4
5
6
7
8
0 1 2 3 4
1=(skewness)2
2 =
kurt
osi
s
Two ordinate
lognormal
Bounded, SB
Unbounded, SU
Forbidden Region
For Any PDF, Kurtosis > Skew2 +1 This Defines The Impossible Area
Skewness = Normalized Third Central Moment
Kurtosis = Normalized Fourth Central Moment
211
21112
3
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 36
JSB, Bounded Density Function2
1ln
2 1( )2 (1 )
y
yf x ey y
,
xy x
( ) ln1
yF x
y
The Johnson bounded
distribution can fit many different
shapes because it has 4 adjustable
parameters
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 37
JSU Unbounded Density Function
xy
221
ln 12
2( )
2 1
y yf x e
y
2( ) ln 1F x y y
Again using 4 parameters allows
for better fitting.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 38
JFIT ToolSteps: Enter Data in Either C6-9 OR C13-17 not both!
2. Set Plot Range (D22-D24)3. Enter Limits Or Target Yield Below PDF chart4. Press compute
4.00000 MeanVar 1.00000 Variancesk 0.18226 Skewk 3.05910 Kurtosis
Do NOT insert data in both C6:C9 and C13:C17
-46.19231651 Types: Problems: 16.49535715 JSL=LogNormal 0 None 1 JSU=Unbounded 1 Variance <= 0 -12.48020369 JSB=Bounded 2 Kurt < Skew^2 + 1
Type J SL JSN=Normal 3 Sb did not convergeProbs? 0 JST=Two-Ordinate 4 Inconsistent Input Data
5 SU did not converge
LL 2.1268 Yield(%) = 95.00% UL= 6.0452
zmin = -3zmax = 3 Right Click on/near the axis values to change the number formats
# of steps = 200z x P(z) PDF(x) CDF(x)
-3.000000 1.2343E+00 4.4318E-03 5.3305E-03 1.3499E-03-2.49 1.663702784 1.7907E-02 2.0884E-02 6.3615E-03-2.28 1.848182597 2.9812E-02 3.4321E-02 1.1373E-02-2.13 1.972738181 4.0850E-02 4.6623E-02 1.6385E-02-2.03 2.0687069 5.1266E-02 5.8124E-02 2.1396E-02-1.94 2.147687153 6.1187E-02 6.8999E-02 2.6408E-02-1.86 2.215323077 7.0697E-02 7.9355E-02 3.1419E-02-1.79 2.274808354 7.9849E-02 8.9268E-02 3.6431E-02-1.73 2.328134588 8.8687E-02 9.8790E-02 4.1442E-02-1.68 2.376631603 9.7241E-02 1.0797E-01 4.6454E-02-1.63 2.421233645 1.0554E-01 1.1682E-01 5.1465E-02-1.59 2.46262341 1.1359E-01 1.2539E-01 5.6477E-02-1.54 2.501315693 1.2143E-01 1.3370E-01 6.1489E-02
Raytheon
First Four Central Moments of The Data
Output Parameters of Johnson Curve if data inserted in C6:C9.
Data Will Be Plotted For if you use the Excel Function
'Kurt' to compute the Kurtosis you must add 3 to
get the correct value for entry into cell c9
Press ENTER if you change Lower or Upper Limits or Change Targeted Yield
Compute
0.0037
0.0434
0.0831
0.1228
0.1625
0.2022
0.2419
0.2817
0.3214
0.3611
0.4008
1.2343 1.8359 2.4376 3.0392 3.6409 4.2425 4.8442 5.4459 6.0475 6.6492 7.2508
PD
F(x
)
CDF
0.0000
0.1000
0.2000
0.3000
0.4000
0.5000
0.6000
0.7000
0.8000
0.9000
1.0000
1.2343 1.8359 2.4376 3.0392 3.6409 4.2425 4.8442 5.4459 6.0475 6.6492 7.2508
CD
F(x
)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 39
JFIT Tool(Mode 1)
Steps: Enter Data in Either C6-9 OR C13-17 not both!2. Set Plot Range (D22-D24)3. Enter Limits Or Target Yield Below PDF chart4. Press compute
4.00000 MeanVar 1.00000 Variancesk 0.18226 Skewk 3.05910 Kurtosis
Do NOT insert data in both C6:C9 and C13:C17
-46.19231651 Types: Problems: 16.49535715 JSL=LogNormal 0 None 1 JSU=Unbounded 1 Variance <= 0 -12.48020369 JSB=Bounded 2 Kurt < Skew^2 + 1
Type J SL JSN=Normal 3 Sb did not convergeProbs? 0 JST=Two-Ordinate 4 Inconsistent Input Data
5 SU did not converge
First Four Central Moments of The Data
Output Parameters of Johnson Curve if data inserted in C6:C9.
Compute
0.0037
0.0434
0.0831
0.1228
0.1625
0.2022
0.2419
0.2817
0.3214
0.3611
0.4008
1.2343 1.8359 2.4376 3.0392 3.6409 4.2425 4.8442 5.4459 6.0475 6.6492 7.2508
Insert values for the four moments into the top set of cells then press compute and the
results appear in the lower set of cells.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 40
JFIT Tool (Mode 2)
MeanVar Variancesk Skewk Kurtosis
Do NOT insert data in both C6:C9 and C13:C17
3.193742536 Types: Problems: 1.195191141 JSL=LogNormal 0 None 45.23630054 JSU=Unbounded 1 Variance <= 0 -0.776525532 JSB=Bounded 2 Kurt < Skew^2 + 1
Type J SB JSN=Normal 3 Sb did not convergeProbs? JST=Two-Ordinate 4 Inconsistent Input Data
5 SU did not converge
First Four Central Moments of The Data
Output Parameters of Johnson Curve if data inserted in C6:C9.
0.0005
0.0236
0.0468
0.0699
0.0931
0.1162
0.1394
0.1625
0.1856
0.2088
0.2319
-0.5239 1.5297 3.5833 5.6370 7.6906 9.7442 11.7979
13.8515
15.9051
17.9588
20.0124
PD
F(x
)
Remove values for the four moments in the top set of cells. Insert known values for Johnson parameters in the lower set of cells. Press compute and the pdf and CDF graphs appear for that particular distribution
0.0005
0.0236
0.0468
0.0699
0.0931
0.1162
0.1394
0.1625
0.1856
0.2088
0.2319
-0.5239
1.5297 3.5833 5.6370 7.6906 9.7442 11.7979
13.8515
15.9051
17.9588
20.0124
PD
F(x
)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 41
JFit TABS (Bottom of JFit Screen)
Johnson SL – Logarithmic Distributions
Johnson SU – Unbounded Distributions
Johnson SB – Bounded Distributions
Johnson SN, Normal Distributions
Johnson ST, Two Ordinate Distributions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 42
Johnson Type 1: Log Normal (JSL)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 43
Johnson Type 2 Unbounded (JSU)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 44
Johnson Type 3 Bounded (JSB)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 45
Johnson Type 4 Normal (JSN)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 46
Johnson Type 5
This is the Histogram corresponding to g = 0 h = . 80 e = .022 l = .078
Johnson ST Fit: Two Ordinate Distribution
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 47
Using JFIT To Estimate PNC or Yield
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 48
Acronyms and Abbreviations
AD - Anderson Darling CLT - Central Limit Theorem JFIT - Johnson Fitting Tool KS - Kolmogorov Smirnoff LSL - Lower Specification Limit PDF -Probability Density Function PNC - Probability of Non Compliance USL - Upper Specification Limit
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 49
End of PDF Fitting Module
(Lecture 3)Back up slides
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 50
Distribution Fitting - Preliminary Steps
This article covers several steps you should consider taking before you analyze your probability data and apply the analysis results. – Step 1 - Define The Goals Of Your Analysis
The very first step is to define what you are trying to achieve by analyzing your data. You should have a clear understanding of your goals as this will help you throughout the entire data analysis process. Try answering the following questions:
What kind of information would you like to obtain? How will you obtain the information you need? How will you apply that information?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 51
Example:
Robert is the head of the Customer Support Department at a large company. In order to reduce the customer service times and improve the customer experience, he would like to do the following:
Determine the probability that a customer can be served in 5 minutes or less. To solve this problem, Robert needs to: – Perform distribution fitting to sample data (customer service times) for a
selected period of time (e.g. last week) – Select the best fitting distribution – Calculate the probability using the cumulative distribution function of the
selected distribution
If the probability is less than 95%, consider hiring additional customer support staff
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 52
Step 2 - Prepare Data For Distribution Fitting
Preparing your data for distribution fitting is one of the most important steps you should take, since the analysis results (and thus the decisions you make) depend on whether you correctly collect and specify the input data. – Data Format
Your data might come in one of the generally accepted formats, depending on the source of data and how it was collected. You need to make sure the distribution fitting software you are using supports the data format you need, and if it doesn't, you might need to convert your data to one of the supported formats.
The most commonly used format in probability data analysis is an unordered set of values obtained by observing some random process. The order of values in a data set is not important and does not affect the distribution fitting results. This is one of the fundamental differences between distribution fitting (and probability data analysis in general) and time series analysis where each data value is connected to some time-point at which this value was observed.
– Sample Size The rule of thumb is the more data you have, the better. In most cases, to get reliable
distribution fitting results, you should have at least 75-100 data points available. Note that very large samples (tens of thousands of data points) might cause some computational problems when fitting distributions to data, and you might need to reduce the sample size by selecting a subset of your data. However, in many cases one has only 10 to 30 data points. Expect larger variances!
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 53
Step 3 - Decide Which Distributions To Fit
Before fitting distributions to your data, you should decide which distributions are appropriate based on the additional information about the data you have. This can be helpful to narrow your choice to a limited number of distributions before you actually perform distribution fitting.
– Data Domain - Continuous or Discrete? The easiest part is to determine whether your data is continuous or discrete. If your data can take
on real values (for example, 1.5 or -2.33), then you should consider continuous distributions only. On the other hand, if your data can take on integer values (1, 2, -5 etc.) only, then you might want to fit both continuous and discrete distributions.
The reason to use continuous distributions to analyze discrete data is that there is a large number of continuous distributions which frequently provide much better fit than discrete distributions. However, if you are confident that your random data follows a certain discrete distribution, you might want to use that specific distribution rather than continuous models.
– The Nature of Your Data In most cases, you have not just raw data, you also have some additional information about the
data and its properties, how the data was collected etc. This information might be very useful to narrow your choice to several probability distributions.
– Example. If you are analyzing the sales data of a company, it should be clear that this kind of data cannot contain negative values (unless the company sells at a loss), and thus it wouldn't make much sense to fit distributions which can take on negative values (such as the Normal distribution) to your data.
In addition, some particular distributions are recommended for use in several specific industries. An obvious example of such an industry is reliability engineering which makes great use of the Weibull distribution and several additional models (Exponential, Lognormal, Gamma) to perform the analysis of failure data. These distributions are widely used in many other industries, but in reliability engineering they are considered "standard".
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 54
Additional Distributions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 55
Geometric distribution
Probability that an event occurs on the kth trial. – Let p = probability of event occurring in a single trial.– f(k)= probability of event occurring on kth attempt.= (1-p)k-1p
the is called the probability mass function or pmf.
Example: Find probability a capacitor shorts due to puncturing of dielectric after there have been k voltage spikes. If the probability of a puncture on the jth voltage spike is pj, then the probability of puncture on the kth spike is– f(k)=(1-p1)(1-p2)…(1-pk-1)pk
If all the p values are the same then f(k)=(1-p)k-1p which is the geometric distribution probability mass function or pmf.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 56
Hypergeometric distribution
Hypergeometric distribution (use in sampling w/o replacement)
P(X=x,n | N,M) = ( ) ( )
population size
M=# bad units in population
n = sample size drawn from N w/o replacement
x = # failed units in sample
M N M
x n xf x P X x
N
n
N
2
Mmean , p
N
(1 )( 1) 1
np
npq N n N nnp p
N N
Ex. N=600 missiles in inventory, M=80 missiles were worked on by Operator #323 and they are believed to have been defectively assembled by this operator. If we take n=20 missiles randomly from the inventory, then f(x) = probability of x defective missiles in the sample. f(0)=0.054
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 57
Poisson Distribution
Used extensively for situations involving counting of events – e.g. probability of x false alarms during time period T=60 seconds when the
average alarm rate is (false alarms/ sec).
– The pmf is f(k) = (T)k*exp(-T) / k!= (60)k*exp(-60) / k!E[k]=1/112
Also used in predicting number of failed units over time period TW, given the mean time to failure MTTF =1/. – f(k units failing in time TW)= (TW)k*exp(-TW)/k!.
If want a 90% probability of x or fewer failures in time span TW , then find– F(x)=0.90= exp(-TW) + (TW)1*exp(-TW)/1! + (TW)2*exp(-TW)/2! +
… + (TW)x*exp(-TW)/x! (solve numerically for TW the warranty period.) – If you were considering the case of x=0 failures during the warranty period, then
you could set the warranty period = TW =-ln(0.9)/ ~ 0.105/ years. If your warranty period TW = 1 year and the MTTF=10 years then you should
expect only 1- exp(-11) ~ 9.5% of units produced to have one or more failures during that time.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 58
Uniform Discrete Distribution
Assume b > a integers f(x|a,b) = 1/(b-a+1) for a≤ x ≤ b. n uniformly distributed pts.
= 0, otherwise
F(x)= 0, x≤a= (x-a+1)/(b-a+1), a≤ x ≤ b=1, x≥b
There are n=b-a+1 discrete uniformly distributed data points– Mean = (a+b)/2– Var(X) = [(b-a+1)2-1]/12
a a+1 k b
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 59
Continuous distributions
Gamma distribution (Erlang for = integer)
distribution of time up to having exactly =k eventsoccur. Used in queueing theory, reliability, etc..
1 /
, 0( ) ( )
0 , 0
xx ex
f x P X x
x
2 2
3 1
4 2
mean
variance
2
3 2
Logistics distribtion
2 2
1 1( )
1 1
x a x a
b b
x a x a
b b
e ef x
b be e
1( )
1x a
b
F x
e
2 2
3
4
mean=a=median=mode
variance 30
4.2
b
0
0.1
0.2
0.3
0.4
0.5
0.6
-2 -1 0 1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 60
Beta & Weibull distributions
Beta distribution
11 1,0 1
( ) ( , )
0 ,otherwise
( ) ( )( , ) beta function
( )
x xx
f x B
B
2
2
mode
=+
+ + 1
1
+ 2x
Weibull distribution
1
, , 0, 0( )
0 ,otherwise
( ) ( ) 1
t
t
te tf t
F t P T t e
2 2 2
11
2 11 1
Exponential distribution
, 0, 0( )
0 ,otherwise
( ) ( ) 1
t
t
e tf t
F t P T t e
22
3 1
4 2
1
1
2
9
0 0.5
1
2
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 61
Rayleigh & Triangle distributionsRayleigh distribution
2
2
1
22
1
2
, 0, 0( )
0 ,otherwise
( ) ( ) 1
x
x
xe xf x
F x P X x e
2 2
3 1 3/2
2
2 24 2
2
2 14
3 2 0.632
32 3 /(4 ) 3.245
Triangle ditributiona= lower limitb = upper limitm = most probable or mode
a m b
2,
( )2
,
x aa X m
b a m af x
b xb X m
b a b m
2 2 2 2
3
3
4
/ 3
/18
/ 3 ( ) ( ) ( )1 1 1 2
10 ( ) ( )
324 2.4135
a m b
a m b ab am bm
b a m a b m m a
b a b a b a
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 62
Lognormal and Normal distributions
Normal distribution (Gaussian distribution)
21
21( )
2
x
f x e
2
3 1
4 2
mean
variance
std dev
0 coef. of skewness
3 coef, of kurtosis
Standard Normal distribution (Unit Normal distribution)
define z = (x - ) /
21
21( )
2
zf z e z
2
3 1
4 2
mean=0
variance=1
std dev=1
0 coef. of skewness
3 coef, of kurtosis
Lognormal distribution
2ln
ln
ln1
2
ln
1,0( ) 2
0 ,
x
e xf x x
otherwise
2ln
ln 2
2ln
ln
2 2
=mean
=variance = 1
m =median
e MTTF
e
e
Used to model mean time to repair for availability calculations
Many statistical process control calculations
assume normal distributions and
incorrectly so!
Any normal distribution can be scaled to the unit normal
distribution. Useful when using tables to look up values
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.817
17.6
18.2
18.8
19.4 20
20.6
21.2
21.8
22.4 23
Lecture 3: PDFs 63
Mixed distribution. Used to model bi-modal processes.
f(x)=pg1(x) + (1-p)g2(x)
Definitions.
)64)(1()64(
)3)(1()3(
))(1()(
)1(
42
22
222
321
421
41
21
211
311
4114
322
22
322
311
21
3113
22
22
21
21
22
21
SKpSKpK
SpSpS
pp
pp
,)( ,)(
,)( ,)(
,
,)( ,)(
,)( ,)(
44
33
22
222211
2121
21
1212
dxxgxKdxxgxS
dxxgxdxxgx
dxxgxdxxfx
dxxxgdxxxf
jjjjjjjj
jj
jr
jrjr
r
jj
