RTFI Process Assurance
Assuring On Target Performance Through Reliability Metrics
Lloyd A Schaefer - NDTU
Flaw Size
Rel
iabi
lity
IC ACHF
RTFI Process Assurance - Agenda
Radiographic process Goals and Objectives RTFI in the Product Design process Defining and Quantifying Reliability
Human Factors Methods & tools to ensure On Target reliability In process metrics to track variation Conclusions and benefits Questions
Radiographic Process Objectives
Why do we perform Radiographic Inspection? To find all defects? To find the “good” parts? To find the “bad” parts?
When we perform RTFI, have we; Found all of the defects? Found most of the defects? Found all of the rejectable defects according to the code?
Examination of Engineering Design can provide some answers
Engineered Product Design Process
Welds, and other products, examined by RT; Are imperfect structures with variable integrity Are the result of design efforts to;
Application optimize products through knowledge of material, engineering, fabrication & cost, variables and behavior
Design variables which influence product integrity: Material specification based on properties & behavior Fabrication process parameters & capability Radiographic Inspection process capability
RTFI’s role: Describe material & fabrication results
Need a way to integrate the inputs & quantify the result
Integrated Results of Design, Fabrication and Inspection
Flaw/Defect size
Nu
mb
er
of
flaw
s/
Pro
bab
ilit
y o
f D
ete
cti
on
0
Flaw and Detection Distributions
n/1
Accep
tan
ce
Cri
teri
a
Domains & Definitions
Flaw/Defect size
Nu
mb
er
of
flaw
s/
Pro
bab
ilit
y o
f D
ete
cti
on
0
Flaw and Detection Distributions
n/1
Accep
tan
ce
Cri
teri
a
Flaws Detected at [ ] Probability & Rejected
Detected and accepted
Material, Process & Inspection Dynamics
Flaw/Defect size
Nu
mb
er
of
flaw
s/
Pro
bab
ilit
y o
f D
ete
cti
on
0
• Flaw distributions change via Material Properties & Process Capability
n/1
Accep
tan
ce
Cri
teri
a
Material, Process & Inspection Dynamics
Flaw/Defect size
Nu
mb
er
of
flaw
s/
Pro
bab
ilit
y o
f D
ete
cti
on
0
• Detection results change via technology and performance, parameters & variation
n/1
Accep
tan
ce
Cri
teri
a
Material, Process & Inspection Dynamics
Flaw/Defect size
Nu
mb
er
of
flaw
s/
Pro
bab
ilit
y o
f D
ete
cti
on
0
• For a given process/inspection, the Acceptance Criteria determines net distribution of flaws
n/1
Accep
tan
ce
Cri
teri
a
Goal for RTFI Process Reliability:Stay on Target!
Flaw/Defect size
Pro
bab
ilit
y o
f D
ete
cti
on
(P
OD
)
0
• Limit POD variability to ensure design estimate of part imperfection distribution is accurate
1
Historical design assumption for RT/RTFI process
What is RTFI, or any NDE, Reliability?
NDE processes can be defined as: A physical process with intrinsic limits (IC) Limited and varied by a specific sets of application
parameters, (AC) Further varied by human factors during the performance
of the inspection (HF)
Conceptual Model for NDE Reliability:R=f [AC,HF] < ICand,AC = f (AP,HF) < IC, where HF=0
Graphical Representations of NDE Reliability
R
IC
AC
HFPredicted Reliability
Flaw size, a, or condition measured
R
… or in terms of probability
Phy. limit
100%------IC AC
HF
Measuring RTFI Process Reliability
RTFI is a Human Factors (HF) intensive process Specific measurements of reliability, are not routinely
made for RTFI Reliance upon a controlled process and trained, certified
inspectors to meet legacy performance assumptions
Possible outcomes from an RTFI activity:
NDT TP FN
TN FP
TP + FN = 100%
TN + FP = 100 %
truth inspection report
TP: true positive indication (hit) FN: false negative indication
TN: true negative indication FP: false positive indication (false alarm)
or
or
system
Measuring RTFI Process Reliability
Receiver Operating Characteristic (ROC) A method for describing reliability via the 4 possible
outcomes
Measuring RTFI Process Reliability
Flaw/Defect size
Pro
bab
ilit
y o
f D
ete
cti
on
0
• The ROC can be related to the POD curve by plotting the true positives proportions as a function of flaw size
1
Methods for keeping RTFI on Target
Round Robin Testing (RRT) identifies keys: Training and certification Years of experience (ref. Fucsok et. al.)
Examined ROC vs. years of experience; 0-5, 6-22, 23-35 True Positive performance maximized in the 6 to 22 year
experience group tested False call performance maximized in the 23-35 year group More data needed to validate findings
Consensus, or peer review Strong & consistent influence on accuracy improvement in
multiple Probability of Detection Studies• Rummel• Suggs• Stanley
Effects of Peer Review on RTFI Reliability
Peer Review Illustration: Statistically significant series of Radiographs shown to
several certified inspectors Both flaws and unflawed samples used with at least 3:1
ratio of flawed to unflawed areas Objective: To determine correct interpretation &
consensus reliability
Which welds are acceptable? Which need to be repaired?
24A/6R 21A/9R 16A/14R
21A/9R
Effects of Peer Review on RTFI Reliability
Intersection of decisions produces most correct results via physical/metallurgical validations
24A/6R 16A/14R
Peer Review fever: Catch it!
Other Tools/Influences on RTFI Reliability
Digital/Quantitative Feedback Training Densitometer as density training aid
Usage results in increased accuracy of film density estimates
Film Digitization/Profiling
Improves quantitative skills
WeldPM
Example of Metrics to Observe RTFI Variation
Hawiyah RTFI Process Quality
0
10
20
30
40
50
60
70
80
90
100
110
6/16/2
000
6/30/2
000
7/14/2
000
7/28/2
000
8/11/2
000
8/25/2
000
9/8/20
00
9/22/2
000
10/6/
2000
10/20
/2000
11/3/
2000
Sam
ple
Siz
e &
Qu
alit
y P
erce
nta
ges
Total Welds Evaluated Submittal % Assessed Flaw Detection % Overcall % Film Quality %
Detection & Film Quality LCL
Overcall UCL
Other Tools/Influences on RTFI Reliability
Identification of Inspector Potential EPRI developed Dynamic Inspector Aptitude Testing, DIAT Testing motivated to quantify human factors which
influence inspector performance in POD studies Five (5) aptitudes measured:
General cognitive ability Abstract reasoning Spatial visualization Pattern recognition Stress tolerance
Saudi Aramco can participate in this testing via its association with EPRI through Henry Stephens
Conclusions & Benefits of on Target RTFI Reliability
Variation in the RTFI process should be managed Key variations & effects in the RTFI process:
Type II errors, “misses”, produce a right shifted POD curve Unique improvements, or creeping elegance, which can result
in a left shifted POD curve, but which are not uniform in effect• Produces an increase in Type I errors and repairs• Leads to an increase in the amount of inherent flaws
When the RTFI Process is On Target: Manufacturing has optimum knowledge of process health Accurate predictions can be made of the condition and life of
the structure Risk management is improved through accurate flaw
distribution inputs to the Risk Based Assessment modules
Reliability concepts should be extended to all NDE