Transcript American-European NDE Reliability Workshop Summary

RTFI Process Assurance
Assuring On Target Performance Through Reliability Metrics
IC
AC
HF
Flaw Size
Lloyd A Schaefer - NDTU
RTFI Process Assurance - Agenda



Radiographic process Goals and Objectives
RTFI in the Product Design process
Defining and Quantifying Reliability





Human Factors
Methods & tools to ensure On Target reliability
In process metrics to track variation
Conclusions and benefits
Questions
Radiographic Process Objectives
Why do we perform Radiographic Inspection?



To find all defects?
To find the “good” parts?
To find the “bad” parts?
When we perform RTFI, have we;



Found all of the defects?
Found most of the defects?
Found all of the rejectable defects according to the code?
Examination of Engineering Design can provide some answers
Engineered Product Design Process
Welds, and other products, examined by RT;


Are imperfect structures with variable integrity
Are the result of design efforts to;

Application optimize products through knowledge of material,
engineering, fabrication & cost, variables and behavior
Design variables which influence product integrity:



Material specification based on properties & behavior
Fabrication process parameters & capability
Radiographic Inspection process capability
RTFI’s role: Describe material & fabrication results
Need a way to integrate the inputs & quantify the result
Integrated Results of Design,
Fabrication and Inspection
Number of flaws/
Probability of Detection
n/1
Acceptance Criteria
Flaw and Detection Distributions
0
Flaw/Defect size
Domains & Definitions
Number of flaws/
Probability of Detection
n/1
Acceptance Criteria
Flaw and Detection Distributions
Flaws Detected at [ ]
Probability & Rejected
0
Detected and accepted
Flaw/Defect size
Material, Process &
Inspection Dynamics
Number of flaws/
Probability of Detection
n/1
Acceptance Criteria
• Flaw distributions change via Material
Properties & Process Capability
0
Flaw/Defect size
Material, Process &
Inspection Dynamics
Number of flaws/
Probability of Detection
n/1
Acceptance Criteria
• Detection results change via technology
and performance, parameters & variation
0
Flaw/Defect size
Material, Process &
Inspection Dynamics
Number of flaws/
Probability of Detection
Acceptance Criteria
• For a given process/inspection, the Acceptance
Criteria determines net distribution of flaws
n/1
0
Flaw/Defect size
Goal for RTFI Process Reliability:
Stay on Target!
Probability of Detection
(POD)
• Limit POD variability to ensure design estimate
of part imperfection distribution is accurate
1
Historical design
assumption for
RT/RTFI process
0
Flaw/Defect size
What is RTFI, or any NDE, Reliability?
NDE processes can be defined as:



A physical process with intrinsic limits (IC)
Limited and varied by a specific sets of application
parameters, (AC)
Further varied by human factors during the performance
of the inspection (HF)
Conceptual Model for NDE Reliability:
R=f [AC,HF] < IC
and,
AC = f (AP,HF) < IC, where HF=0
Graphical Representations
of NDE Reliability
IC
Phy. limit
AC
HF
R
Predicted Reliability
… or in terms of probability
IC
100%------
AC
HF
R
Flaw size, a, or condition measured
Measuring RTFI Process Reliability
RTFI is a Human Factors (HF) intensive process


Specific measurements of reliability, are not routinely
made for RTFI
Reliance upon a controlled process and trained, certified
inspectors to meet legacy performance assumptions
Possible outcomes from an RTFI activity:
inspection report
truth
or
NDT
TP
TP + FN = 100%
FN
system
TN + FP = 100 %
or
TN
TP: true positive indication (hit)
TN: true negative indication
FP
FN: false negative indication
FP: false positive indication (false alarm)
Measuring RTFI Process Reliability
Receiver Operating Characteristic (ROC)
A method for describing reliability via the 4 possible
outcomes
Comparison ofdifferentNDT -systems
bability of correct detections

1.0
S
tandarddeviationsfor
Noise : 1.0
S
ignal : 1.0
M
eanvalueofnoise: 0
0.5
Differenceofthem
eanv
signal -noise:
1-0.0
2-0.5
3-1.0 inc
Measuring RTFI Process Reliability
Probability of Detection
• The ROC can be related to the POD curve by plotting the
true positives proportions as a function of flaw size
1
0
Flaw/Defect size
Methods for keeping RTFI on Target
Round Robin Testing (RRT) identifies keys:


Training and certification
Years of experience (ref. Fucsok et. al.)





Examined ROC vs. years of experience; 0-5, 6-22, 23-35
True Positive performance maximized in the 6 to 22 year
experience group tested
False call performance maximized in the 23-35 year group
More data needed to validate findings
Consensus, or peer review

Strong & consistent influence on accuracy improvement in
multiple Probability of Detection Studies
• Rummel
• Suggs
• Stanley
Effects of Peer Review on
RTFI Reliability
Peer Review Illustration:



Statistically significant series of Radiographs shown to
several certified inspectors
Both flaws and unflawed samples used with at least 3:1
ratio of flawed to unflawed areas
Objective: To determine correct interpretation &
consensus reliability
24A/6R

21A/9R
16A/14R
Which welds are acceptable? Which need to be repaired?
Effects of Peer Review on
RTFI Reliability

Intersection of decisions produces most correct
results via physical/metallurgical validations
24A/6R 16A/14R
21A/9R
Peer Review fever: Catch it!
Other Tools/Influences on
RTFI Reliability
Digital/Quantitative Feedback Training
 Densitometer as density training aid


Usage results in increased accuracy of film density
estimates
Film Digitization/Profiling
PM
Weld

Improves quantitative skills
Example of Metrics to
Observe RTFI Variation
Other Tools/Influences on
RTFI Reliability
Identification of Inspector Potential



EPRI developed Dynamic Inspector Aptitude Testing,
DIAT
Testing motivated to quantify human factors which
influence inspector performance in POD studies
Five (5) aptitudes measured:






General cognitive ability
Abstract reasoning
Spatial visualization
Pattern recognition
Stress tolerance
Saudi Aramco can participate in this testing via its
association with EPRI through Henry Stephens
Conclusions & Benefits of
on Target RTFI Reliability


Variation in the RTFI process should be managed
Key variations & effects in the RTFI process:


Type II errors, “misses”, produce a right shifted POD curve
Unique improvements, or creeping elegance, which can result
in a left shifted POD curve, but which are not uniform in effect
• Produces an increase in Type I errors and repairs
• Leads to an increase in the amount of inherent flaws

When the RTFI Process is On Target:



Manufacturing has optimum knowledge of process health
Accurate predictions can be made of the condition and life of
the structure
Risk management is improved through accurate flaw
distribution inputs to the Risk Based Assessment modules
Reliability concepts should be extended to all NDE