Transcript HCLPF: High-Confidence, Low-Probability-of

High-Confidence, LowProbability-of-Failure Screening
January 11, 2014
1
0.9
0.8
0.7
0.6
P[P[Failure|Eq]<0.05]95%
0.5
0.4
0.3
0.2
0.1
0
0
2
RLGM?
4
6
8
10
12
14
16
Magnitude
18
20
HCLPF depends on whom you believe
• Which components need seismic safety improvements?
– If HCLPF < RLGM (Review Level Ground Motion), improve them
– “HCLPF is defined as the earthquake [ground] motion level at which there is a
high (95 percent) confidence of a low (at most 5 percent) probability of
failure” Interim Staff Guidance, DC/COL-ISG-020
• HCLPF: SoV vs. CDFM
– AmExp[-2.32bC] “CDFM” (Conservative Deterministic Failure Margin, “because
engineers understand it”) [Kennedy et al.]
– AmExp[-1.65(bR+bU)] [“SoV” (Separation of Variables), (EPRI TR-103959,
Ravindra, and others]
• Am should be the a “High-Confidence” limit on the natural logarithm of median strength
• -1.65 and -2.32 are the number of 5% and 1% standard deviations below the normal
distribution [log] mean Am
• The b-parameters represent randomness or aleatory variability (R), epistemic uncertainty (U),
or combined uncertainty (C)
• Notice any sample uncertainty?
Objective of Seismic Screening:
Component HCLPF < RLGM?
• Guidance unspecific, encourages industry initiative, abhors
statistics?
– ASCE/SEI 41-06 and -13, FEMA E-74 Chapter 4.1, and IEAE NS-G1.6
– RG 1.208, “A Performance-based Approach to Define the Sitespecific Earthquake Ground Motion” (NRC, 2007b) “The desired
performance being expressed as the target value of 10-5 for the
mean annual probability of exceedance (frequency) of the onset
of significant inelastic deformation “
• Plants with higher seismic risk are already shut down
– Vallecitos, Humboldt Bay, Zion, Fukushima Daiichi, San Onofre
– Older plants have little time left, except for 20-or-30-year
extensions!
Alternative Uncertainty Model
  a 
 In
A 
bU 
f     m 

bR



-1

Q  





Q = P[f < f’ | a]; i.e., the subjective probability
(confidence) that the conditional probability of
failure, f, is less than f’ for a peak ground
acceleration a (e.g., RLGM)
-1(.)= the inverse of the standard Gaussian
cumulative distribution function.
How about some
Rational Economic Resource Allocation?
• Cost-benefit analysis and budget constrained resource
allocation is not under current consideration
– Cost of alternatives? Fix some? Shut down plant?
• How to allocate resources to aleatory (random) or
epistemic (unknown) uncertainty?
– Is reducing epistemic uncertainty [unknown unknowns] a scam?
Quants use scenarios. Elizabeth Paté-Cornell says insure or
hedge (Allin’s wife)
• Price-Anderson act provides insurance to nuclear utility industry
• NRC insures industry for costs incurred by delays caused by NRC
– How to allocate to high confidence vs. low probability of failure
• “The public appears to heavily value confidence and places a much
smaller, although still positive, emphasis on accuracy.” [Smith and
Wooten]
Legal, Precautionary Principle, and
Social Equity
• “…if an action or policy has a suspected risk of causing
harm to the public or to the environment, in the
absence of scientific consensus that the action or
policy is harmful, the burden of proof that it is not
harmful falls on those taking an action.”
http://en.wikipedia.org/wiki/Precautionary_principle
• Search for “Kip Viscusi and Risk Equity” for guidance
• ISO 26000 Social Responsibility, ISO 16269-6
“Determination of Statistical Tolerance Intervals”
HCLPF Borrows Credibility
• To statisticians, HCLPF is a tolerance limit or a confidence limit on a
sample percentile (ISO 16269-6 and
http://en.wikipedia.org/wiki/Tolerance_limit
– The “High-Confidence” part of HCLPF corresponds to the statistical
confidence limit, typically 95%
– The “Low-Probability-of-Failure” part of HCLPF corresponds to the
percentile, typically 5-th percentile of the fragility function
– Tolerance limits are supposed to be estimated from data, perhaps
censored, perhaps extrapolated
• To reliability engineers, HCLPF is a R95C95 reliability demonstration
test: 95% confidence that reliability > 95%
– RnnCmm demonstration uses sample test or field (observed) data
• HCLPF uses subjective opinions or response analysis to relate
component fragility to RLGM instead of local component response
– Awkwardly related to material strength distributions [ASME et al.]
Methods: Economics
• Minimize seismic risk subject to budget constraint
– Allocate resources to biggest bang-per-buck =
Seismic Risk/$$$(component fragility parameter)
• Seismic risk = E[Discounted future costs due to earthquakes]
– = exp(-dt)*P[Eq of magnitude m at age t]*E[Cost|Eq of
magnitude m at age t]dF(m,t)
– integrate from now to license expiration
– Conditional on components to be improved
• $$$(component fragility parameter) = cost per unit change
in component fragility parameter
– Use chain rule (next slide)
– Use law of diminishing marginal returns: the more you
spend on one component, the less you get
Chain Rule
•  Seismic Risk /  $$$(component fragility parameter) =
–  Seismic Risk /  Subsystem Risk*  Subsystem Risk / 
$$$(component fragility parameter)
–  Subsystem Risk /  $$$(component fragility parameter) =
 Subsystem Risk /  Component type risk *  Component
Type Risk /  $$$(component fragility parameter)
–  Component Type Risk /  $$$(component fragility
parameter) =  P[Component Failure|Eq] / 
$$$(component fragility parameter)
– Where component parameters include fragility
correlations!
Why is HCLPF = AmExp[-1.65b]?
• Assume strength at failure RV is AmeReU
– Where Am is lower 5-th confidence limit on median component
strength at failure and
– ln(eR) and ln(eU) are normally distributed, independent random
variables with means 0 and (logarithmic) standard deviations bR
and bU
• Then P[Stress > Strength] = [ln(Stress/Am)/bR2+bU2)] and
– Screen is P[RLGM > HCLPF?] = 5% 
– HCLPF? = -1[0.05, ln(Am), bR2 + bU2)] or
– AmExp[1.64485bR2+bU2)] where standard normal z(0.05) =
1.64485
– i.e., P[Stress/(AmeReU) > 1] = [ln(Stress/Am > 0] = 0.05
Methods: Data and Subjective
Opinions to Obtain HCLPF
• Data: ASME and other material strength distributions,
component test data, and post-earthquake failure
observations
– Maximum likelihood, Bayes, and method of moments
– “The seismic capacity of such equipment in regard of their
functionality during and after an earthquake is impossible,
difficult or unreliable to evaluate by other methods” [Other than
seismic test, Tengler]
• Subjective opinions on median, percentiles, (logarithmic)
standard deviations, and Y1|Y2 [NUREG/CR-3558 and
others]
– Least squares and weighted least squares
– Output Am and b from “High-Confidence” fragility function, and
r
Excel Workbook Implementations
• HCLPF….xlsx
– Computes HCLPFs and P[RLGM > HCLPF] for various
components from various input parameters
• SubjFrag.xlsx
– Computes High-Confidence fragility functions from test
and seismic observation data, subjective opinions, and
multiple-failure-mode fragility functions
– Estimates fragility correlations from subjective opinions of
P[X1|X2]
• NoFail.xlsm
– Estimates lognormal fragility function parameters,
including correlation, from earthquake responses and
component no-failure observations
HCLPF…xlsx Spreadsheets
Spreadsheet Name
Contents
IP2ESEL
Jim Moody’s list of components
BoM
FLEX PWR list of components
Inherent
ESEL Summary table list of components
HCLPF
Compute HCLPF and P[RLGM > HCLPF]
from component lists and fragility
parameters
HELParameters
Bandyopadhyay fragility parameters
Structures
Structure CDFM fragility parameters
Equipment
Equipment CDFM fragility parameters
ToleranceInterval
Estimate correct tolerance limit from
sample data 95% confidence on 5-th
percentile
HCLPF Inputs
• List of component candidates for screening, QPA, structure
function (series, parallel, RBD, fault tree)
• Design basis and RLGM and their units
• Method(s) (CDFM or SoV?)
• Component “fragility” parameters Am and b
– Strength-at-failure distributions for materials and some
components or “High-Confidence” Am (median) and bR
(logarithmic) standard deviation assuming lognormal strength
– Uncertainties due to components’ strength and responses to
seismic ground motions, bU
– Safety and other fudge factors for CDFM computation of
structure and equipment fragility parameters
HCLPF…xlsx:HCLPF Spreadsheet
• HCLPF spreadsheet does the HCLPF and
P[RLGM>HCLPF]? and other computations
– From component lists, parameter lists, and parameter
computations in other spreadsheets
– Table 1 contains notes
– Table 2 originated from AREVA proposal form
• Added QPA, Units, Method, Factors, Am, b, and
computation columns
• Computations include: z(p)Amb, RLGM>HCLPF?,
(ln(RLGM/Am)/b), ({ln(RLGM/Am)+ bU-1(Q(RLGM)))/bR),
and P[All QPA components survive]
• Could use VLOOKUP() function or links to enter parameters
from other spreadsheets
HCLPF Spreadsheet Table 2
ESEL Item
#
ID
Review
Referenc Level
QPA e Ground Ground
assumed Motion Motion
in series (Design) RLGM Units
Description
Mechanical
Turbine driven AFW pump
Turbine driven AFW valves
SG PORVs
Condensate storage tank
Tanks, Ch. 7 TR-103959
Tanks, Table 7-11 and NP-6041
SG injection valves
RCS injection valves
Other mechanical equipment
Electrical
Batteries
DC distribution panels
DC MCCs
DC switchgear
Vital AC distribution panels
Battery chargers
Inverters
Instrument racks
1
2
2
1
2
2
2
2
10
2
2
2
2
2
1
1
4
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
0.2 pga, g
TRS(C)Te RRS(C)
st
Required F(D)
Method Respons Respons Device
SoV? e
e
capacity
CDFM? Spectrum Spectrum factor
SoV? CDFM?
SoV? CDFM?
SoV? CDFM?
SoV? CDFM?
SoV
CDFM
SoV? CDFM?
SoV? CDFM?
SoV? CDFM?
SoV? CDFM?
SoV? CDFM?
SoV
SoV
SoV
SoV
SoV? CDFM?
SoV? CDFM?
SoV
F(RS)
Scale
Respons S(A) Avg. Factor SF
e factor spectral for
for accelerat referenc
structure ion e ground Am
If bC is not
SQRT(bR^2+bU^2),
please enter your
value
Units
1 pga, g
1 pga, g
1 pga, g
1 pga, g
1.14 0.537736 0.053774 pga, g
0.52 pga, g
1 pga, g
1 pga, g
1 pga, g
3.3
2.8
4.4
3.3
1
1
1
1
1
0.5 pga, g
0.33 pga, g
0.28 pga, g
0.44 pga, g
0.33 pga, g
0.5 pga, g
0.5 pga, g
0.1 pga, g
bR
bU
bC
((ln(RLGM
/Am)+bU*
Am*exp(Am*exp((ln(RLGM/ ^(Am*exp(- RLGM > 1.65*(bR RLGM > 2.326*bC RLGM > (ln(RLGM/ Am)/Sqrt(bR 1)(Q(a)))/bR P[min >
1.65*bR) HCLPF? +bU)) HCLPF? )
HCLPF? Am)/bR) ^2+bU^2)) )
RLGM]
0.2 0.2 0.282843 0.718924 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.22 0.17 0.278029 0.037404 TRUE
0.229 0.244 0.33463 0.356373 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.2 0.2 0.282843 0.718924 FALSE
0.516851
0.516851
0.516851
0.516851
0.028255
0.238264
0.516851
0.516851
0.516851
FALSE
FALSE
FALSE
FALSE
TRUE
FALSE
FALSE
FALSE
FALSE
0.517942
0.517942
0.517942
0.517942
0.028165
0.238765
0.517942
0.517942
0.517942
FALSE
FALSE
FALSE
FALSE
TRUE
FALSE
FALSE
FALSE
FALSE
4.23585E-16 6.3437E-09 4.23585E-16
4.23585E-16 6.3437E-09 4.23585E-16
4.23585E-16 6.3437E-09 4.23585E-16
4.23585E-16 6.3437E-09 4.23585E-16
0.999999999 0.999998846 0.999999999
1.50612E-05 0.002148932 1.50612E-05
4.23585E-16 6.3437E-09 4.23585E-16
4.23585E-16 6.3437E-09 4.23585E-16
4.23585E-16 6.3437E-09 4.23585E-16
0.00E+00
0.00E+00
0.00E+00
0.00E+00
1.00E+00
3.01E-05
0.00E+00
0.00E+00
4.44E-15
0.2
0.11
0.1
0.13
0.11
0.2
0.2
0.2
0.258426
0.176283
0.162437
0.192823
0.176283
0.258426
0.258426
0.051685
FALSE
TRUE
TRUE
TRUE
TRUE
FALSE
FALSE
TRUE
0.258971
0.167495
0.156246
0.176722
0.167495
0.258971
0.258971
0.051794
FALSE
TRUE
TRUE
TRUE
TRUE
FALSE
FALSE
TRUE
2.30877E-06 0.000598536 2.30877E-06
2.65058E-06 0.042930687 2.65058E-06
0.000383104 0.089862431 0.000383104
6.59536E-10 0.022190135 6.59536E-10
2.65058E-06 0.042930687 2.65058E-06
2.30877E-06 0.000598536 2.30877E-06
2.30877E-06 0.000598536 2.30877E-06
0.999735609 0.992869981 0.999735609
4.62E-06
5.30E-06
7.66E-04
1.32E-09
5.30E-06
2.31E-06
2.31E-06
1.00E+00
0.2
0.27
0.23
0.37
0.27
0.2
0.2
0.2
0.282843
0.291548
0.250799
0.392173
0.291548
0.282843
0.282843
0.282843
0.359462
0.275226
0.23741
0.355056
0.275226
0.359462
0.359462
0.071892
FALSE
FALSE
FALSE
FALSE
FALSE
FALSE
FALSE
TRUE
HCLPF Computations Include…
• AmExp[z(p)b]: HCLPF = AmExp[z(p)b] etc. for alternative
z(p) and b (p = “Low-Probability-of-Failure” 5% or 1%)
• RLGM>HCLPF?: TRUE or FALSE
• (ln(RLGM/Am)/b) = P[RLGM > HCLPF] for alternative b
• ({ln(RLGM/Am)+bU-1(Q(RLGM))}/bR) [Ravindra,
Kennedy, et al.]
• P[min>RLGM] = P[All QPA components survive]
– Assumes QPA co-located, series components with iid
fragilities
– Change the formula for parallel components or other
configuration (FTA) or correlated fragilities
SubjFrag.xlsx Spreadsheets
Spreadsheet Name
Contains
ToleranceInterval
Same as HCLPF…xlsx:ToleranceInterval
19experts
Compute least-squares lognormal fit to
subjective lower 5th percentiles
19expertsWeighted
Ditto with weighted least squares
10expertsBootstrap
Bootstrap 95% “High-Confidence”
lognormal fit with fewer than 19 experts
SubjCorr
Estimate r from subjective distribution of
Y1|Y2 by least squares
Tolerance Limit on Lognormal RV
Fragility Function
• Estimate correct tolerance limit on a lognormal distribution
from a sample of means m (= ln(Am)) and (logarithmic)
standard deviations b
• Input desired “High-Confidence” and “Low-Probability-ofFailure) and a sample of means and standard deviations
from tests or subjective opinions
– "Exposure Assessment: Tolerance Limits, Confidence Intervals,
and Power Calculation…" K. Krisnamoorthy et al.
• Confidence limit for m + z(p)*b is constructed of the form…
• m +Q(p,alpha)*b from estimates m and b are the MLEs and Q(p,alpha)
is the tolerance factor determined so that …
• P[m +Q(p,alpha)* b < something] > 1  alpha…
• where (somethingm)/b is approximately ~ (z(p)m)/b
• Output is tolerance limit or HCLPF (table 2 is simulated)
“High-Confidence” Subjective
Fragility Function Estimation
• Suppose 19 experts give 19 opinions on fragility
median Am and (logarithmic) standard deviation
• For each (discrete) value of strength y, find maximum
of 19 cdfs and connect with a curve Fmax(y)
– P[Strength < y|Expert Am and b], (or Am and some
percentile) and
– Assume each represents the upper 95% confidence limit
(“High-Confidence”)
– Fit a lognormal distribution to minimum curve to represent
a 95% “High-Confidence” fragility function using
(weighted) least squares
• Note that P[F(y) ≤ Fmax(y) for all y]  0.95
Lognormal Fit to Lower 5-th
Percentiles
19 opinions, maxima, and 5th percentile fragility function
1
lower 5th %ile
0.9
Expert 1
Expert 2
0.8
Expert 3
Expert 4
0.7
Expert 5
0.6
Expert 6
Expert 7
0.5
Expert 8
Expert 9
0.4
Expert 10
0.3
Expert 11
Expert 12
0.2
Expert 13
Expert 14
0.1
Expert 15
Expert 16
0
0
1
2
3
4
5
ln[strength at failure]
6
7
8
9
10
Expert 17
By Weighted Least Squares
1
19 opinions, maxima, and 5th percentile fragility function
0.9
lower 5th %ile
0.8
Expert 1
Expert 2
0.7
Expert 3
Expert 4
0.6
Expert 5
Expert 6
Expert 7
0.5
Expert 8
Expert 9
0.4
Expert 10
Expert 11
0.3
Expert 12
Expert 13
0.2
Expert 14
Expert 15
0.1
Expert 16
Expert 17
0
0
1
2
3
4
5
ln[strength at failure]
6
7
8
9
Expert 18
10
Expert 19
Max
What if there aren’t 19 experts?
• Bootstrap
• Correlation estimate requires at least one
subjective opinion of distribution of X1|X2
• Use least squares to combine experts’
subjective distribution information
– Sum of squared errors indicted magnitude of
experts’ deviation from lognormal distribution
Bootstrap 10 experts
1
0.9
0.8
lower 5th %ile
0.7
Expert 1
Expert 2
0.6
Expert 3
Expert 4
Expert 5
0.5
Expert 6
Expert 7
0.4
Expert 8
Expert 9
0.3
Expert 10
Max
0.2
0.1
0
0
2
4
6
8
10
12
14
16
18
20
NoFail.xlsm Spreadsheets
• Imagine inspections after earthquakes
indicate component responses and failure or
non-failure
Spreadsheet Name
Contents
Freq
Mle of lognormal fragility parameters
from 19 non-failure responses, assuming
20-th would be failure
Bayes
MoM estimates of lognormal fragility
parameters of a-posteriori distribution of
P[Failure|Eq] from 19 non-failure
responses assuming non-informative prior
BayesCorr
Same as Bayes, including estimate of
fragility correlations
Freq Spreadsheet
• Input iid responses for which no failures occurred
– 19 responses were simulated for example and
convenient interpretation of mean and standard
deviation estimates as “High-Confidence”
– Assume ln[Stress]-ln[Strength} ~ N[muX-muY,
Sqrt(sigmaX^2+sigmaY^2)]
• Use Solver to maximize log likelihood of PP[Nonfailure|Response] subject to constraint
– Either constrain CV or P[Failure]
• Output is ln(Am) and b
Bayes Spreadsheets
• Bayes estimate of reliability r = P{ln[Stress]ln{strength] > 0]
– Non-informative prior distribution of r
– Same inputs as Freq: responses and non-failures
– Use MoM to find ln(Am) and b to match posterior E[r]
= n/n+1 and Var[r] = n/((n+1)^2*(n+2))
– Ditto to find correlation r from third moment of aposteriori distribution of r
• Bayes posterior P[ESEL component life > 72
hours|Eq and plant test data]
Parameter Estimates from
19 Non-Failure Responses
• Given 19 earthquake responses with ln(Median) = 0.5 and b = 0.1 and
reliability P[ln(Stress) < ln(strength)] ~ 95%
• Bayes non-informative prior on reliability P[Response < strength] =>
posterior distribution
• Use Method of Moments to estimate parameters for a-posteriori
distribution of reliability
Parameter
Assume 20-th
is failure
m(ln(strength))
0.781404
0.148132
b ln(strength)
R ln(strength)
N/A
Bayes
Bayes
Correlation
0.781389 0.950047891
0.148123 0.226336118
N/A
0.752266708
What is the correlation of fragilities?
• See SubjFrag.xlsx:SubjCorr and
NoFail.xlsm:BayesCorr spreadsheets to
estimate correlations from subjective opinions
on Y1|Y2 or from no-failure response
observation
• HCLPF ignores fragility correlation
• Risk doesn’t ignore it
What if multiple, co-located
components?
• Responses are same (Refer to work for
Howard last year)
• In series? Parallel? RBD? Fault tree?
• Using event trees, Jim Moody argues that
HCLPF for one component is representative of
all like, co-located components
What if like-components are
dependent?
• Fragilities could be dependent too!
• But not necessarily all fail if one fails
– True, P[Response > strength] may be same for all
like, co-located components
– But what is P[g(Stress, strength) = failure] for
system structure function g(.,.)?
More References
•
•
•
•
•
•
•
NAP, “Review of Recommendations for Probabilistic Seismic Hazard Analysis:
Guidance on Uncertainty and the Use of Experts (1997) / Treatment of
Uncertainty,” National Academies Press,
http://en.wikipedia.org/wiki/Quantification_of_margins_and_uncertainties
Viscusi, W. Kip, “Risk Equity,” ISSN 1045-6333, (2000)
http://www.law.harvard.edu/programs/olin_center/papers/pdf/294.pdf
Der Khiureghian, Armen and Ove Ditlevsen, “Aleatory or Epistemic? Does It
Matter?” (2007), Risk Acceptance and Communication Workshop, Stanford
Mannes, Albert and Don Moore, “I Know I’m Right, A Behaviourial View of
Overconfidence,” Significance, vol. 10, issue 4, August 2013
Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol.
10, issue 4, August 2013
Tengler, Marek, “Seismic Qualification of NPP Structures, Systems and Other
Components,” Seminar, Nov. 2011
Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol.
10, issue 4, August 2013