Transcript Uncertainty in Engineering A Bayesian Approach

Design For Variation
UCM 2012
Sheffield, UK July 2-4, 2012
Grant Reinman, Senior Fellow, Statistics and Design For Variation
Pratt & Whitney, East Hartford, CT
–
Gaussian Process Emulation
Uncertainty Quantification
Sensitivity Analysis
Design of Experiments
Increase Life
Improve Quality
Improve Producibility
Monte Carlo Simulation
–
Bayesian Model Calibration
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 1 of 26
Pratt & Whitney Engineering
A Passion for Innovation
PurePower® PW1000G Engine
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 2 of 26
Deterministic Design, Uncertain World
Traditional Approach: Empirical Design Margins, Factors of Safety
▲ Usage
▲ Manufacturing
Leading Edge Hole Diameter
PW2000 Hours per Flight, by Operator
OP
AAL
AMT
AMX
CFG
CSH
DAL
ETH
FEA
FIN
LCO
LPR
MXA
NWA
RAM
SIA
TCV
TWA
UAL
UPS
UZB
VIMA
Hole Diameter Value
0
▲ Computational Models
1
2
3
Hours per Flight
4
5
▲ Materials
(x)
Discrepancy (bias) function
0.3
0.4
Time To Creep/Rupture
Rupture
Strain 
Time to Rupture
0.2
0.1
0.0
-0.1
Discrepancy, in.
X
Secondary
Tertiary
-0.2
Primary Creep
-0.3
Stress
0.00
0.02
0.04
0.06
0.08
Concentrated Load, lbs
Reinman, Rev Date 6/19/2012
(x)
0.10
0.12
Time t
0.14
© United Technologies Corporation (2012)
This document contains no technical data subject to the EAR or the ITAR.
Slide 3 of 26
Probabilistic Design, Uncertain World
Why?
0.2
0.1
0.0
Discrepancy, in.
Bias, in.
-0.1
Part-Part
-0.2
 Meanline Miss, by using Bayesian model calibration
process
 Margin Miss, by replacing legacy margins with a
probabilistic model of uncertainty and variability
Age
Die/Config/Batch
-0.3
To Help Prevent Design Iterations due to a Model’s
0.3
0.4
Discrepancy (bias) function
0.00
0.04
0.06
0.08
0.10
0.12
0.14
Concentrated Load, lbs
Concentrated Load, lbs
Supplier
Total Effect of Leading Edge Parameters on Oxidation Life
(Total effect is how much variation in life would be left if you knew precisely the values of all other
parameters.)
To Reduce Cost
45
40
Remove cost from
low-impact features
35
To Maximize Stage Life (Time on Wing)

Rotor life depends on max distress / min life airfoil

Weakest-link structure pervasive in gas turbines

Reducing variation increases rotor life
Total Effect (%)
Focus on important features
Relax requirements on unimportant features
Use Robust Design to reduce sensitivity
30
25
20
15
10
5
w
t_
ss
w
_im t_p
s
p_
s_
ht
d
is
_t
t
rip
_s
trip
cr
s
os
so
ve
rs
sa
_d
le
_f
ia
ee
m
d_
sh
sa
ap
_c
e
ov
er
ag
e
l,a
h_
vg
co
ol_
ho
le
k_
m
et
P_ al
su
pp
ly
co
re
_s
hif
t
h_
le
_im
p
T_
su
pp
ly
Model Inputs
le
)
l(r
l,a
vg
1,
re
Pt
,re
et
a_
f
1,
re
T4
T4
LE
h_
ex
ho
t
le
dia
m
0
Parameter Name
DISTRESS



0.02
CYCLES
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 4 of 26
Probabilistic Design, Uncertain World
Why?
To increase the speed of design parametric studies and optimization
using engineering model emulators
Hours / Run
Seconds / Run
Maximin Latin Hypercube DOE
Inputs Design
Space
•Geometric dimensions
•Loads
•Temperatures
•Material properties
•Heat transfer coefficients
•Etc.
Computer Model
•Structural FEM
•CFD model
•Matlab code
•Fortran code
•Other models
iSight-FD, etc
•Drive the DOE
through the
model
Output in Design
Space
•Stress
•Deflection
•Temperature
•Life
•Performance
•Etc.
GEMSA, GPMSA, etc
•Emulator
•Sensitivity
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 5 of 26
DFV Estimated Benefits
▲ Component-level Design For Variation has yielded an estimated 64%-88% return
on internal investment. The savings resulted from:






Optimized inspection procedures and tolerances
Reduced quality-related analysis and investigation time
Reduced design iterations
Improved reliability
Improved on-time engine deliveries
Improved root cause investigation process
▲ Based on Six Sigma history and internal trends, the return is expected to increase
rapidly in subsequent years
▲ System-level Design For Variation is predicted to yield 40x return on investment
due to
 Achieving system-level performance and reliability goals earlier in the development
cycle
 Shorter development programs
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 6 of 26
Design For Variation (DFV) Strategic Plan
Vision: All Key Modeling Processes will be DFV-enabled
▲
Strategy
☑
☑
☑
Identify Key Processes
Define elements of a DFV-enabled modeling process
Provide Resources under Strategic Initiative
Mechanical Systems and Externals
Carbon Seal Performance
Ball & Roller Bearing Design
FDGS Durability
Externals: Forced Response Analysis
Combustor and Augmentor
Combustor pattern factor
Combustor Liner TMF
Augmentor Ignition Margin Audit
Mid Turbine Frame Robust Design
DFV Infrastructure
Air Systems
Thermal Management Model
Internal Air System Model
Engine Data Matching
(Statistics & Partners)
Emulation, Calibration Software
High Intensity Computing
Parametric Modeling
Optimization
Training
ESW
Communications
Input Data
Tech Support
Fan & Compressor
HFB Producibility
Parametric Airfoil
Compressor Aero Design
Compressor Tip Clearances
Validation Testing
Engine Validation Planning
Vehicle Systems
Probabilistic Ambient
Temp Distribution
Performance Analysis
Performance Monte Carlo Risk Assessment
Engine Test Confidence, Uncertainty
Turbine
Structures
Uncertainty
in Engine System Predictions
Turbine
Blade
Durability
Probabilistic Rotor Lifing
Production
Test
Data Trending and Analysis
Turbine
Vanes
and
BOAS
Durability
Probabilistic Fracture Mechanics
Statistical
Data-match
Rotor
Thermal
Model
Probabilistic HCF
System-Level
Risk
Communication and
Airfoil
LCF
Lifing
Parametric Geometry Simulation Model
Decision
Making
HSE
Combustor
/
Turbine
DFV
Engine Dynamics and Loads
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 7 of 26
10 Elements of a DFV-Enabled Modeling Process
Physics-Based Models


Model Preparation
1. A robust parametric physics-based model
Model Input Variability and Uncertainty Quantification
2. Process for retrieving data needed to quantify variability and uncertainty in model inputs
3.
Process for performing statistical analysis/developing statistical model of input data
a.



Preserve correlations
Model Sensitivity Analysis
4. Process for generating a matrix of space-filling computer experiments (model runs) for emulator development
5. Process for running the computer code at the space-filling design points
6. Process for
a.
Building and validating the model emulator
b.
Performing a variance-based sensitivity analysis
Model Calibration
7. Process for determining what experimental/field data are required for model calibration and measurement uncertainty
(amount, characteristics to be measured, ..)
8. Process for performing Bayesian model calibration: calibrate and bias correct (if needed) and assess residual variation
Uncertainty Analysis
9.

Process for generating a Monte-Carlo sample and driving it through
• Parametric model (if fast enough),
• Model emulator, or
• Bias corrected and calibrated model
Enable Practice
10. Update local ESW and local training. Put in place a process to ensure the model is capable over time.
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 8 of 26
Design For Variation
DEFINE Customer requirements (probabilistic)
0.0
DEFINE
0.1
0.2
0.3
0.4
Five Steps for Executing a DFV-Enabled Process
-3
-2
-1
0
1
2
3
y
ANALYZE
SOLVE
VERIFY
VALIDATE
ANALYZE Quantify model input variation / uncertainty, emulate
and calibrate model, perform sensitivity and uncertainty analyses
SOLVE Identify ‘optimum’ design that satisfies requirements
VERIFY/VALIDATE Variability/Uncertainty model
SUSTAIN Stable system of causes of performance variation
SUSTAIN
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 9 of 26
Design For Variation (DFV): Five Steps
Define Customer Requirements
▲ How do we define the allowable risk of not meeting a requirement?
Risk
0.0
0.1
Safety Impact:
Follow Regulatory Requirements
0.2
0.3
0.4
Explicit customer requirement
-3
System-Level Job Ticket Metric Impact:
Follow flow-down or roll-up process
-2
-1
0
1
2
3
y
Requirement
Engine Certification Test Impact
DEFINE
None of the above
• Previous acceptable experience or other business considerations
• 6 Sigma Criteria
• Solve for the probability or rate that minimizes expected total cost
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 10 of 26
Design For Variation
Input18
Input19
Input20
0.67
0.68
0.75
0.37
0.57
0.48
0.98
0.94
0.71
0.47
0.64
0.04
0.45
0.12
0.09
0.23
0.42
0.05
0.85
0.13
0.22
0.25
0.08
0.33
0.80
0.95
0.07
0.44
0.77
0.61
0.46
0.31
0.28
0.19
0.56
0.81
0.32
0.82
0.11
0.10
0.79
0.86
0.84
0.59
0.20
0.51
0.99
0.06
0.24
0.52
0.41
0.88
0.01
0.60
0.26
0.93
0.96
0.35
0.73
0.89
0.53
0.70
0.63
0.36
0.49
0.83
0.14
0.02
0.65
0.92
0.03
0.69
0.90
0.66
0.29
0.62
0.78
0.00
0.76
0.91
0.21
0.39
0.15
0.87
0.54
0.30
0.38
1.00
0.34
0.17
0.97
0.72
0.58
0.55
0.18
0.43
0.74
0.16
0.40
0.27
0.82
0.80
0.53
0.92
0.74
1.00
0.81
0.10
0.23
0.42
0.38
0.48
0.90
0.04
0.86
0.58
0.96
0.05
0.95
0.34
0.15
0.17
0.37
0.33
0.09
0.56
0.40
0.83
0.47
0.22
0.78
0.89
0.85
0.61
0.07
0.30
0.60
0.97
0.31
0.71
0.70
0.45
0.03
0.00
0.18
0.06
0.16
0.63
0.36
0.75
0.02
0.64
0.24
0.98
0.25
0.66
0.72
0.43
0.65
0.73
0.68
0.20
0.76
0.51
0.01
0.69
0.49
0.93
0.77
0.54
0.11
0.41
0.08
0.67
0.27
0.88
0.12
0.35
0.99
0.55
0.94
0.19
0.14
0.28
0.57
0.29
0.44
0.84
0.13
0.32
0.87
0.26
0.39
0.91
0.52
0.62
0.46
0.79
0.21
0.59
0.92
0.17
0.40
0.89
0.42
0.10
0.97
0.21
0.53
0.74
0.28
0.11
0.85
0.31
0.60
0.05
0.27
0.32
0.43
0.91
0.79
0.78
0.56
0.13
0.72
0.39
0.15
0.45
0.90
0.61
0.67
0.16
0.86
0.65
0.30
0.76
0.46
0.26
0.36
0.20
0.73
0.23
0.47
0.52
0.29
0.62
0.94
0.01
0.55
0.96
0.75
0.99
0.14
0.04
0.35
0.06
0.34
0.77
0.84
0.22
0.48
0.09
0.58
0.18
0.51
1.00
0.59
0.87
0.00
0.19
0.66
0.81
0.93
0.57
0.44
0.69
0.64
0.33
0.38
0.03
0.49
0.63
0.71
0.88
0.07
0.24
0.37
0.95
0.83
0.54
0.12
0.08
0.80
0.82
0.98
0.25
0.70
0.41
0.68
0.02
0.22
0.31
0.88
0.02
0.21
0.96
0.43
0.72
0.17
0.89
0.73
0.23
0.94
0.85
0.53
0.39
0.93
0.41
0.67
0.07
0.99
0.36
0.70
0.69
0.37
0.61
1.00
0.34
0.13
0.75
0.83
0.62
0.10
0.68
0.60
0.55
0.90
0.15
0.33
0.26
0.01
0.49
0.77
0.87
0.27
0.42
0.79
0.91
0.30
0.35
0.00
0.16
0.14
0.08
0.25
0.78
0.81
0.46
0.40
0.52
0.97
0.57
0.06
0.32
0.04
0.92
0.51
0.19
0.20
0.65
0.98
0.29
0.44
0.24
0.84
0.28
0.80
0.76
0.18
0.71
0.03
0.38
0.86
0.05
0.11
0.58
0.74
0.09
0.56
0.45
0.63
0.12
0.47
0.54
0.48
0.82
0.64
0.95
0.66
0.59
Design Space Filling
Experiment Over
Model Input Space
9
8
7
6
0.00
0.20
0.40
0.60
0.80
1.00
5
4
3
2
Real-World Validation Data
1
0
0
1
2
3
4
5
6
7
8
9
10
-1
x
Engineering
Model
0.00
0.20
0.40
0.60
0.80
1.00
Bayesian
Model
Calibration
Model Output
Total Effect of Leading Edge Parameters on Oxidation Life
(Total effect is how much variation in life would be left if you knew precisely the values of all other
parameters.)
45
40
• Parameter uncertainty update
• Bias correction
• Residual variation
35
30
25
20
15
10
5
0
l,a
vg
Pt
,re
l,a
h_
vg
co
ol
_h
ol
e
k_
m
et
P_ al
su
pp
ly
co
re
_s
hi
ft
h_
le
_im
p
T_
su
pp
ly
w
t_
ss
w
le
t_
_i
ps
m
p_
s_
ht
di
_t
st
rip
_s
trip
cr
s
os
so
ve
rs
sa
_d
le
_f
ia
ee
m
d_
sh
sa
ap
_c
e
ov
er
ag
e
Input17
0.02
0.30
0.38
0.12
0.48
0.61
0.90
0.31
0.86
0.62
0.89
0.22
0.69
0.04
0.74
0.47
0.18
0.40
0.75
0.14
0.20
0.41
0.91
0.95
0.43
0.46
0.59
0.51
0.58
0.55
0.70
0.82
0.16
0.57
0.63
0.98
0.17
0.80
0.81
0.88
0.76
0.56
0.94
0.19
0.07
0.33
0.24
0.23
0.27
0.54
0.93
0.08
0.65
0.71
0.92
0.36
0.49
0.87
0.68
0.03
0.72
0.78
0.37
0.83
0.79
0.52
0.73
0.09
0.25
1.00
0.26
0.67
0.42
0.45
0.11
0.34
0.00
0.13
0.84
0.99
0.64
0.10
0.53
0.60
0.21
0.35
0.32
0.29
0.77
0.96
0.05
0.85
0.28
0.66
0.44
0.97
0.01
0.39
0.06
0.15
y
Input16
0.61
0.22
0.55
0.17
0.09
0.98
0.43
0.30
0.05
0.88
0.59
0.99
0.84
0.18
0.87
0.66
0.01
0.93
0.36
0.49
0.14
0.76
0.35
0.41
0.94
0.15
0.34
0.26
0.44
0.16
0.95
0.24
0.23
0.81
0.92
0.58
0.54
0.86
0.91
0.38
0.89
0.75
0.57
0.80
0.85
0.21
0.33
0.90
0.53
0.46
0.60
0.39
0.25
0.56
0.04
0.47
0.20
0.70
0.74
0.73
0.02
0.68
0.96
0.10
0.64
0.32
0.72
0.79
0.37
1.00
0.71
0.42
0.08
0.31
0.48
0.11
0.78
0.62
0.69
0.40
0.52
0.45
0.06
0.77
0.27
0.28
0.67
0.29
0.82
0.03
0.97
0.12
0.19
0.65
0.07
0.13
0.00
0.83
0.63
0.51
)
Input15
0.19
0.49
0.00
0.46
0.89
0.64
0.04
0.17
0.16
0.80
0.26
0.77
0.12
0.36
0.62
0.23
0.57
0.37
0.94
0.01
0.24
0.66
0.72
0.68
0.75
0.33
0.87
0.51
0.84
0.38
0.67
0.09
0.06
0.81
0.56
0.98
0.28
0.44
0.59
0.76
0.40
0.95
0.69
0.31
0.73
0.03
0.79
0.14
0.13
0.02
0.90
0.54
0.61
0.91
0.97
0.55
0.35
0.21
0.78
0.99
0.48
0.18
0.22
0.25
0.20
0.82
0.74
0.10
0.86
0.41
0.83
0.85
0.96
0.71
0.05
0.45
1.00
0.88
0.65
0.15
0.52
0.30
0.42
0.07
0.93
0.63
0.08
0.92
0.60
0.58
0.29
0.27
0.11
0.70
0.53
0.32
0.47
0.34
0.43
0.39
l(r
Input14
0.37
0.34
0.11
0.42
0.82
0.19
0.58
0.18
0.67
0.80
0.73
0.26
0.91
0.01
1.00
0.95
0.54
0.56
0.39
0.12
0.83
0.89
0.38
0.63
0.52
0.17
0.65
0.46
0.88
0.21
0.03
0.76
0.47
0.33
0.78
0.53
0.96
0.93
0.05
0.60
0.71
0.48
0.06
0.30
0.62
0.41
0.74
0.00
0.43
0.04
0.13
0.79
0.22
0.25
0.51
0.75
0.08
0.84
0.23
0.02
0.49
0.86
0.55
0.32
0.64
0.81
0.92
0.90
0.61
0.35
0.29
0.69
0.40
0.15
0.09
0.66
0.36
0.27
0.28
0.70
0.97
0.85
0.94
0.10
0.14
0.77
0.07
0.24
0.16
0.98
0.68
0.72
0.87
0.31
0.57
0.20
0.45
0.44
0.99
0.59
1,
re
Input13
0.82
0.43
0.17
0.24
0.23
0.88
0.68
0.42
0.74
0.11
0.28
0.86
0.38
0.46
0.66
0.73
0.32
0.87
0.12
0.15
0.55
0.77
0.34
1.00
0.06
0.84
0.37
0.39
0.59
0.14
0.41
0.52
0.04
0.78
0.31
0.20
0.08
0.85
0.22
0.76
0.19
0.96
0.91
0.98
0.40
0.13
0.21
0.57
0.10
0.58
0.67
0.53
0.16
0.09
0.02
0.69
0.79
0.35
0.75
0.33
0.36
0.61
0.80
0.72
0.62
0.27
0.93
0.49
0.89
0.81
0.65
0.01
0.71
0.92
0.99
0.97
0.48
0.26
0.05
0.51
0.94
0.44
0.90
0.25
0.95
0.07
0.18
0.47
0.29
0.03
0.54
0.45
0.63
0.83
0.56
0.60
0.00
0.64
0.70
0.30
1,
re
Input12
0.67
0.16
0.30
0.82
0.38
0.23
0.70
0.91
0.14
0.28
0.81
0.71
0.96
0.89
0.20
0.53
0.83
0.13
0.62
0.45
0.80
0.56
0.09
0.32
0.42
0.24
0.94
0.68
0.35
0.07
0.51
0.40
0.47
0.72
0.17
0.19
0.49
0.33
0.11
0.57
0.60
0.31
1.00
0.97
0.77
0.61
0.74
0.00
0.08
0.99
0.03
0.69
0.59
0.44
0.26
0.21
0.66
0.29
0.54
0.37
0.75
0.12
0.86
0.84
0.43
0.22
0.88
0.76
0.95
0.63
0.06
0.36
0.27
0.05
0.55
0.01
0.52
0.73
0.34
0.64
0.02
0.25
0.58
0.85
0.78
0.92
0.39
0.18
0.93
0.65
0.41
0.46
0.79
0.10
0.15
0.48
0.04
0.87
0.90
0.98
T4
Input11
0.39
0.46
0.63
0.13
0.82
0.17
0.62
0.20
0.44
0.02
0.77
0.22
0.73
0.67
0.52
0.89
0.42
0.09
0.97
0.14
0.29
0.98
0.87
0.10
0.41
0.19
0.84
0.36
0.68
0.64
0.92
0.27
0.99
0.18
0.55
0.90
0.38
0.15
1.00
0.74
0.85
0.32
0.69
0.30
0.25
0.80
0.51
0.23
0.96
0.66
0.12
0.37
0.94
0.33
0.05
0.78
0.07
0.86
0.54
0.81
0.47
0.08
0.04
0.00
0.71
0.53
0.06
0.31
0.72
0.59
0.43
0.93
0.58
0.79
0.88
0.83
0.16
0.26
0.65
0.45
0.24
0.76
0.34
0.95
0.03
0.61
0.56
0.49
0.57
0.28
0.35
0.75
0.01
0.11
0.60
0.21
0.70
0.40
0.91
0.48
di
am
Input10
0.66
0.46
0.62
0.88
0.03
0.20
0.39
0.84
0.21
0.28
0.41
0.48
0.78
0.67
0.56
0.61
0.59
0.38
0.45
0.68
0.18
0.36
0.75
0.22
0.15
0.14
0.47
0.65
0.83
0.32
0.63
0.96
0.33
0.02
0.97
0.81
0.98
0.57
0.08
0.42
0.00
0.40
0.25
0.60
0.64
0.73
0.09
0.93
0.27
0.82
0.71
0.89
0.52
0.30
0.26
0.13
0.49
0.23
0.79
0.54
0.06
0.53
0.24
0.11
0.94
0.17
0.69
0.95
0.99
0.90
0.31
0.87
0.12
1.00
0.43
0.35
0.80
0.05
0.19
0.91
0.34
0.01
0.55
0.04
0.44
0.85
0.92
0.76
0.77
0.74
0.72
0.70
0.58
0.07
0.16
0.86
0.51
0.37
0.10
0.29
et
a_
f
Input9
0.89
0.65
0.39
0.23
0.26
0.28
0.95
0.83
0.85
0.30
0.97
0.74
0.82
0.35
0.21
0.87
0.55
0.75
0.81
0.46
0.86
0.61
0.52
0.22
0.77
0.73
0.71
0.33
0.69
0.59
0.09
0.47
0.12
0.37
0.16
0.92
0.62
0.63
0.19
0.25
0.68
0.72
0.24
0.13
0.20
0.48
0.58
0.36
0.79
0.45
0.17
0.53
0.56
0.88
0.08
0.27
0.34
0.06
0.40
0.10
0.66
0.54
0.29
0.76
0.05
0.02
0.91
0.18
0.41
0.32
0.98
0.67
0.51
0.99
0.57
0.07
0.96
0.15
0.43
0.94
1.00
0.01
0.44
0.90
0.64
0.93
0.84
0.14
0.78
0.49
0.80
0.42
0.31
0.60
0.70
0.04
0.03
0.00
0.11
0.38
T4
Input8
0.09
0.88
0.63
0.75
0.43
0.55
0.71
0.40
0.41
0.68
0.02
0.57
0.79
0.12
0.65
0.17
0.78
0.60
0.83
0.80
0.77
0.15
0.94
0.92
0.38
0.96
0.45
0.05
0.08
0.22
0.01
0.66
0.39
0.90
0.14
0.54
0.48
0.30
0.26
0.25
0.00
0.44
0.13
0.67
0.47
0.76
0.59
0.42
1.00
0.99
0.10
0.53
0.36
0.20
0.81
0.28
0.19
0.64
0.89
0.31
0.51
0.18
0.69
0.35
0.73
0.33
0.86
0.03
0.62
0.97
0.72
0.16
0.11
0.29
0.07
0.21
0.58
0.06
0.49
0.61
0.24
0.87
0.32
0.70
0.04
0.95
0.91
0.85
0.23
0.84
0.52
0.82
0.56
0.37
0.98
0.74
0.27
0.46
0.34
0.93
Total Effect (%)
Input7
0.38
0.98
0.51
0.91
0.87
0.64
0.00
0.32
0.83
0.03
0.70
0.16
0.30
0.34
0.80
0.18
0.11
0.08
0.65
0.27
0.77
0.21
0.92
0.35
0.53
0.74
0.86
0.07
0.78
0.67
0.93
0.81
0.45
0.89
0.79
0.61
0.54
0.23
0.43
0.84
0.59
0.36
0.40
0.44
0.05
0.90
0.28
0.63
0.62
0.46
0.49
0.95
0.52
0.41
0.56
0.20
1.00
0.12
0.66
0.01
0.09
0.33
0.99
0.25
0.97
0.31
0.76
0.22
0.48
0.47
0.68
0.13
0.39
0.37
0.57
0.14
0.06
0.42
0.71
0.85
0.60
0.96
0.15
0.72
0.24
0.10
0.88
0.26
0.69
0.73
0.29
0.17
0.82
0.04
0.02
0.75
0.58
0.19
0.55
0.94
h_
ex
t
Input6
0.58
0.99
0.97
0.62
0.42
0.23
1.00
0.33
0.08
0.71
0.77
0.96
0.65
0.57
0.41
0.80
0.22
0.01
0.53
0.54
0.04
0.09
0.40
0.95
0.32
0.46
0.74
0.73
0.63
0.44
0.00
0.13
0.34
0.94
0.86
0.18
0.78
0.39
0.79
0.67
0.25
0.10
0.49
0.98
0.31
0.19
0.38
0.52
0.68
0.11
0.17
0.36
0.21
0.69
0.06
0.27
0.35
0.15
0.26
0.02
0.83
0.75
0.05
0.47
0.82
0.37
0.16
0.88
0.12
0.76
0.60
0.85
0.91
0.89
0.07
0.03
0.64
0.20
0.84
0.93
0.87
0.28
0.24
0.43
0.81
0.48
0.14
0.90
0.45
0.72
0.30
0.56
0.92
0.66
0.61
0.51
0.59
0.55
0.29
0.70
e
Input5
0.07
0.17
0.02
0.59
0.45
0.10
1.00
0.73
0.88
0.66
0.58
0.27
0.36
0.28
0.79
0.56
0.13
0.12
0.92
0.35
0.78
0.43
0.19
0.98
0.04
0.16
0.15
0.72
0.90
0.22
0.46
0.01
0.65
0.55
0.83
0.24
0.89
0.08
0.51
0.06
0.64
0.75
0.70
0.37
0.48
0.96
0.85
0.74
0.63
0.23
0.94
0.84
0.67
0.20
0.05
0.34
0.86
0.14
0.41
0.30
0.99
0.38
0.54
0.40
0.25
0.03
0.42
0.68
0.53
0.49
0.31
0.00
0.33
0.47
0.76
0.77
0.91
0.81
0.71
0.95
0.69
0.61
0.82
0.32
0.11
0.29
0.18
0.87
0.52
0.44
0.21
0.57
0.60
0.97
0.39
0.09
0.62
0.26
0.80
0.93
ho
l
Input4
0.65
0.66
0.83
0.61
0.85
0.16
0.07
0.10
0.03
0.48
0.94
0.86
0.81
0.18
0.38
0.04
0.36
0.41
0.77
0.23
0.80
0.69
0.59
0.30
1.00
0.92
0.52
0.90
0.46
0.13
0.20
0.12
0.22
0.91
0.49
0.14
0.43
0.96
0.09
0.54
0.31
0.53
0.71
0.42
0.72
0.78
0.98
0.51
0.02
0.27
0.40
0.29
0.44
0.74
0.37
0.76
0.60
0.70
0.93
0.15
0.47
0.79
0.24
0.88
0.21
0.35
0.87
0.00
0.28
0.25
0.64
0.97
0.56
0.58
0.95
0.55
0.89
0.19
0.26
0.57
0.67
0.34
0.01
0.39
0.45
0.99
0.17
0.75
0.33
0.08
0.05
0.62
0.11
0.06
0.32
0.73
0.84
0.82
0.68
0.63
LE
Input3
0.43
0.09
0.33
0.15
0.93
0.80
0.85
0.30
0.72
0.40
0.12
0.98
0.74
0.57
0.78
0.59
0.88
0.51
0.49
0.64
0.25
0.68
0.89
0.55
0.23
0.01
0.99
0.92
0.73
0.97
0.21
0.17
0.19
0.02
0.61
0.10
0.29
0.27
0.47
0.28
0.11
0.75
0.35
0.39
0.08
0.70
0.67
0.42
0.38
0.86
0.65
0.06
0.04
1.00
0.54
0.03
0.53
0.46
0.16
0.22
0.37
0.18
0.77
0.71
0.83
0.87
0.58
0.07
0.45
0.66
0.44
0.79
0.36
0.05
0.00
0.69
0.14
0.26
0.90
0.31
0.48
0.56
0.76
0.81
0.24
0.13
0.20
0.82
0.63
0.34
0.32
0.62
0.91
0.95
0.52
0.94
0.60
0.84
0.96
0.41
Output2
Input2
0.38
0.01
0.89
0.96
0.24
0.66
0.26
0.43
0.31
0.47
0.09
0.83
0.87
0.04
0.64
0.85
0.62
0.77
0.11
0.97
0.40
0.90
0.94
0.80
0.32
1.00
0.29
0.98
0.75
0.10
0.49
0.15
0.72
0.81
0.07
0.69
0.67
0.00
0.19
0.84
0.46
0.22
0.28
0.71
0.08
0.55
0.73
0.60
0.18
0.39
0.13
0.03
0.42
0.16
0.30
0.25
0.51
0.17
0.37
0.41
0.52
0.76
0.06
0.20
0.82
0.57
0.02
0.34
0.86
0.45
0.48
0.91
0.59
0.56
0.33
0.23
0.70
0.65
0.99
0.95
0.58
0.93
0.44
0.63
0.12
0.53
0.74
0.92
0.54
0.88
0.36
0.21
0.68
0.61
0.78
0.35
0.79
0.27
0.05
0.14
Output1
Input1
ANALYZE Quantify model input variation & uncertainty, emulate & calibrate model, perform sensitivity
and uncertainty analyses
Model Inputs
0.00
0.20
0.40
0.60
0.80
Accounting for
uncertainty in
• Model input
• Model itself
1.00
Parameter Name
Run Experiment
Through
Engineering
Model
Develop Model
Emulator,
Sensitivity Analysis
Refine
Distributions
of Important
Model Inputs
Perform
Bayesian
Model
Calibration
Run
Real World
Uncertainty
Analysis
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 11 of 26
Design For Variation
ANALYZE : Key Technologies
3. Variance-Based Sensitivity Analysis
Total Effect of Leading Edge Parameters on Oxidation Life
(Total effect is how much variation in life would be left if you knew precisely the values of all other
parameters.)
45
1. Latin Hypercube Experimental Designs
40
35
Total Effect (%)
1.0
0.8
0.6
30
25
20
15
10
5
Pt
,re
l,a
h_
vg
co
ol
_h
ol
e
k_
m
et
P_ al
su
pp
ly
co
re
_s
hi
ft
h_
le
_i
m
p
T_
su
pp
ly
w
t_
ss
w
le
t_
_i
ps
m
p_
s_
ht
d
is
_t
t
rip
_s
tri
ps
cr
os
so
ve
rs
sa
_d
le
_f
ia
ee
m
d_
sh
sa
ap
_c
e
ov
er
ag
e
)
vg
l(r
l,a
1,
re
1,
re
Model Inputs
T4
T4
h_
LE
ex
ho
t
le
di
am
0.2
et
a_
f
0
0.4
Parameter Name
0.0
0.2
0.4
0.6
0.8
1.0
4. Bayesian Model Calibration
F
14
0.0
w
12
2. Gaussian Process Emulators
10
X
7
w
8
8
y
6
Deflection, in
9
Simple function
f(x) = x + 3sin(x/2)
4
6
4
2
y
5
2
3
3
4
E (psi) x 10^7
0
2
1
0
1
2
3
4
5
6
7
8
9
10
6
7
8
9
10
0.02
12
0.00
0
0.04
0.06
0.08
0.10
0.12
Y
0.14
Concentrated Load, lbs
(x)
Discrepancy (bias) function
-1
1
0.4
0.3
0.2
-0.1
4
2
1
-0.2
2
3
2
0.1
4
3
0.0
5
4
Discrepancy, in.
6
5
8
7
6
6
8
7
Deflection, in.
8
y
9
10
x
9
2.0
0
1
2
3
4
5
6
-1
x
7
8
9
10
-1
2
3
4
5
3.0
3.5
4.0
-0.3
1
0
0
0.00
x
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.14
Concentrated Load, lbs
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
2.5
Modulus E (psix10^7)
0
0
Concentrated Load, lbs
This document contains no technical data subject to the EAR or the ITAR.
Slide 12 of 26
Design For Variation
SOLVE Identify optimum design that satisfies requirements
▲ Performance characteristic y = f (x1, x2, …, xp) depends on p inputs
▲ The variance of y can be approximated by
 2y
2
2
2
f
f
f
   x   x2    x   x2      x   x2
1
2
p
 p
 1
 2
▲ We can reduce  y2 by
1. Reducing  x2 : the variance in the inputs x1, x2, …, xp
i
2. Reducing  f : the sensitivity of y to variation in x1, x2, ... , xp
 xi
SOLVE
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 13 of 26
Design for Variation
SOLVE: Robust Design Strategies
Noise Factors
•
•
•
•
Input Signal
• Alter/smooth
• Selectively block
Filter
Isolate
Reduce at source
Inoculate (anneal, heat treat)
System
Output Response
• Calibrate
• Average
Control Factors
Adapted from: Jugulum, R. and Frey, D.
(2007). Toward a taxonomy of concept
designs for improved robustness, Journal
of Engineering Design, 18:2, 139 - 156
• Robust optimization
• Material change
• Create multiple operating modes
SOLVE
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 14 of 26
Design For Variation
VERIFY/VALIDATE Assumptions made in variability and uncertainty modeling
▲ VERIFY/VALIDATE includes
– Data collection and analysis to validate model input probability distributions
Manufacturing process data
Material property data
Temperatures, pressures, rotor speeds, airflows
Flight characteristics (e.g. length, T2 at takeoff, taxi time, ..)
– Additional calibration of physics-based models
– Trending in-service parts (wear, performance, etc) where feasible to validate
models and their inputs
VAL/VER
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 15 of 26
Design For Variation
SUSTAIN Stable system of causes of performance variation
▲ The SUSTAIN phase requires process control to ensure stable and consistent
distributions over time
– Manufacturing
– Assembly
– Acceptance Testing
▲ Process Certification is vitally important
– Sustaining capabilities to meet design requirements
– Identifying production & design improvement opportunities
▲ Design Sensitivity and Uncertainty Analyses indicate where process control resources
should be focused
I-MR Chart of Measured Value of a Key Characteristic
Individual Value
35
Process Capability of Measured Value of a Key Characteristic
U C L=34.19
LSL
USL
30
25
P rocess D ata
LS L
10.00000
Target
*
USL
40.00000
S ample M ean
25.48909
S ample N
30
S tD ev (Within)
2.89922
S tD ev (O v erall)
3.30970
W ithin
O v erall
_
X=25.49
20
LC L=16.79
15
1/4/2006
P otential (Within) C apability
Cp
1.72
C PL
1.78
C PU
1.67
C pk
1.67
C C pk 1.72
U C L=10.69
10.0
Moving Range
1/7/2006 1/10/2006 1/13/2006 1/16/2006 1/19/2006 1/22/2006 1/25/2006 1/28/2006 1/31/2006
Date
7.5
O v erall C apability
5.0
Pp
PPL
PPU
P pk
C pm
__
M R=3.27
2.5
0.0
LC L=0
1/4/2006
1/7/2006 1/10/2006 1/13/2006 1/16/2006 1/19/2006 1/22/2006 1/25/2006 1/28/2006 1/31/2006
Date
12
16
20
24
28
32
36
40
1.51
1.56
1.46
1.46
*
SUSTAIN
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 16 of 26
Design For Variation
Systematic Process for Designing for and Managing Uncertainty and Variability
▲ Establish probabilistic design requirements
▲ Emulate, calibrate engineering models
▲ Solve for design that meets probabilistic requirements
– Look for opportunities for making design less sensitive to variation
▲ Validate and sustain model
▲ Write Engineering Standard Work, develop local training
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 17 of 26
Design For Variation
What’s New in 2012?
▲ Additional Training Courses Developed
▲ Automated Multi-physics Workflow
▲ System-Level Design
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 18 of 26
Design For Variation – What’s New?
Infrastructure: Enabling Design For Variation
▲ Software
–
–
Emulation, Sensitivity Analysis, Model Calibration
Statistical Analysis, Monte Carlo Simulation, Optimization
▲ High Performance Computing Resources
▲ Training
–
–
–
–
–
INTRODUCTION
PRACTITIONERS I: SENSITIVITY ANALYSIS, EMULATION, AND
DOE
PRACTITIONERS II: ISIGHT-FD FOR SENSITIVITY AND
UNCERTAINTY ANALYSIS
PRACTITIONERS III: MODEL CALIBRATION AND UNCERTAINTY
ANALYSIS
MANAGERS: INTRODUCTION, REVIEW CHECKLIST
▲ Communication
–
Wiki, Website, Meetings
▲ Input Data Quality and Availability
– Process Capability, Material Properties
– Systems Performance, Mission Analysis
▲ Engineering Standard Work
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 19 of 26
Design For Variation - What’s New?
Multi-discipline Automated Workflows
• Link disciplines: Aero, Thermal, Structures, Materials, Design
• Link components
• Enable probabilistic analyses, optimization
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
20
Slide 20 of 26
What’s New - PADME Program
System Level Probabilistic Design & Validation of Engines
• Determine optimum path to reduce risk
Nominal Design
Performance
Concept
 Test changes
Bad
Performance
Requirement
Requirement
Concept
 Design changes
Confidence bound
Uncertainty
Uncertainty
Bounds on Design:
Risk & Confidence
Good
• Determine design drivers
Probability Distribution (pdf)
• Quantify uncertainty/risk in system-level metrics
Job Ticket Metric
• PADME is a System-Level Extension of Design For Variation
Service
Validation Nominal/Expected
Design
Design
Demo
Design
Prototype
Production Maintenance &
Customer Use
Test
Service
Leveraged Technologies: Design For Variation
Strategy
☑ Identify Key Processes
☑ Define elements of a DFV-enabled modeling process
☑ Provide Resources under Strategic Initiative
• PADME Goals
• Improve Mature vs. EIS Performance Gap by 33%
• Improve Mature vs. EIS Reliability Gap by 33%
• Reduce EVP Time by up to 50%
Mechanical Systems and Externals
Carbon Seal Performance
Ball & Roller Bearing Design
FDGSDurability
Externals: Forced Response Analysis
Combustor and Augmentor
Combustor pattern factor
Combustor Liner TMF
Augmentor Ignition Margin Audit
Mid Turbine Frame Robust Design
Air Systems
Thermal Management Model
Internal Air System Model
Engine Data Matching
DFV Infrastructure
Validation Testing
Engine Validation Planning
(Statistics & Partners)
Sens / Uncert / Opt Software
High Perf Computing
Training
ESW
Communications
Input Data
Tech Support
Vehicle Systems
Probabilistic Ambient
Temp Distribution
Performance Analysis
Performance Monte Carlo Risk Assessment
Engine Test Confidence, Uncertainty
Turbine
Uncertainty in Engine System Predictions
Turbine Blade Durability
Structures
Turbine Vanes and BOAS Durability Production Test Data Trending and Analysis
Probabilistic HCF
Rotor Thermal Model
Parametric Geometry Simulation Model
Airfoil LCF Lifing
Engine Dynamics and Loads
Page 12
Fan & Compressor
HFB Producibility
Parametric Airfoil
Compressor Aero Design
EAR Export Classification: ECCN: EAR 99
PADME: Probabilistic Analysis and Design of Materials and Engines
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
21
Slide 21 of 26
PADME Vision
Concept
Concept
Service
Validation Nominal/Expected
Design
Performance
Requirement
Requirement
Bad
Confidence bound
Uncertainty
Uncertainty
Bounds on Design:
Risk & Confidence
Good
Probability Distribution (pdf)
Fuel Consumption
Delay/Cancellation Rate
Weight
Cost
Job Ticket Metric
Entire Engine Life Cycle Governed By Uncertainty Quantification and Management
Nominal Design
Performance
Demo
Design
Design
Prototype
Test
Production Maintenance &
Customer Use
Service
Rigorously Manage Uncertainty Throughout Life Cycle, Target
Validation Testing to Address Largest Sources of Uncertainty
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
22
Slide 22 of 26
PADME: Manage Uncertainty Throughout Engine Life Cycle
PADME
Governed
By Optimized
System-Level
Quantification
of Uncertainty
Enables
Trades on Networks
System Level Metrics
Populated By Calibrated Component-Level Emulators
Concept (CI/CO)
Methods
Design (PD/DD)
Validation (V&V)
Service
Bayesian Network Development and Updating
Sensitivity Analysis, Bayesian Model Calibration
Large Scale Optimization Under Uncertainty
Robust Design, Real Options, Quantitative TRLs
DFV-Enabled
Design Models
Redesign
System Reliability
Bayes Network
Parametric
Rel. Network
Redesign
Parametric
Rel. Network
System
Performance
Bayes Network
Parametric
Perf. Network
Parametric
Perf. Network
Probability Distribution (pdf)
Uncertainty
Bounds on Design:
Risk & Confidence
Performance
Requirement
Nominal
Surprises;
Design
New
Performance
Test Data
Concept
Design
Prototype
EVP
Optimizer
Test
Production Maintenance &
Customer Use
Probability Distribution (pdf)
EVP
Optimizer
Re-optimize EVP
Performance
Requirement
Nominal
Surprises;
Design
New
Performance
Test Data
Concept
Design
Prototype
DFV-Enabled
Design Models
Production Maintenance &
Customer Use
= Needs DARPA Support
Parametric
Perf. Network
Parametric
Perf. Network
Nominal
Surprises;
Design
New
Performance
Test Data
Design
Prototype
UBL /
Prognosis
Test
Performance
Requirement
Production Maintenance &
Customer Use
Uncertainty
Bounds on Design:
Risk & Confidence
Performance
Requirement
Nominal
Surprises;
Design
New
Performance
Test Data
Concept
Design
Prototype
Production Maintenance &
Customer Use
= Supported by P&W or Prior Programs
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
DFV-Enabled
Design Models
Redesign
Parametric
Rel. Network
Uncertainty
Bounds on Design:
Risk & Confidence
Concept
Bayes ian
Uncertainty
Update
Redesign
Parametric
Rel. Network
EVP
Optimizer
Test
Uncertainty
Bounds on Design:
Risk & Confidence
Bayesian
Uncertainty
Update
Probability Distribution (pdf)
PD/DD
Emulators
Component & SubSystem Models
Bayesian
Uncertainty
Update
Probability Distribution (pdf)
Bayesian
Uncertainty
Update
This document contains no technical data subject to the EAR or the ITAR.
23
Slide 23 of 26
PADME Strategy
▲ Uncertainty-Based Design Approach Relies on Calibration of
Multivariate Aero-Thermal-Structural Models Using Highly
Instrumented Engine
Deterministic
Design
Deterministic
Redesign
Engine Test
Deterministic
Redesign
Engine Test
Engine Test
crack
Legacy
Approach
oxidation
Probabilistic
Design
Robust
Design
Engine
Endurance Test
R&D Rig/Engine Test
DFVPADME
Approach
Gas Temps
Combustor
1s t
Vane
Blade Temps
1s t
Blade
Gas Temps
2nd
Vane
Blade Temps
2nd
Blade
Gas Temps
Exit
Vane
EAR Export Classification: EAR 99
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 24 of 26
Design For Variation – For More Information
▲ Statistical Engineering Issue
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 25 of 26
Design For Variation
▲ Goal: quantify, understand, and control the risk of not meeting design
criteria or exceeding thresholds
▲ “The revolutionary idea that defines the boundary between modern times
and the past is the mastery of risk: the notion that the future is more than a
whim of the gods and that men and women are not passive before nature.”
– Peter Bernstein, “Against the Gods: The remarkable story of risk”
Model
Prediction
Design
Criteria
True Process Value
© United Technologies Corporation (2012)
Reinman, Rev Date 6/19/2012
This document contains no technical data subject to the EAR or the ITAR.
Slide 26 of 26