NASA: Engineering Space Exploration

National Aeronautics and Space Administration

Presented to the Conference on Quality in the Space and Defense Industries 2007 Cape Canaveral, Florida March, 2008

www.nasa.gov

C. Herbert Shivers, PhD, PE, CSP Deputy Director, Safety and Mission Assurance Directorate NASA/Marshall Space Flight Center

From

National Aeronautics and Space Administration

Determinism to “Probabilism”

www.nasa.gov

Changing our mindsets, or why PTC isn’t an easy sell - yet

My Muddled Mind Deterministic Design Robust Design

?

Probabilistic Design Safety Factors SDF Block Diagrams, Redundancy Predictive Models & DOE PRA Algorithms Life Cycle Prediction Systems of Systems PDF Models FMECA, FTA, RAM Design SWP DFMR Diagnostics & Prognostics PD

System Safety Tools Feed Risk Informed Decision Making

Method Technique Application Bryan O’Connor, Chief, Safety and Mission Assurance, NASA SEPT 8, 2007, Huntsville, AL Legend: FMEA HA ESD ETA FTA RBD Failure Modes & Effects Analysis Hazard Analysis Event Sequence Diagram Event Tree Analysis Fault Tree Analysis Reliability Block Diagram

S&MA in Ares Design – Summary • Using the Ares quantitative safety and reliability requirement to enforce the “design for reliability and safety” paradigm shift • Improving system safety by using a functional analysis system approach to model and understand integrated system failures similar to the Shuttle foam problem.

• Improving design reliability by using probabilistic engineering physics-based modeling • Evaluating and understating design uncertainty and design margins using probabilistic engineering techniques Excerpted from Chris Cianciola, S&MA, MSFC, CQSDI March 2008

Uncertainty Quantification and Risk Assessment risk-informed decision making Uncertainties sources manufacturing, storage, aging, use environments, and scenarios Uncertainties types randomness, lack of knowledge

M

odel to predict uncertainties in systems

U

ncertainties outside acceptable limits?

U

ncertainties produce inadequate margins?

M

ost important contributors to uncertainty (sensitivity analyses)?

Physics model form uncertainty Sensitivity Analysis Only several contribute most of the uncertainty in system response Requirements Uncertainty

Probabilistic Models Limitations

One of many decision variables Subjective information - scientific and engineering judgment is necessary Known unknowns and unknown unknowns exist Resources are limited Results uncertainty and credibility – challenges decision makers Social, economic, and political factors exist

Credibility

• Is the methods framework credible?

• Address the credibility concern with verification and validation

Special study data

• Start with robustness and margin in design • A 30% design margin is not plausible for space flight – it’s just too expensive • Our margins are so thin that we really need to investigate margin • In our case we want to whittle the margin down as much as possible but still maintain the robustness we need • Getting those design margin trades into the system analysis is critical NASA special study data 2006

Special study data

• We still use deterministic design and build margin into designs even though we have the computational capabilities to us PT methods • We do not have probabilistic design methods in place • Probabilistic methods will have to be an investment by NASA into the long term technology base and tools • First you need a proper deterministic goal and then you can successfully apply a probabilistic model • Numerical Propulsion System Simulation is something that must be implemented to get this country to Mars • The entire thought process needs to change, this is not a deterministic world appropriately – everything is probabilistic • In the deterministic method there is no role for uncertainty or margin • Right now technology is so fast we are dumping technology on people who are not educated to use these technologies NASA special study data 2006

Special study data

• Redundancy is a part of our culture right now because it is much easier than looking for a solution using margin • When you consider uncertainty in the design phase you are able to examine the trade space more efficiently and thoroughly.

• The nature of the probabilistic skills dealing in design is very different from the traditional PRA. NASA special study data 2006

Why PTC Now?

• Design engineers don’t understand how to account for uncertainty in the design process and how to explore the trades throughout • In the past we’ve had so much money and time for testing, but we don’t have this anymore • Apollo did

14,000

tests on the LEM, do you think we could do that now?

• We must trade ability to test versus desired reliability demonstration NASA special study data 2006

Other Notable Thoughts

• Probabilistic requirements ultimately relate to risk in achieving a level of performance or safety. • The most important reason for using a structured approach is to ensure that the potential failure modes of a design are identified early in the design process to better understand the risks.

• An understanding of risks due to incomplete or inaccurate modeling of a design and the operational environment, or due to uncertainty related to the data used in a model is necessary so that a requirement can be verified with confidence. Surendra N. Singhal, Engineering Directorate, Marshall Space Flight Center, Sept. 18, 2007

A Way Forward

• Need a coherent institutionalized effort • Need a regular training of the entire Center staff (many unaware of PT and its benefits) • Need a core group where people and projects can go for help

PT Selling Points



Utilizes Physics based behavioral model



Considers inherent uncertainties, modeling uncertainties, lack of data, human error, measurement error



Compensates for unknowns using statistical methods



Utilizes past performance data to develop behavioral model



Quantifies safety measures



Qualifies prediction accuracy

Robert J. Kuper, Executive for Reliability and Quality, Systems Engineering Conference, 23 October 2002