Testing safety-critical software systems

Marcos Mainar Lalmolda Quality Assurance and Testing 20 th November 2009

Contents

 What a safety-critical software system is  Standards  Programming features and languages  Approaches on design  Testing  Conclusion

What a safety-critical software system is

  A safety-critical software system is a computer system whose failure or malfunction may severely harm people's lives, environment or equipment.

Some fields and examples:  Medicine (patient monitors)  Nuclear engineering (nuclear power station control)  Transport (railway systems, cars anti-lock brakes)  Aviation (control systems: fly-by-wire)  Aerospace (NASA space shuttle)  Civil engineering (calculate structures)  Military devices  Etc.

Safety-critical Standards

  Industries specific  Medical device software: IEC 62304  Nuclear power stations: IEC 60880  Aerospace: AS9100A  Airbone: DO178B  … Scale of 5 safety integrity levels: 4 is very high, 0 not safety related.

 Safety engineering

Programming features and languages (I)

 General principle: Try to keep the system as simple as possible.

 Programming features not recommended:  Pointers and dynamic memory allocation/deallocation.

 Unstructured programming (gotos)  Variant data  Implicit declaration and initialisation  Recursion  Concurrency and interrupts

Programming features and languages (II)

 Features which increase reliability:  Strong typing  Run time constraint checking  Parameter checking  Language to be avoided: C  Language recommended: Ada  Ada subset for safety-critical software: SPARK  Other languages: increased overhead

Approaches on design

 Formal methods  Assume that errors exist and design prevention and recovery mechanisms.

 “Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987).

Testing safety-critical software systems (I)

     Basic idea: Identify hazards as early as possible in the development life-cycle and try to reduce them as much as possible to an acceptable level.

Remember: Always test software against specifications!

Independent verification required If formal methods have been used then formal mathematical proof is a verification activity.

Already known techniques used for typical systems  White box testing  Black box testing  Reviews  Static analysis  Dynamic analysis and coverage

Testing safety-critical software systems (II)

 Specific procedures and techniques from safety engineering:  Probabilistic risk assessment (PRA)  Failure modes and effects analysis (FMEA)  Fault trees analysis (FTA)  Failure mode, effects and criticality analysis (FMECA)  Hazard and operatibility analysis (HAZOP)  Hazard and risk analysis  Cause and effect diagrams ( or Ishikawa diagrams) aka fishbone diagrams

Probability Risk Assessment

Hazard Severity Probability Risk

Risk Criteria Tolerable?

No Yes Risk Reduction Measures *From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger, February 16, 2007, Institute of Computer Technology, Wien 10

Fault tree analysis (FTA)

 A graphical technique that provides a systematic description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure).  An undesired effect is taken as the root of a tree of logic  Each situation that could cause that effect is added to the tree as a series of logic expressions.

 Events are labelled with actual numbers about failure probabilities.

 The probability of the top-level event can be determined using mathematical techniques.

An example of a Fault tree

*From http://syque.com/quality_tools/toolbook/FTA/how.htm

Conclusions

 Complex subject  Suitably trained and experienced people are key to the success of any software development.

 Main objective of testing techniques: minimise risk of implementation errors.

 Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple.

Questions

¿?

References

       Wikipedia. http://en.wikipedia.org

IPL Information Processing Ltd, An Introduction to Safety Critical Systems , Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0820 IPL Information Processing Ltd, An Introduction to Software Testing , Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0826 Evangelos Nikolaropoulos,

Testing safety-critical software

, Hewlett-Packard Journal, June 1997. http://findarticles.com/p/articles/mi_m0HPJ/is_n3_v48/ai_19540814/?tag=content;col1 Frederick P. Brooks, Jr. ,

No Silver Bullet: Essence and Accidents of Software Engineering

, 1986.

Andreas Gerstinger,

Safety-Critical Computer Systems – Open Questions and Approaches presentation

, February 16, 2007, Institute of Computer Technology, Wien.

Fault Tree Analysis: How to understand it. http://syque.com/quality_tools/toolbook/FTA/how.htm