Testing safety-critical software systems Marcos Mainar Lalmolda Quality Assurance and Testing 20th November 2009
Download ReportTranscript Testing safety-critical software systems Marcos Mainar Lalmolda Quality Assurance and Testing 20th November 2009
Testing safety-critical software systems
Marcos Mainar Lalmolda Quality Assurance and Testing 20 th November 2009
Contents
What a safety-critical software system is Standards Programming features and languages Approaches on design Testing Conclusion
What a safety-critical software system is
A safety-critical software system is a computer system whose failure or malfunction may severely harm people's lives, environment or equipment.
Some fields and examples: Medicine (patient monitors) Nuclear engineering (nuclear power station control) Transport (railway systems, cars anti-lock brakes) Aviation (control systems: fly-by-wire) Aerospace (NASA space shuttle) Civil engineering (calculate structures) Military devices Etc.
Safety-critical Standards
Industries specific Medical device software: IEC 62304 Nuclear power stations: IEC 60880 Aerospace: AS9100A Airbone: DO178B … Scale of 5 safety integrity levels: 4 is very high, 0 not safety related.
Safety engineering
Programming features and languages (I)
General principle: Try to keep the system as simple as possible.
Programming features not recommended: Pointers and dynamic memory allocation/deallocation.
Unstructured programming (gotos) Variant data Implicit declaration and initialisation Recursion Concurrency and interrupts
Programming features and languages (II)
Features which increase reliability: Strong typing Run time constraint checking Parameter checking Language to be avoided: C Language recommended: Ada Ada subset for safety-critical software: SPARK Other languages: increased overhead
Approaches on design
Formal methods Assume that errors exist and design prevention and recovery mechanisms.
“Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987).
Testing safety-critical software systems (I)
Basic idea: Identify hazards as early as possible in the development life-cycle and try to reduce them as much as possible to an acceptable level.
Remember: Always test software against specifications!
Independent verification required If formal methods have been used then formal mathematical proof is a verification activity.
Already known techniques used for typical systems White box testing Black box testing Reviews Static analysis Dynamic analysis and coverage
Testing safety-critical software systems (II)
Specific procedures and techniques from safety engineering: Probabilistic risk assessment (PRA) Failure modes and effects analysis (FMEA) Fault trees analysis (FTA) Failure mode, effects and criticality analysis (FMECA) Hazard and operatibility analysis (HAZOP) Hazard and risk analysis Cause and effect diagrams ( or Ishikawa diagrams) aka fishbone diagrams
Probability Risk Assessment
Hazard Severity Probability Risk
Risk Criteria Tolerable?
No Yes Risk Reduction Measures *From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger, February 16, 2007, Institute of Computer Technology, Wien 10
Fault tree analysis (FTA)
A graphical technique that provides a systematic description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure). An undesired effect is taken as the root of a tree of logic Each situation that could cause that effect is added to the tree as a series of logic expressions.
Events are labelled with actual numbers about failure probabilities.
The probability of the top-level event can be determined using mathematical techniques.
An example of a Fault tree
*From http://syque.com/quality_tools/toolbook/FTA/how.htm
Conclusions
Complex subject Suitably trained and experienced people are key to the success of any software development.
Main objective of testing techniques: minimise risk of implementation errors.
Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple.
Questions
¿?
References
Wikipedia. http://en.wikipedia.org
IPL Information Processing Ltd, An Introduction to Safety Critical Systems , Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0820 IPL Information Processing Ltd, An Introduction to Software Testing , Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0826 Evangelos Nikolaropoulos,
Testing safety-critical software
, Hewlett-Packard Journal, June 1997. http://findarticles.com/p/articles/mi_m0HPJ/is_n3_v48/ai_19540814/?tag=content;col1 Frederick P. Brooks, Jr. ,
No Silver Bullet: Essence and Accidents of Software Engineering
, 1986.
Andreas Gerstinger,
Safety-Critical Computer Systems – Open Questions and Approaches presentation
, February 16, 2007, Institute of Computer Technology, Wien.
Fault Tree Analysis: How to understand it. http://syque.com/quality_tools/toolbook/FTA/how.htm