Transcript Software and Safety: A Software Engineer's Perspective

Programme
• Introduction
• What is High Integrity Software?
• Reliable Programming in Standard Languages
– Coffee
• Standards Overview
• DO178B and the Lockheed C130J
– Lunch
• Def Stan 00-55 and SHOLIS
• ITSEC, Common Criteria and Mondex
– Tea
• Compiler and Run-time Issues
• Conclusions
Copyright © 2001 Praxis Critical Systems Limited

Aerospace
Standards
Standards
Rail
Defence
Nuclear
Copyright © 2001 Praxis Critical Systems Limited
Automotive

Examples of (Safety-related) Standards
• RTCA DO-178B (civil avionics)
more on these later
• Def Stan 00-55/56 (UK military)
• IEC 61508 (generic ‘programmable systems’)
– CENELEC 50128 (railway industry)
– IEC 601 (medical equipment)
• IEC 880 (nuclear power control)
• MISRA (automotive industry)
• Vary in their power to mandate or recommend
Copyright © 2001 Praxis Critical Systems Limited

Security Standards
• Itsec
• “Orange book”
• Common Criteria
– More on these later
Copyright © 2001 Praxis Critical Systems Limited

Defence Standards
Safety Case
Requirement
Software Specific
UK Def Stan 00-55


UK Def Stan 00-56


US MIL STD 882D


NATO STANAG 4404


NATO STANAG 4452


Def (Aust) 5679


Copyright © 2001 Praxis Critical Systems Limited

Rail Standards
Safety Case
Requirement
Software Specific
CENELEC EN 50126


CENELEC EN 50128


CENELEC ENV 50129


Copyright © 2001 Praxis Critical Systems Limited

Other Standards
Safety Case
Requirement
Software Specific
IEC 61508


IEC 880 (Nuclear)


DO 178 B


MISRA Guidelines (Automotive)


Copyright © 2001 Praxis Critical Systems Limited

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/RTOS etc.
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/RTOS etc.
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited

Hazard/risk Analysis
• Identification of hazards and their causes
• Hazards have :
– An estimated probability of occurrence
– An associated severity
• ‘Risk’ is combined probability and severity
• ‘Risk regions’ can be identified:
– Intolerable
– ALARP – as low as reasonably possible
– Broadly acceptable
• ‘Safety’ is freedom from unacceptable risk
Copyright © 2001 Praxis Critical Systems Limited

Hazard/risk Analysis: Techniques
• Various formalised techniques exist:
– HAZOP (hazard and operability)
– Failure modes and effects analysis
– Fault tree analysis
• A hazard log will be used to detail identified
hazards and state steps being taken to mitigate
them.
• Different HAs:
– Preliminary HA (PHA)
– Design HA (DHA)
Copyright © 2001 Praxis Critical Systems Limited

Hazard Identification: HAZOP
• A systematic, creative examination of a design
• Performed by a multi-disciplinary team
• Inspect each component in turn
– Consider the design intention
– Apply a list of guide words
– Derive plausible deviations from the design intention
Copyright © 2001 Praxis Critical Systems Limited

Consequence Analysis
• Purpose
– To determine the intermediate conditions and final
consequences arising from the identified hazards
• Approach
– A diagrammatic representation is recommended
– Ideally quantitative estimate of probabilities
• Techniques
– Cause consequence diagrams
– Event trees
Copyright © 2001 Praxis Critical Systems Limited

Event Trees
Server
Extinguisher
Ignition
Extinguisher works
Extinguisher fails
Copyright © 2001 Praxis Critical Systems Limited
Alarm
Result
No Incident
Alarm sounds
Minor Fire
Alarm fails
Major Fire

Causal Analysis
• Purpose
– To determine credible combinations or sequences of causal
factors which can lead to the hazards
• Approach
– A diagrammatic representation is recommended
– Ideally quantitative estimate of probabilities
• Techniques
– Failure Modes and Effects Analysis (FMEA)
– Fault Tree Analysis (FTA)
Copyright © 2001 Praxis Critical Systems Limited

Accident Severity Categories
Catastrophic multiple deaths
Critical
a single death; and/or multiple
severe injuries or severe
occupational illnesses
Marginal
a single severe injury or
occupational illness; and/or
multiple minor injuries or minor
occupational illnesses
Negligible
at most a single minor injury or
minor occupational illness
Copyright © 2001 Praxis Critical Systems Limited

Probability Ranges
Frequent
likely to be continually experienced
Probable
Occasional
Remote
Improbable
likely to occur often
likely to occur several times
likely to occur some time
unlikely, but may exceptionally occur
Incredible
extremely unlikely that the event will
occur at all
Copyright © 2001 Praxis Critical Systems Limited

What Is Risk ?
Severity
Risk
Catastrophic
Intolerable
Tolerable
Negligible
Minor
Incredible
Copyright © 2001 Praxis Critical Systems Limited
Probability
Frequent

How Is Risk Regulated?
ALARP
ALARP = As Low As Reasonably Practicable
Copyright © 2001 Praxis Critical Systems Limited

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/RTOS etc.
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited

Safety Integrity Levels
• System will be assigned a safety integrity level,
typically:
– 4 – 1 : ‘high’ – ‘low’
• Can be based on probabilistic or MTBF
concepts (e.g. 00-55)
• DO-178B talks about levels A-E.
• Techniques (and costs) will then be determined
by the assigned SIL
• Partitioning by SIL within system is normal
Copyright © 2001 Praxis Critical Systems Limited

What Is a SIL?
• A general indicator of quality for design/build
processes
• Applicable to software and hardware
• Probability that the system meets its
requirements
• Probability of systematic failure
• Often misunderstood and misused
Copyright © 2001 Praxis Critical Systems Limited

Safety Integrity Levels (Probabilistic IEC 6 1508)
SIL
Low Demand Mode of
Operation
Probability of failure
(on demand)
High Demand Mode
of Operation
Dangerous failure
rate (per year)
4
 10-5 to 10-4
 10-5 to 10-4
3
 10-4 to 10-3
 10-4 to 10-3
2
 10-3 to 10-2
 10-3 to 10-2
1
 10-2 to 10-1
 10-2 to 10-1
Copyright © 2001 Praxis Critical Systems Limited

Development for Different Sils (IEC 6 1508)
SIL1
Informal
SIL2
SIL3
Specification
Informal SemiFormal
Prototyping
R
R
R
Coding
HLL
HLL
SafePreferred
Subset
HLL
Defensive Code R
HR
Static Analysis R
HR
HR
Formal Proof
R
R
Dynamic
R
HR
HR
Testing
Performance
R
R
HR
Testing
SIL4
Formal
R
SafeSubset
HLL
HR
HR
HR
HR
HR
Partial summary of IEC 61508 recommendations for SILs.
Copyright © 2001 Praxis Critical Systems Limited

IEC 61508: Software Detailed Design
•
•
0
1
•
2
•
3
•
4
none
structured methodology (CORE, JSD,
MASCOT, Yourdon)
+ semi-formal methods (function block
diagrams, data-flow diagrams, pseudo
code)
+ formal methods (VDM, Z, CCS, CSP, HOL,
OBJ, LOTOS, Petri nets, state transition
diagrams)
+ formal proof to establish conformance to
software requirements specification.
Copyright © 2001 Praxis Critical Systems Limited

EN 50128: Software Development and
Implementation Requirements
TECHNIQUE/MEASURE
SIL 2
SIL 4
Formal Method (eg VDM, Z)
R
HR
Design and Coding Standards
HR
M
-
HR
Functional/Black Box Testing
HR
M
Data Recording and Analysis (eg
DRACAS, FRACAS)
HR
M
Language Subset (eg Spark Ada)
Copyright © 2001 Praxis Critical Systems Limited

Common SIL Myths
• “What are the safety requirements for the
system?”
• “It needs to be SIL4”
Copyright © 2001 Praxis Critical Systems Limited

Common SIL Myths
• “We’ve identified a new hazard, so there are
new safety requirements”
• “That’s ok, the software’s SIL4 so it won’t do
that”
Copyright © 2001 Praxis Critical Systems Limited

Common SIL Myths
• “This software is SIL0 so it’s unsafe”
• “The operating system is SIL3 so it must be
safe”
Copyright © 2001 Praxis Critical Systems Limited

What Is a SIL?
• Very similar to stars for hotel ratings!
Copyright © 2001 Praxis Critical Systems Limited

Safety Requirements and Sils
• Safety requirements define what we have to
demonstrate
• SILs define how thorough the demonstration
needs to be
• SILs apply to specific requirements
• Applying a consistent process to a software
package is a sensible management approach,
but may have little direct relation to safety
Copyright © 2001 Praxis Critical Systems Limited

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/RTOS etc.
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited

Lifecycle Definition
The V Model
Req’ts
Spec
Spec
Test
HL
Design
Test
Detailed
Design
Test
Code
Copyright © 2001 Praxis Critical Systems Limited
•Some standards are
lifecycle independent.
•Others are explicit
that the V-Model is
assumed.
•QMS will define
deliverables:
>ISO9000-3
•There are Version and
Configuration
Management issues.

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/RTOS etc.
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited
more later

Safety-related Software Techniques
• Common aspects:
–
–
–
–
–
–
–
–
Hazard/risk analysis
Safety integrity levels
Lifecycle definition
Formal methods for requirements and design
Modelling, for various purposes
Choice of language/rtos etc
Validation, verification and testing (VVT)
Accumulation of evidence – safety case
Copyright © 2001 Praxis Critical Systems Limited

Evidence and the Safety Case
• A successful safety assessment depends on the
inspection of evidence. Example:
–
–
–
–
–
–
–
–
Config. Management plan (including environment)
Software verification plan
Software quality assurance plan
Standards for design/coding
Specifications for requirements/design/tests
Software verification results
Software configuration index
Traceability matrix
Copyright © 2001 Praxis Critical Systems Limited

Resources
•
Ainsworth, Mike; Estaughffe, Katherine; Simpson, Alan. Safety
Cases for Software Intensive Systems. In Aspects of Safety
Management, Redmill & Anderson (Eds). Springer 2001. ISBN
1-85233-411-8
Copyright © 2001 Praxis Critical Systems Limited

Programme
• Introduction
• What is High Integrity Software?
• Reliable Programming in Standard Languages
– Coffee
• Standards Overview
• DO178B and the Lockheed C130J
– Lunch
• Def Stan 00-55 and SHOLIS
• ITSEC, Common Criteria and Mondex
– Tea
• Compiler and Run-time Issues
• Conclusions
Copyright © 2001 Praxis Critical Systems Limited

Lockheed C130J
Copyright © 2001 Praxis Critical Systems Limited

C130J Key Features
• A new propulsion system with 29% more thrust and 15% better fuel
efficiency.
• An all composite six-blade propeller system which is lighter in
weight and has fewer moving parts than previous Hercules propellers.
• Advanced avionics technology includes Liquid Crystal Display
(LCD) instrument readouts for aircraft flight control, operating
systems, and navigation.
• Two mission computers and two backup bus interface units provide
dual redundancy for the Hercules' systems. These computers also
provide for an integrated diagnostics system to advise the crew of the
status of the aircraft's various systems.
Copyright © 2001 Praxis Critical Systems Limited

Copyright © 2001 Praxis Critical Systems Limited

C130J Software Certification
DO-178B/FAA
Civil
C130J
Military
Copyright © 2001 Praxis Critical Systems Limited
RAF lead
customer
UK MoD IV&V
programme
comparisons
