Transcript Document

Toward a New ATM Software
Safety Assessment
Methodology
dott. Francesca Matarese
Scenario
• An increasing proportion of ANS functions is implemented by
software, which are becoming more and more safety-critical.
• The introduction of new navigation systems highlights the need for
efficient tools to assess the possible safety impact of these systems.
• There is the need of guidance on how to provide software safety
assurance, but today no ANS software-related standard exists, which
neither fulfils ANS specificities (especially for ground part), nor is
widely spread and extensively used by ANS community to become a
de facto standard.
• To be compliant with EC Regulations on ATM Safety (e.g. 482/2008),
EUROCAE delivered ED-153, derived from EUROCONTROL SAM,
not standards but definition of practices to assure safety of an ANS
system during its whole lifecycle.
Toward a New ATM Software Safety Assessment Methodology
Main objective
• Main limitation of SAM and ED-153 consists of not evaluating safety
level of existing legacy systems, which have been developed over an
extended period of time and for whom the only evidence that they are
‘tolerably safe’ is that they have proved themselves to be so over
years of operation.
• Our objective is to define a new methodology and to establish a
future reference against which to certify safety of software
systems of ATM ground segments.
• This methodology assesses safety of new integrated systems,
constituted by old legacy and new ones.
Toward a New ATM Software Safety Assessment Methodology
System upgrade assessment
• ANSPs are responsible for ANSs they provide.
• Whenever they need to upgrade a system, they have to demonstrate
to the NSA that it is still reliable and that it will not impact on existing
safety level.
• To this aim, ANSPs ask to the stakeholders to provide safety
assessment of the new system, composed by newly developed
elements and already existing ones.
• Implementation of new functionalities may inversely impact or
be in conflict with the original system: the more upgrades that
occur, the greater the likelihood that the overall system design will be
compromised, increasing the potential for increased failure rate.
• But existing safety assessment methodologies evaluate safety
level of new subsystems as stand-alone, not in combination with
existing legacy ones, so to ensure reliability of the “change”, ignoring
possible new failures that could occur in the new integrated system.
Toward a New ATM Software Safety Assessment Methodology
Legacy vs. new systems
•
Existing sub-systems are assessed as black-boxes, to be tested
just indirectly through tests on new sub-system functionalities.
•
No additional tests are usually performed on old functionalities, which
could on the contrary be affected by the new ones.
•
Moreover, each system upgrade is composed by different subsystems, some of them of new concept, others already developed.
•
So two different kind of difficulties have to be faced:
1. evaluation of the integration between old legacy software systems
with new ones,
2. deployment of new software components integrated with already
existing ones.
Toward a New ATM Software Safety Assessment Methodology
Assessment of ANS systems upgrade
Toward a New ATM Software Safety Assessment Methodology
SW Reliability and SWAL
• Software reliability is defined as the probability that software will
not cause a system failure and can be used to assess probability of
occurrence of hazards, based on service history metrics for
existing legacy systems, or on the quality of new subsystems.
• EUROCONTROL defined SWAL as a uniform measure of how the
software was developed, transferred into operation, maintained and
decommissioned and a measure of the ability of the product to
function as intended.
• To be able to assess the whole new operating integrated system, an
innovative approach has been proposed, based on the verification
of SWAL for new software components and of service history
evidences collection for the old ones.
Toward a New ATM Software Safety Assessment Methodology
Steps of the methodology
• FTA is performed to determine Safety Requirements, by deriving a
functional breakdown that allows apportioning the requirements coming
from the Safety Objectives to physical components functionalities
• FMECA shows the direct link of these Requirements to the physical
components failures that affect these functions.
• Then Safety Requirements are translated into SWAL Objectives, by
identifying the likelihood that, once software fails, this software failure
can generate an end effect, which has a certain severity.
• SWAL allocation is possible only for new designed software and is
satisfied by showing evidences that are produced by lifecycle
activities logs.
• For already existing software, Safety Requirements that correspond
to a certain range of acceptable likelihood, have to be demonstrated
by service history.
Toward a New ATM Software Safety Assessment Methodology
FTA Example
Corruption of
flight data
exchanged with
CWP
2. H-FDPS-2
Mitigation of
AFS failure
Mitigation of
CDB failure
GATE1
GATE2
Contribution
of AFS CSCI
failure
ATC controller
communicate by
radio with
adjacent sectors.
Middleware
Contribution
Syntactic and
semantic check
algorithms of
CDB.
2.1 AFS
AFS MM
2.2 CDB
CDB MM
r=0.00224
r=0.00224
r=0.00224
r=0.00224
Toward a New ATM Software Safety Assessment Methodology
FMECA Example
Failure Mode
Definition
loss or
corruption of
data flows
Failure Effects
Definition
Safety
Requirements
Identification
local effect and
system effect
failure rate of
the software
components
involved in the
examined
failure mode
assessed in
FTA
Severity
Assignment
Mitigation
Means
identification
Safety
Requirements
After Mitigation
categorized in
accordance
with Severity
Classification
Scheme
ESARR 4
any means that
allow
avoidance,
detection,
propagation
control of
failures and/or
mitigation of the
effects, to meet
safety
requirements
failure rate is
re-evaluated
after the
application of
Mitigation
Means
Toward a New ATM Software Safety Assessment Methodology
Risk Classification Scheme
PROBABILITY OF OCCURRENCE
Extremely rare
Rare
Occasional
Frequent
Very Frequent
< 10-7
from 10-7 to 10-5 from 10-5 to 10-3 from 10-3 to 10-1
>10-1
Acceptable
Tolerable
Unacceptable
SEVERITY CLASS
I
Accident
II
Serious Incident
III
Major Incident
IV
Significant Incident
V
No Safety Effect
Toward a New ATM Software Safety Assessment Methodology
SWAL Scheme
Effect Severity
1
2
3
4
5
Very Frequent
SWAL1
SWAL2
SWAL3
SWAL4
SWAL4
Frequent
SWAL2
SWAL3
SWAL3
SWAL4
SWAL4
Occasional
SWAL3
SWAL3
SWAL4
SWAL4
SWAL4
Rare
SWAL4
SWAL4
SWAL4
SWAL4
SWAL4
Extremely Rare
SWAL4
SWAL4
SWAL4
SWAL4
SWAL4
Likelihood of generating such an effect
(Pe x Ph)
Toward a New ATM Software Safety Assessment Methodology
Evidences collection: Service History
• The compliance to each analysed Safety Requirements implies the
compliance to each Safety Objectives.
• To demonstrate system compliance with Safety Requirements, two
options can be considered: Service History Analysis or SWAL
Assessment.
• The new proposed methodology requires that each element of existing
legacy subsystem is considered as part of the new integrated system.
• Safety Requirements are expressed in terms of Failure Rates.
These requirements are then compared with the Failure Rates
resulting from Service History Analysis, to prove compliance.
Toward a New ATM Software Safety Assessment Methodology
Evidences collection: SWAL Assessment
• SWAL is designed to provide a level of confidence that the software
will be developed and can be integrated in the equipment and then in
the system to manage risks due to software failure.
• The way to provide this level of confidence and assurance is by
defining some objectives that will satisfy this level of assurance.
• These objectives address all processes of the software lifecycle and
identify what is to be done to satisfy a level of assurance.
• The assurance level is satisfied by showing evidences that are
produced by activities, which achieve these objectives.
• To provide evidences that such activities have been correctly
performed, logs about all software lifecycle phases are produced.
Toward a New ATM Software Safety Assessment Methodology
Conclusions
• ANSPs require the assessment of safety impact of the introduction of
new software components in existing legacy systems, but no standard
or guidance material exists for evaluating this complex situation.
• To this purpose, we developed and proposed an innovative approach,
based on the analysis of the updated system and on the verification of
SWAL for new safety components and of service history evidences for
the old ones.
• The new methodology has been proposed to several ANSPs
around Europe (Italy, Luxembourg, Cyprus, Malta, Georgia,
Turkey, Romania) who accepted and validated it.
Toward a New ATM Software Safety Assessment Methodology
Thank you for your attention!
Toward a New ATM Software Safety Assessment Methodology