Transcript Aeronautical Data Quality (ADQ) update

Implementing the DAL – A Phased
Approach
DAL/DQR Workshop
Brussels, 19-20 February 2013
Presented by:
Miguel Rodrigues Paulo
SES unit
EUROCONTROL
The European Organisation for the Safety of Air Navigation
Phased Approach
PREPARATION
DEFINITION
PRE
DEPLOYMENT
DEPLOYMENT
POST
DEPLOYMENT
Designing the overarching process for handling aeronautical
data and aeronautical information
Defining the processes, constituents and their associated
procedures
Validating the processes, constituents and their associated
procedures
Implementing and monitoring the processes, constituents
and their associated procedures
Demonstrating compliance
2
Preparation Phase (1)
•
Prepare the Data Quality Assurance Plan (DQAP)
How data quality will be addressed in relation to the objectives of the DAL
Specification
•
Conduct the Safety Assessments for new or upgraded systems
Article 10(3) “[…] any changes to the existing systems […] or the introduction of
new systems are preceded by a safety assessment, including hazard
identification, risk assessment and mitigation”
•
•
Demonstrate that your ISO 9001:2008 Certificate complies with the
DAL Objectives in ANNEX J
Develop a Safety Policy
•
•
•
Procedures to address the safety management objectives from Annex VII,
Part B of the IR
Define the roles and responsibilities
Develop a Security Policy
•
Procedures to address the security management objectives from Annex
VII, Part C of the IR
3
Preparation Phase (2)
Data Quality Assurance Plan (DQAP)
•
Planned party’s Overarching Process, describing the systems, their
constituents and associated procedures involved in the origination,
production, storage, handling, processing, transfer and distribution of
aeronautical data and aeronautical information
•
•
•
•
Data and products are handled and comply with the relevant standards and
specifications (e.g. EUROCONTROL Specifications, ICAO standards)
Data and products are complete, correct and provided in a timely manner
History of data and products is traceable
Criteria for compliance
•
•
•
•
Start with ANNEX B CONFORMITY MATERIAL
Text of the Objective
Scope, the applicability and what should be the key or relevant aspects to
be considered
Timelines for implementation
Partial compliances or non-compliances
4
Definition Phase (1)
Work Instructions
•
Define work instructions giving the details of the set of actions that are
undertaken relating to each process of the systems from the party
overarching process
•
•
•
•
Classify the Work Instructions: Origination, Data Checking, Data
Processing or Other
Assign DPALs to the Work Instructions
•
•
•
Minimum content
Involve relevant stakeholders and use relevant published material
Based only on the DAL or also in the degree of reliance
Develop a Work Instruction Implementation Plan (contingency,
responsibilities, training)
Get approval of the Work Instruction at the Organisation functional
level
5
Definition Phase (2)
Constituents
•
•
•
•
List the constituents that are part of the systems included in the party
overarching process
Document the constituent operational requirements (defining the
required operational functionality, performance and integrity)
Classify the constituents: Origination, Data Checking, Data Processing
or Other
Assign TQLs to the Constituents
•
•
Based only on the DAL or also in the degree of reliance
Select the software safety standard to be followed (for software based
constituents)
6
Tools and Software
DAL/DQR Workshop
EUROCONTROL Brussels
Roy Langridge
2013-02-19/20
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Contents
 Background
 Role of Tools in the Data Chain
 Data Assurance Levels – Tools and Software
 Software Safety Assurance – What Does it Mean?
 Principle of Software Assurance Levels
 Air Traffic Control - Application
 EUROCAE ED-153 and CEN Technical Specification
 Recommendations
 Summary
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Background
 Commission Regulation (EU) 73/2010 contains a number of
requirements that relate to tools and software.
 These are intended to ensure that data quality is both achieved
and maintained throughout the data process:
 Even when being automatically processed.
 Indeed, the fact that Commission Regulation (EU) 73/2010
promotes automation / digital transfer brings new risks which
must be managed.
 These risks must be understood, managed and mitigated to
ensure that the tools themselves do not either contribute data
errors or fail to detect errors.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Background: Role of Tools in the Data Chain(1)
 Traditionally, a limited number of tools were used in the data chain:




Survey equipment;
Word processors;
NOTAM facilities;
Printers etc.
 Over the past few years, this list has been extended as the data
chain has become more electronic-based:
 EAD;
 GIS;
 eAIP tools.
 Nonetheless, the data has mainly still been managed in a “human
readable form” with manual processing.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Background: Role of Tools in the Data Chain(2)
 The provisions of Commission Regulation (EU) 73/2010 enforce
changes to these methods of working.
 Digital transfer will become the norm, with systems being used
for the delivery and receipt of data.
 An increased level of metadata will need to be managed in
association with the core data.
 ICAO requirements are changing also:
 Electronic terrain and obstacle data requirements lead to a significant
increase in the quantity of data to be handled.
 New tools will be needed throughout the data chain:
 It is likely that common tools used by many actors will be seen.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Data Assurance Levels – Tools and Software(1)
 The DAL specification contains a number of objectives that must
be met by tools and software.
 These are allocated the DAL-TS prefix:
 DAL-TS-010 – DAL-TS-250;
 DAL-TS-500 – DAL-TS-540.
 The objectives apply to:
 Existing tools;
 Minor modifications to existing tools;
 New tools or upgraded tools*.
 Article 2(40) of Regulation (EC) No 549/2004 defines an upgrade as “any
modification that changes the operational characteristics of a system”.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Data Assurance Levels – Tools and Software(2)
 The DAL objectives are established primarily to ensure that an
appropriate level of assurance is obtained from the tools:
 Through their specification, design, implementation, testing, putting into
service and operation.
 However, key to the objectives is an understanding of the level of
reliance that is placed upon the tool:
 If it is supposed to catch an error and fails to, will it be caught elsewhere in
the process?
 If it introduces an error, will this be caught elsewhere in the process?
 This is addressed by the allocation of a Tool Qualification Level
(TQL) for each tool:
 TQL1 – TQL3.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Data Assurance Levels – Tools and Software(3)
 Some basic principles are applied:
 Tools should have well defined requirements:
 Tool Operational Requirement.
 Tools are classified depending on their purpose:








Data processing;
Data checking;
Data origination (measurement);
Other.
Data checking tool cannot amend data values;
Security should be considered in development, as well as in operation;
Tools should be validated against their requirements;
Their introduction into service should be planned:
 Including the need for staff training, etc.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance – What Does it Mean?
 ‘Software Safety Assurance’ is a phrase that is often used but
which can be misunderstood.
 In essence, it means the degree of confidence that a software
tool will not behave in such a way that it may result in a safety
incident:
 If you consider the AIS, the tool itself would not be unsafe, however, it may
result in data that is incorrect which, when used, leads to a safety incident:
 Key point: the data itself is not unsafe.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
Principle of Software Safety Assurance Levels(1)
 Safety assurance is achieved by understanding the risks to safety and
ensuring that they are mitigated.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
Principle of Software Safety Assurance Levels(2)
 Software is often developed using the V model:
 Good to use this to explain software safety assurance.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
Air Traffic Control - Application
 Such software safety assurance processes are already applied
within the aviation domain:
 For both ground-based and airborne components.
 ATC systems are developed in accordance with software safety
assurance standards:
 And have been for many years.
 Most ANSPs will have some knowledge of such standards:
 Although not common within the AIS.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
EUROCAE ED-153 and CEN Technical Specification(1)
 The application of software safety assurance should utilise a
formal process.
 EUROCAE developed ED-153 “Guidelines for ANS Software Safety
Assurance”.
 This refers to the interoperability regulation and describes it
relationship as guidance to this regulation, as follows:
 “This document provides guidance aiming to partially satisfy EC regulations
(552/2004, 482/2008).
 Therefore, both safety regulatory requirements (482/2008) as well as
Interoperability (552/2004) aspects of Software are addressed by this
document.”
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
EUROCAE ED-153 and CEN Technical Specification(2)
 The guidance provided indicates what must be achieved as a
minimum for each level of assurance needed:
 SWAL 1 through to SWAL 4.
 The guidance provided is sound and, whilst the terminology may
be a little confusing to those unfamiliar with it, can be relatively
easily applied.
 The European Committee for Standardisation (CEN) has also
identified the need for standardisation of software safety
assurance.
 It has developed a draft Technical Specification (TS-16501-2012)
which is currently under final review.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Software Safety Assurance:
EUROCAE ED-153 and CEN Technical Specification(3)
 This technical specification states in its introduction:
 “This Technical Specification specifies the Technical, Operational and
Maintenance requirements for Software Assurance Levels to support the
demonstration of compliance with some elements of the Essential
Requirements “Safety” and “Principles governing the construction of
systems” of the Regulation (EC 552/2004) of the European Parliament and
of the Council on the interoperability of the European Air Traffic network
(“the Interoperability regulation”).”
 It should be noted, however, that the actual requirements
specified in the technical specification refer, in the main, to
specific sections EUROCAE ED-153.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Recommendations(1)
 Have a clearly documented understanding (requirement) for all
tools:
 For those developed in-house, bought-in or COTS.
 Consider whether as an organisation you are still able / willing to
develop tools yourself:
 Formality may need to be increased, new skills gained and, possibly, more
resource provided.
 If approved, use CEN TS-16501-2012 for the application of
software safety assurance:
 Implies the use of EUROCAE ED-153 also;
 For bespoke developments manufactured on your behalf, mandate that
this is applied.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Recommendations(2)
 Consider applying the same safety assurance regime for
hardware tools, as well as for software:
 The main principles are the same.
 If you are procuring a bespoke development from a
manufacturer, ensure that it applies mature development
processes:
 Look for evidence / counter-evidence of sound development:
 A high number of bugs in released software may be indicative of a lack of
control in the development processes.
 Finally, don’t be scared!
 The DAL objectives can look very prescriptive, but they are not that bad:
 Just apply them in a sensible manner for your organisation.
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Summary
 Tools are an essential component of the data chain:
 Hardware and software.
 They can result in risks to the quality of data and must be
managed.
 Guidance exists for the safety assurance of software that has
been developed to meet the requirements of SES.
 Similar approaches may be applied for hardware.
 Requirements are essential:
 If you do not define what the tool should do, how do you know it does it?
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
 Thank you for your attention.
 Any questions?
© 2013, Mileridge Limited. No part of this presentation may
be reproduced in any form without the express written
permission of Mileridge Limited.
Definition Phase (3)
Error Reporting
•
Define an effective error reporting, measurement and corrective
process
•
•
•
Mechanism for data errors
Mechanism for work instructions and constituent errors
Most of the objectives for these mechanisms are in ANNEX J
Personnel
•
•
Define the set of skills and competency requirements
Define a training/briefing program
•
•
ICAO requirements and standards (supplements, amendments and
NOTAM procedures, update cycles…)
ADQ IR requirements
26
Definition Phase (4)
Data Suppliers
•
•
Identify all data suppliers
Establish Formal Arrangements
•
•
•
Agreed data set
Agreed data exchange format
Security requirements
Security
•
Define secure areas
•
•
•
•
protection against physical damage or degradation
protected from unauthorised access
Define requirements for security assessment and clearance
Define a contingency plan to address eventual loss of ability to access
or publish data
27
Pre-Deployment Phase (1)
•
Validation of the party’s overarching process [independent validation]
•
All the work instructions are defined
•
Validation of work instructions [independent validation] & [internal
validation]
•
Validation of constituents [independent validation] & [internal validation]
•
•
New or upgraded constituents
Existing constituents
28
Pre-Deployment Phase (2)
•
Validation of the error reporting mechanisms
•
•
•
Validation of the data set [independent validation]
•
•
•
•
Data error reporting mechanism [independent validation]
Work instruction and constituents error reporting mechanism [independent
validation]
Ensure completeness (all aeronautical features in Appendix 1 of ICAO
Annex 15 and in ED99A – Airport Mapping Requirements)
Ensure any missing data item is declared
Validate the procedure to annotate ADQ non-compliant data (Article 7(2))
Validation of the data exchange mechanism [independent validation]
•
•
•
Direct electronic connection
Exchange Format
Mechanism for integrity protection
29
Pre-Deployment Phase (3)
•
Personnel
•
•
•
Data Suppliers
•
•
Validation of the Formal Arrangements [independent validation]
• process for contingency for continuity
• process for handling errors/inconsistencies
Security
•
•
•
Validation of the set of skills and competency requirements [independent
validation]
Validation of the training/briefing program [independent validation]
Perform basic employment checks
Validation of the contingency plan for the eventual loss of ability to access
or publish data [independent validation]
Safety
•
Safety Policy shall be communicated to all employees
30
Deployment Phase (1)
•
Data Origination Request Procedures
•
•
Ensure currency of the requested data
Data Exchange Procedures
•
•
•
•
Authentication of data originators (authorised source)
Reasonableness checks
Independent verification of data handled manually
Data verification (integrity) and validation (accuracy, resolution,
completeness) procedures
• probability-based sampling
• positional accuracy of critical data items shall be validated by direct
methods
• manually derived objects shall be validated by appropriate
visualisation techniques
31
Deployment Phase (2)
•
Data Protection Procedures
•
•
•
•
•
•
•
•
Encryption
Digital Certification
Non-repudiation
Protect the network (malicious and/or unauthorised access)
Protect stored data (malicious and/or unauthorised access)
Archiving and back-up
Synchronisation of shadowed data
Data Consistency Procedures
•
Ensure consistency between data published in several publications
32
Deployment Phase (3)
•
Data Timeliness Procedures
•
•
•
Error Reporting Procedures
•
•
Meet publication deadlines
Handle last minute changes/cancellations
Adequacy of the mitigation actions for work instructions and constituents
errors shall be independently validated
Personnel Procedures
•
Retain (or have access) to a sufficient level of qualified and competent staff
33
Deployment Phase (4)
•
Formal Arrangements Procedures
•
•
Security Procedures
•
•
Monitor compliance with the Formal Arrangement requirements
Monitor periodically security regulations, standards and notices and use
them to update the Security Management Procedures
Safety Procedures
•
Monitor periodically safety regulations, standards and notices and use them
to update the Safety Management Procedures
34
Post-Deployment Phase (1)
Data Quality Assurance Summary (DQAS)
Provide the arguments and evidence for compliance and/or partial compliance
with the objectives of the DAL Specification
•
•
Compliance statement
Documents demonstrating compliance
•
•
•
•
•
•
Description of the new overarching process
Evidence of the independent validation/verification activities
Results of the Safety Assessments
Partial compliances and/or non-compliances
Justifications that the partial compliances or non-compliances do not
adversely affect the quality of the data
List of planned corrective actions for partial compliances or noncompliances
35
Phased Approach
PREPARATION
DEFINITION
PRE
DEPLOYMENT
DEPLOYMENT
POST
DEPLOYMENT
Designing the overarching process for handling aeronautical
data and aeronautical information
Defining the processes, constituents and their associated
procedures
Validating the processes, constituents and their associated
procedures
Implementing and monitoring the processes, constituents
and their associated procedures
Demonstrating compliance
36