Transcript Safety Management Systems for Design and Manufacturing Organizations

Safety Management Systems
for
Design and Manufacturing
Organizations
Federal Aviation
Administration
Concepts and Implementation for
the Pilot Project Participant
Part I - Concepts
MSMS Team Member
FAA Aircraft Certification Service
Federal Aviation
Administration
SL-1
Agenda for the Kickoff
Day 1
Afternoon - Introduce SMS Concepts & the
Pilot Project
.
Day 2
All Day
- Commence Preliminary Gap Analysis
Day 3
Morning
- Conclude Preliminary Gap Analysis
Afternoon - Wrap-up and Next Steps
- Feedback on the Orientation
Federal Aviation
Administration
SL-2
To ensure a common understanding of Design &
Manufacturing (D&M) SMS concepts, the Pilot Project
implementation approach, and expectations
Federal Aviation
Administration
SL-3
Detailed Agenda for this Afternoon
Related SMS Activities
SMS Fundamentals
SMS Pilot Project Implementation
Guidance & Tools
Next Steps and Discussion
Federal Aviation
Administration
SL-4
Major SMS Activities
• ICAO Activities and Products
• Rulemaking – Part 5 & Part 119
• AFS Activities
AVS SMS
Initiatives
ARC D&M
Workgroup
SMS
for
D&M
Int’l
Collaboration
Group
AIR SMS
Initiatives
Federal Aviation
Administration
SL-5
Aviation Safety (AVS)
Activities
• Flight Standards pilot project
– Involved many air carriers and repair facilities for
over three years
– Drafted and evolved a variety of SMS policies
Consistency
Interoperability
• Aircraft Certification pilot project
– Adopted many of their concepts and products
– Goal is to remain collaborative and consistent
Federal Aviation
Administration
SL-6
As Applied to the D&M Community…
• Many SMS requirements are mature and
already being adopted by the aviation
industry
– Four SMS Components are the basis
– Part 5 requirements are in the process of being
codified
– AIR goal is to remain harmonized
• We must define SMS policies
and procedures to meet unique
D&M environment
Federal Aviation
Administration
SL-7
How does SMS affect ODA
• FAA was requested by the ARC to consider how ODA
would work with SMS
• ODA unit functions and processes are not affected by SMS
requirements
– ODA certification functions follow FAA policy and the ODA manual.
– Showing and finding of compliance to regulations are unaffected
• ODA may interact with some company QMS processes
– i.e. planning conformity inspections based on a mature quality system
– PC holder may obtain an A/W certificate without further showing … ODA
may inspect for conformity with the type design
• Likewise some SMS processes may be useful
Federal Aviation
Administration
SL-8
Agenda
Related SMS Activities
SMS Fundamentals
SMS Pilot Project Implementation
Guidance & Tools
Next Steps and Discussion
Federal Aviation
Administration
SL-9
Safety Management System
“A systematic approach to managing safety,
including the necessary organizational
structures, accountabilities, policies and
procedures.”
ICAO Doc 9859
“… formal, top-down, organization-wide
approach to managing safety risk and assuring
the effectiveness of safety risk controls. ”
Part 5 NPRM – Definition
Federal Aviation
Administration
SL-10
SMS Purpose and Methods
Provides:
 A systematic way to identify hazards and control risks
 Assurance that risk controls remain effective
 A formal means of meeting regulatory requirements
(Title 14)
 A means for the FAA to evaluate an organization’s
safety management capability
SMS is intended to be performance-based
Federal Aviation
Administration
SL-11
What SMS is not and what it is…
What it isn’t:
What it is:
A substitute for
compliance
Compliance is integral to
safety management
A substitute for
oversight
A means for industry to
meet safety responsibilities
A replacement for
system safety
SMS completes the
systems approach
A requirement for a
new department
A set of decision-making
processes for management
Federal Aviation
Administration
SL-12
SMS Components
Safety Policy
Safety Risk
Management
Safety
Assurance
Safety Promotion
Safety Policy and Objectives
• All management systems must define
policies, procedures, and organizational
structures to accomplish their goals.
• Policy establishes the structure of the SMS.
Federal Aviation
Administration
SL-13
Safety Policy
• Establishes executive and management
commitment and objectives for ensuring safety–
what the management wants
• Sets up framework of organizational structures,
accountabilities, plans, procedures, and controls to
meet objectives
• Establishes clear standards of acceptable behavior
• Documented
• Communicated
• Regularly reviewed
Federal Aviation
Administration
SL-14
Required Safety Management Personnel
• Appoint an Accountable Executive with ultimate
accountability for the SMS
• Designate a management representative to
manage, monitor and maintain the SMS processes
–
–
–
–
–
Ensure SMS processes are established and maintained
Facilitate hazard identification and risk analysis
Monitor effectiveness of risk controls
Promote safety
Report to accountable executive
Federal Aviation
Administration
SL-15
Emergency Preparedness
• Develop and implement procedures, as
necessary, that will be followed in the event
of an accident or incident
Federal Aviation
Administration
SL-16
SMS Documentation and Records
• Documents safety policies, objectives,
processes and procedures
• May be a stand-alone manual or integrated
into existing documentation systems
Federal Aviation
Administration
SL-17
SMS Components
Safety Policy
Safety Risk
Management
Safety
Assurance
Safety Promotion
Safety Risk Management
• A formal system of hazard identification, analysis
and risk management is essential in controlling risk
to acceptable levels
Federal Aviation
Administration
SL-18
Safety Risk Management
1. Understand the system and environment
2. Identify hazards and their causes
3. Analyze and assess risk
4. Develop risk controls
Federal Aviation
Administration
SL-19
Safety Risk Management and Safety Assurance
SMS provides a
systematic way to
control risk and to
provide assurance
that those risk
controls are effective
SRM
SA
System
Description
System
Operation
Hazard
Identification
Data
Acquisition
Risk
Analysis
New
Hazard/
Ineffective
Control
Risk Acceptable
Assessment
Analysis
SystemConformity
Assessment
Unacceptable
Nonconformity
Risk
Control
Corrective
Action
Design
Performance
Federal Aviation
Administration
SL-20
Describe the System
System
Description
Hazard
Identification
• What?
– Analyze systems (organization, process,
product) to understand critical factors
– For the purpose of identifying hazards
• When?
Risk
Analysis
Risk
Assessment
Risk
Control
– Initial design of systems, organizations or
products and its operation and maintenance
– Development of design and manufacturing processes
and procedures
– New or recurring hazards being identified
– Planned changes
Federal Aviation
Administration
SL-21
Describe the System (cont’d)
Effectively describe your system and tasks to
the level necessary to identify hazards
• Consider your which organizations, processes and
products are relevant
• Hazards should be identified along the way
• System descriptions do not necessarily have to be
overly complex
• Whatever process is used it should be collaborative
Federal Aviation
Administration
SL-22
Hazard Identification
System
Description
Hazard
Identification
A hazard is any existing or potential
condition that can lead to injury, illness,
or death to people; damage to, or loss
of, a system, equipment, or property;
and/or damage to the environment.
Risk
Analysis
Risk
Assessment
Risk
Control
FAA Order VS 8000.367
Federal Aviation
Administration
SL-23
Focusing the Definition of Hazard
Identification D&M Organizations
System
Description
Hazard
Identification
• Hazard: a condition that could foreseeably cause or
contribute to an aircraft accident.
Risk
Analysis
•Accident: an occurrence associated with the operation of
an aircraft that takes place between the time any person
boards the aircraft with the intention of flight and all such
persons have disembarked, and in which any person suffers
death or serious injury, or in which the aircraft receives
substantial damage (from 49 CFR § 830.2, Definitions).
Federal Aviation
Administration
Risk
Assessment
Risk
Control
SL-24
Traditional approach to preventing accidents
•
•
•
•
Focus on the direct causes of past accidents
Identify unsafe acts committed by personnel
Attach blame/punish for failures to “perform safely”
Address identified safety concern exclusively
Identifies:
WHAT?
WHO?
WHEN?
But does not always disclose:
WHY?
HOW?
Federal Aviation
Administration
SL-25
Accident Causation
Organization
Workplace
People
Defenses
Accident
Management
decisions and
organizational
processes
Working
Conditions
Errors
and
Violations
Latent conditions and trajectory
Federal Aviation
Administration
SL-26
“Swiss Cheese” Model of Organizational Failure
Design Policies and Procedures
RISK
CONTROLS
Reviews and Quality Auditing
Staffing and Resources
HAZARD 1
Procedures and
practices do not
represent how
tasks are
performed or
they are not
used or
followed
Inter-Department Communication
HAZARD 2
Design
reviews and
audits fail to
address the
‘real’ issues,
only
addressing
‘low hanging
fruit’
Safety Assurance uses metrics and
audits to monitor the performance of
risk controls
CONSEQUENCE
HAZARD 3
Insufficient
resources or
trained staff
available to
perform
tasks
HAZARD 4
Poor visibility
and reporting of
issues with the
design to staff
Manifest Safety
Issue
Risk = Severity x Probability of
the Consequence
Federal Aviation
Administration
SL-27
Example of an Organizational Failure
Staffing reductions are now evaluated
against product impact
RISK
CONTROLS
Design Policy requires above
5 EOs must be incorporated in drawing
Drawing checks required before release
and audits are performed
Elect Install specification changes acceptance
standards and mandatory key inspections
HAZARD 1
Company
management
reduces engineers
HAZARD 2
to save $
Reduction in
engineers results in
multiple EOs not
incorporated into
electrical drawings
Alert Service Bulletins and ADs are
issued to correct arcing and
sparking caused by nonconforming
assemblies
CONSEQUENCE
HAZARD 3
Production & QA
personnel can’t
determine actual
design and
proceed building
electrical
assemblies
HAZARD 4
Aircraft are
delivered with
nonconforming
electrical
assemblies
Nonconforming
electrical assemblies
cause arcing &
sparking in flight
Federal Aviation
Administration
SL-28
Risk Analysis
System
Description
 Important to distinguish between:
Hazard
Identification
Risk
Analysis
 Hazard – a condition
Risk
Assessment
 Consequence – result
Risk
Control
 Risk – likelihood & severity of the
consequence
 Analyzing risk involves the consideration
of both the likelihood and the severity of
any consequences.
ICAO Doc. 9859
Federal Aviation
Administration
SL-29
SRM
From Hazard to Risk
System
Description
Deficient Conditions
impacting activities =
Variable
Performance
Hazard
Identification
Hazards
Causing…
Active Failures
Resulting in…
Consequences
Risk
Analysis
Likelihood
Risk
Assmt
Risk
Control
Severity
Risk Analysis
Risk Assessment
Federal Aviation
Administration
SL-30
Risk Analysis
• Risk is the composite of the predicted likelihood
or probability and the severity of each possible
consequence of each identified hazard
Hazard
Active Failure
Consequence
Likelihood
x
Severity
Risk Level
Adapted from ICAO Doc. 9859
Federal Aviation
Administration
SL-31
Risk Assessment
Risk assessment determines the level of risk to use in
making a bottom line decision.
System
Description
Hazard
Identification
High Risk
Medium Risk
Risk
Analysis
Low Risk
Risk
Likelihood
Risk Severity
Negligible
Minor
Major
Hazardous
Catastrophic
E
D
C
B
A
5D
5C
4D
4C
Frequent
5
5E
Occasional
4
4E
Remote
3
3E
Improbable
2
2E
Extremely
improbable
1
1E
5B
5A
Unacceptable
4B
mitigation
3B
3D
3C
2B
2D
2Crequired
1B
1D
1C
Acceptable
4A
3A
2A
1A
Examples of Risk Levels
Risk
Assessment
Risk
Control
A risk matrix is one tool used for risk assessment. There are
many other tools available to accomplish the same result.
Federal Aviation
Administration
SL-32
Risk Control = Risk Mitigation
System
Description
Hazard
Identification
A major component of any safety system
is the defenses (controls) put in place to
protect people, property or the
environment.
Risk
Analysis
Risk
Assmt
Risk
Control
These defenses are used to reduce the
likelihood or severity of the
consequences associated with any
given hazard or condition.
ICAO Doc. 9859
Federal Aviation
Administration
SL-33
Risk Control - General
Order of Precedence
System
Description
Hazard
Identification
Risk
Analysis
1. Design out the hazard
2. Physical guards or barriers
Risk
Assessment
3. Warnings or alert signal
Risk
Control
4. Administrative controls
• Procedures
• Training
Federal Aviation
Administration
SL-34
SMS Components
Safety Policy
Safety Risk
Management
Safety
Assurance
Safety Promotion
Safety Assurance
• Once controls are implemented, the SMS
must assure they are continually practiced
and continue to be effective in a changing
environment.
Federal Aviation
Administration
SL-35
Safety Assurance Functions
The organization shall monitor their
systems and operations to:
– identify new hazards
– measure the effectiveness of safety risk
controls
– ensure compliance with regulatory
requirements
Federal Aviation
Administration
SL-36
vs
• SA focuses on ensuring risk controls meet
safety objectives
• QA focuses on product conformity and
customer satisfaction on a continual basis
• Integration of both management systems can
be beneficial
Federal Aviation
Administration
SL-37
Why do we need SMS if we already have
AS9100?
• Short answer: they’re different.
– “Quality” is important, but it does not encompass all of the elements of
an SMS and visa versa.
• We performed a comparison and evaluated the strength
of the coverage of each element in the SMS standard to
AS9100B
• Although some QMS and SMS are identical in words
differences in the scope may be substantial in one
system verses another
Federal Aviation
Administration
SL-38
Comparison between AS9100 and SMS
•Excerpt from comparison between SMS and AS9100B
Relevant
item in
AS9100
Strength of
coverage
(-5 to 5 scale)
item in AS9100
Federal Aviation
Administration
SL-39
SMS vs AS 9100 Requirements
• In several cases it took two or more AS9100 requirements
to equal one SMS requirement
AS 9100B
SMS
Number of requirements in each
document.
259
197
Number of requirements in each
document having a relationship with
the other.
72
39
Federal Aviation
Administration
SL-40
Conclusions
• Safety Policy is reasonably well covered with
AS9100B, with changes in semantics
• SRM and SA are very lacking in AS9100B,
particularly in:
– Risk Management & Controls
– Employee Reporting System
• Safety Promotion is somewhat lacking, especially:
– Data Sharing
Federal Aviation
Administration
SL-41
Data Acquisition
•
Types of Information Sources
1. Continuous Monitoring
2. Audits
3. Evaluations
4. Investigations
5. Employee Reporting Systems
6. Other
•
Opportunities for improvement are
found in the data / information
Federal Aviation
Administration
SL-42
Data Acquisition - Employee Reporting
System
• Employee safety reporting & feedback system
is required
• Must provide confidentiality
• Employees must be encouraged to use the
system
• Data may identify emerging hazards
• Data must be included in analysis
Federal Aviation
Administration
SL-43
Analysis
• Analyze data to understand effectiveness
of risk controls
• Identify potential new hazards that need
risk controls
• Analyze information gathered about the
organization’s risk controls
Federal Aviation
Administration
SL-44
System Assessment
• Are safety objectives being met?
• Risk controls effective?
• Is the organization in compliance with the
regulations?
• Is the SMS doing what it was designed to do?
• Are there new hazards?
• Is the system catching new hazards?
If new hazards are identified – return to SRM to evaluate the
hazard and develop a risk control (if necessary).
Federal Aviation
Administration
SL-45
Preventive/Corrective Actions
Examples
• Revised policies
• Redesign/modification
• New procedures
• Process changes
• Enhanced training
• Assignment of responsible persons
Federal Aviation
Administration
SL-46
Management Review
Top management will conduct regular
reviews of the SMS, including:
• The outputs of SRM & SA
• Lessons learned
• Need for changes
Federal Aviation
Administration
SL-47
Continuous Improvement
The organization shall continuously improve
the effectiveness of the SMS through:
• Safety and Quality Policies
• Safety Objectives
• Audits & Evaluations
• Analysis of Data
• Corrective and Preventive Actions
• Management Reviews
Federal Aviation
Administration
SL-48
SMS Components
Safety Policy
Safety Risk
Management
Safety
Assurance
Safety Promotion
Safety Promotion
• The organization must promote safety as a
core value with practices that support a
positive safety culture.
Federal Aviation
Administration
SL-49
Safety Promotion: Definition
• Safety promotion = a combination of:
• Safety Culture
• Training
• Knowledge Sharing
• They result in activities that support the
implementation and operation of SMS in an
organization
Federal Aviation
Administration
SL-50
Personnel Expectations
• Identification of competency requirements
• Selection and hiring criteria and standards
• Training
• Skill competency
– Initial training
– Recurrent training
– Continuous communication
Federal Aviation
Administration
SL-51
Communications & Awareness
• Employees must understand the SMS
• Employees benefit from safety lessons learned
• Explain why particular actions are taken
• Develop awareness of hazards
• Foster open reporting of safety concerns
• Initial and ongoing training
Federal Aviation
Administration
SL-52
Safety Culture
Management
Employees
Communication
Committed:
Management commits resources and “walks
the talk”.
Informed:
People understand the hazards & risks.
Learning:
The company learns from mistakes. Staff are
updated on safety issues by management.
Just:
Employees know what is acceptable
& unacceptable behavior.
Reporting:
All personnel freely share critical safety
information.
Federal Aviation
Administration
SL-53
Commitment to SMS
• Documents alone will not
guarantee development of
a positive safety culture
• Employees must see evidence of
management commitment to SMS
Management Attitudes & Actions =
the most important factors
Federal Aviation
Administration
SL-54
Safety Management System
Provides a systematic way to:
1. Identify hazards and control risk
2. Provide assurance that risk controls are effective
Policy
(Structure)
Safety Risk
Managemen
t
Safety
Assurance
Safety Promotion
(Culture)
Federal Aviation
Administration
SL-55
Evolving SMS Roles
Safety
Safety and Production
FAA
S
R
M
S
A
Process-based
oversight
FAA’s Safety
Management
(Oversight)
Compliance-based oversight
D&M’s Safety
Management
System
S
R
M
S
A
Design &
Production
Activities
D&M Company
Federal Aviation
Administration
SL-56