Transcript No Slide Title

Monitoring Normal Operations
and the Line Operations
Safety Audit (LOSA):
The Perspective of ICAO
Captain Dan Maurino
Flight Safety and Human Factors – ICAO
Royal Aeronautical Society Conference on
Mitigating Human Error
London, 15 October 2003
When the Book Goes Out of the Window
Baseline
performance
“Practical
Baseline performance
System
design Operational
deployment
Drift”
Operational
performance
Raiding the Lost Ark
Regulator
Line
Checks
Nobody
Natural
Performance
“Angel”
Performance
LOSA
A Day at the Office
To achieve daily safety and production goals, flight crews must
1
Manage operational
complexity
Threat
Management
2
Manage their own errors
Error Management
Manage aircraft deviations
Undesired Aircraft
State Management
3
LOSA
Indices
LOSA Index: Threats
Not flight crew errors
Not deficiencies in the system
Factors external to the flight deck that
increase complexity of flight operations
Must be managed by the flight crew
Threats = Red flags
Threats Are the Context
Distractions
Passenger events
ATC
Cabin Crew
Terrain
Weather
Similar call signs
Maintenance
Time pressures
Ground Crew
Flight
diversions
Heavy traffic
System
malfunctions
Unfamiliar airports
Automation events
Missed approaches
Threat Prevalence
Threat Types
(Total = 300 LOSA flights)
10%
20%
30% `
(Percentage of LOSA flights w ith a threat)
40%
50%
60%
70%
80%
90%
All Threats
97%
Adv erse Weather
58%
ATC
57%
Airline Operational Pressure
31%
Aircraft Malfunction
MEL Items
22%
Threat Mismanagement
Threat Types
(Total = 300 LOSA flights)
10%
20%
All Threats
13%
ATC
Aircraft Malfunctions
MEL Items
30%
(Percentage of LOSA flights w ith a mismanaged threat)
40%
41%
Adv erse Weather
Airline Operational Pressure
100%
13%
5%
3%
50%
60%
70%
80%
90%
100%
LOSA Index: [Expert] Errors
Error Prevalence (Percentage of LOSA flights w ith an error)
Error Types
(Total = 300 LOSA flights)
10%
20%
30% `
40%
50%
60%
70%
All Errors
10%
19%
Error Mismanagement (Percentage of LOSA flights w ith a mismanaged error)
Error Types
10%
20%
30%
All Errors
Callout
40%
42%
Manual Aircraft Handling
Checklists
100%
34%
Callout
(Total = 300 LOSA flights)
90%
81%
Manual Aircraft Handling
Checklists
80%
27%
9%
5%
50%
60%
70%
80%
90%
100%
LOSA Provides Data Only
LOSA Raw Data
LOSA Report
Systemic Strengths and
Weaknesses
Corporate Culture
Airports / Navigational Aids
Aircraft Design and Automation
Standards / Maintenance /
Training
Crew Support
Crew Strengths and
Weaknesses
Proficiency
Knowledge
CRM Skills
Procedural Compliance
NCAs Affected By ATC (47%)
NCA: Type Of Approach
80%
47%
80%
50%
60%
27%
40%
40%
20%
30%
13%
7%
20%
20%
10%
0%
Visual
Non-Prec
Prec
NCA: Pilot Flying
0%
App Chng
Rwy Chng
Slam Dnk
NCA - Crew Response At The Window
93%
100%
80%
60%
40%
20%
0%
100%
47%
53%
50%
Capt
FO
7%
0%
Force a Ldg
Go - Around
Targets for Improvement
Stabilized Approaches
Checklists
Procedural errors
Automation errors
ATC communications
Intentional non-compliance
SCP: Data Develops Into Information
Systemic Strengths and Weaknesses
LOSA Raw Data
LOSA Report
Corporate Culture
Airports / Navigational Aids
Aircraft Design and Automation
Standards / Maintenance / Training
Crew Support
Crew Strengths and Weaknesses
Proficiency
Knowledge
CRM Skills
Procedural Compliance
Re-measure
Safety
Change
Process
(SCP)
Reduced
49%
Reduced
59%
Reduced
59%
LOSA 1996 vs. 2000 Checklist Errors
Flights with protocol
checklist errors
Flight with unintentional
checklist errors
LOSA 1996
389 flights
33% of the flights
9% of the flights
LOSA 2000
385 flights
9% of the flights
11% of the flights
Checklist
Error
Examples
Performed from memory
Nonstandard phraseology
Self initiated
Not called “complete”
Intentional checklist omission
Missed items
Wrong response to a challenge
Wrong checklist performed
Unintentional checklist omission
LOSA 2000 indicated a significant drop in protocol checklist errors while
unintentional checklist errors have remained unchanged
Threat & Error Management Model (Doc 9803)
Threats
Inconsequential
Threat Management
Threat Induced
Incident or
Accident
Crew Error
Crew Error
Responses
Undesired Aircraft
State
Crew Undesired
Aircraft State
Responses
Error Induced
Incident or
Accident
The Expanding Role of the TEM Model
TEM Model as
Licensing tool (ICAO)
Training tool (Numerous airlines)
Safety management tool (IATA)
Research tool (Boeing)
TEM - Licensing Tool (ICAO FCLT/P)
Competencies
SKA’s
Threat Management
CRM skills
Error Management
Undesired Aircraft
State Management
TEM
principles
Performance criteria
•Follow SOPs
•Perform Briefing
•State plans
•Assign workload
•Contingency planning
•Monitor/Cross-check
•Manage automation
•…
•…
•…
Managing the Red Flags
System resources
GPWS
Automation
ATC
Dispatch
SOP’s
Checklists
Briefings
Training
…
Managing the Red Flags (cont.)
The human contribution
Proficiency
Vigilance
Assertiveness
Monitoring
Decision making
Experience
Leadership
Discipline
…
Real World Human Performance
Flaps
omitted
Checklist
works
Effective
warning
Normal
operation
Descent, Approach & Landing
TOD

Highest risk
phase of flight
10000 ft.
Configuration
FAF
Bottom
Lines
Flare
Most errors (LOSA)
Most consequential errors (LOSA)
Most accidents worldwide (ALAR)
Landing
Briefing the Approach
Crews who brief before TOD commit 1.6 times less
errors during descent, approach and land than
crews who brief after TOD
Landing runway changes occurred 8 percent of the
time
Training point: brief approach prior TOD
TEM Model - Training Tool
CRM, circa 2003
Beyond behavioural stereotypes
Operational performance
Develop and support threat and error
management competencies
The bottom line is safe and efficient
operational performance, not CRM
SMS Data Triangle
Voluntary
Reporting
FDA
Normal
Operations
A Casque Bleu Perspective
LOSA, CRM and NOTECHS
LOSA: a tool to capture systemic data
NOTECHS: a CRM evaluation template
LOSA: context and cognition
NOTECHS/CRM: behaviours
CRM: just one piece of the infinitely broader
information picture developed from LOSA
A situation, generated by misperceptions and
misinformation, that must be put behind