Transcript Document

JHSAT Status Briefing to IHST
June 29, 2006
Mark Liptak FAA ANE-110
Jack Drake HAI
JHSAT - June 2006
1
JHSAT Status Briefing
Objective:
Brief the IHST on the status of the JHSAT
Obtain IHST concurrence on key program elements,
avoid surprises or questions downstream:
dataset selection
adequacy of team membership
scoring criteria
general program timing
outreach approach
use of existing safety reports
JHSAT - June 2006
2
JHSAT Status Briefing
Slide Topics:
Synopsis
Charter
Web Site
Team Composition
Dataset Selection
Process Flow - Timing/Schedule
JHSAT Scoring Criteria
Establishing Standard Problem Statements
Scoring Method
International Partners
Leveraging Existing Safety Reports
Issues for IHST Awareness/Concurrence
JHSAT - June 2006
3
Synopsis
The JHSAT team is fairly well solidified, 14 members, making
progress on adapting the CAST/JSAT process for analysis of
helicopter events. We’re off to a good start.
Team members are enthusiastic about the basic CAST process,
believing it will produce high value results. Have engaged in
many detailed discussions on dataset selection, event scoring,
data quality, mission diversity, etc.
Recent meetings spent evolving and detailing the JHSAT process
and analyzing helicopter accident event data. We have made
significant progress, but its been more time consuming than expected.
Looking for ways to boost efficiency.
Industry is stretched thin, IHST should consider reiterating it’s
appreciation for the commitment needed to make this happen.
The following slides contain information on key JHSAT topics.
JHSAT - June 2006
4
Charter
Final charter developed and agreed to by team,
posted on JHSAT web site:
Goal: Provide a prioritized assessment of the most safety
critical hazards to commercial, private and military
rotorcraft in worldwide operations as derived from
selected rotorcraft data sources.
Provide intervention strategies to the IHST and Joint
Helicopter Safety Implementation Team (JHSIT) that maximize
the likelihood of reducing worldwide helicopter accident
rates by 80 percent by 2016.
Provide a JHSAT report documenting the team’s
findings to the IHST by 1Q 2007.
JHSAT - June 2006
5
Web Site
The JHSAT has established a web site at
http://www.ihst.org/jhsat/jhsat_top.htm
Id=jhsat, password=helicoptersafe
This is an access controlled site, please limit to the IHST only.
The JHSAT is using it to post NTSB data, team analysis results,
meeting minutes, agendas, action lists, etc.
JHSAT - June 2006
6
JHSAT Team Composition
Mark Liptak (FAA ANE) JHSAT co-chair
Barry Rohm (Rolls Royce)
Roy Fox (Bell)
Ed Stockhausen (Airmethods)
Laura Iseler (IHST)
Clark Davenport (FAA ASW)
Ann Azevedo (FAA Risk consultant)
Jack Drake (HAI) JHSAT co-chair
Ray Wall (Bristow)
Tony Alfalla (Sikorsky)
Joe Syslo (Eurocopter)
Sandy Hart (NASA)
Matt Rigsby (FAA ASW)
Steve Gleason (Schweizer)
JHSAT team membership is light on small helicopter experts. We have
discussed participation with Robinson, they do not appear too enthusiastic.
Schweizer has committed only as a part time participant. Industry is
stretched thin and long term commitment is difficult. We could use additional
support from the pilot/operational side and perhaps a training expert.
JHSAT - June 2006
7
JHSAT Dataset Selection
The initial dataset selected for JHSAT analysis is year 2000 NTSB
accidents. This will serve as the basis for the 1Q 07 report out.
The JHSAT will conduct a detailed analysis of these accidents.
There are approximately 190 accidents to be analyzed.
This NTSB dataset is nearly fully populated with final narrative
causal/factual information. Also, less likely to encounter litigation
constraints with this dataset.
Consideration of accident causal factor trends for the last 24
years gives the team high confidence that detailed analysis
of year 2000 events will yield high value recommendations
that will mitigate long standing trends (see next slide)
Future JHSAT activity will target year 2001, 2002, etc.
JHSAT - June 2006
8
24 year look back – Relative stability in helicopter
accident causal factors
24 Years - Little Change
100%
90%
80%
Final
causal
data
not yet
fully
developed
70%
60%
50%
40%
30%
20%
10%
0%
1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Aircraft
Powerplant
Auxilary Equipment
Human Performance
Environment / Facilities
Unknown
US dataset represents 50% of worldwide fleet
JHSAT - June 2006
9
JHSAT Basic Process Flowchart
Charter
Development
Score
Intervention
Ability/Usage
Establish
Team
Identify
Intervention
Strategies
Score
Problem
Validity &
Importance
Yes
Prioritize by
Overall
Effectiveness
Conflicts?
Review
NTSB
Docket Data
Select
Data Set
No
Assign Std
Problem
Statements
Technical
Review
&
Expert
Validation
Report
Results
Develop
Event
Sequence
Identify
Problems
(what/why)
IHST
JHSIT
JHSAT - June 2006
10
JHSAT Process
JHSAT is structured to be similar to the CAST/JSAT process
which has been highly successful in the part 121 community
Basic tenets:
Engaging stakeholders/experts from the
helicopter community
Findings based on real world helicopter accident data
Recommendations ranked by a structured scoring method
JHSAT is sticking to these basic tenets but making significant
modifications elsewhere in the process. Our last two meetings have
centered around completing foundational process work and
processing the event data.
JHSAT - June 2006
11
JHSAT Scoring Criteria
Validity (V) - How
valid do we think the
Problem Statement is
in this accident based
on the available
accident information?
Importance (I)- How
important is this
particular Problem
Statement in the
causal chain in this
accident?
Range:
Guess = 0
Absolutely Sure = 4
Range:
No importance = 0
Cause of accident = 4
Ability (A) - How
effective is this particular
intervention in mitigating
the problem statement of
this accident?
Usage (U) - How sure are
we that this intervention
will be utilized as expected
in the real world.
Range:
No effect on Problem
Statement = 0
Would Prevent Problem
Statement = 4
Range:
No Utilization = 0
100% Utilization = 4
Ability (A) - How effective
is this particular
intervention in mitigating
the cause or contributing
factors of this accident?
Confidence (C) - How
strongly do we believe the
intervention will perform
as expected in the real
world?
Range:
No effect on Problem
Statement = 0
Would Prevent Problem
Statement = 7
Range:
No Confidence = 0
Complete Confidence = 7
JSAT Scoring Criteria
Applicability (App)
– likelihood that the
problem statement
will be present in
future
aircraft/operations
Importance (I) How important is this
particular Problem
Statement in the
causal chain in this
accident?
Range:
No Utilization = 0
100% Utilization = 7
Range:
No importance = 0
Cause of accident = 7
JHSAT - June 2006
12
JHSAT Scoring Criteria
The scoring criteria used by the JHSAT are similar to those used in
CAST/JSAT with the following exceptions:
JSAT’s Applicability was a forward look at fleet characteristics and
assessing intervention effectiveness. JHSAT has eliminated this
criterion due to the belief that future fleet mix and mission will not
change enough to make this a meaningful issue to consider.
JHSAT has added a Validity criterion. This is a scoring factor based on
the spectrum of documentation of helicopter accidents. Some are
very well documented, others very poorly.
The Confidence and Usage criteria are similar, however some of the
detailed definition language is still being worked.
JHSAT - June 2006
13
Determination of Standard Problem Statements (SPS)
A key element to the JSAT process is establishing standard problem
statements (SPS) when identifying specific causal factors in accidents.
Correctly executed, this allows for identifying high value common
threads between accidents, and thus identifies high value mitigation
areas.
JHSAT is developing a standard problem statement list from three
sources:
To jumpstart our process we drew from the SPS developed by the
HAAT team(1). We also pulled SPS statements determined to be
applicable from the CAST SPS list (~500 topics) These two sources
gave us a starting point. As we analyze events we are continuing to
populate our SPS list with unique helicopter problem areas as we
encounter them.
Helicopter Accident Analysis Team, NASA, circa 1997, early version of the Boeing APS/CAST JSAT
process.
(1)
JHSAT - June 2006
14
Scoring Method
The JSAT process developed and used an arithmetic relationship to
rank recommendations by an Overall Effectiveness (OE) rating. Using
the scoring criteria from the previous slide, the relationship is as
follows:
OE = ((I x A) x 2/(I +A)) x C/6 x App/6
This relationship places weighting factors on the Importance and
Applicability scoring criteria.
The JHSAT has not yet made a determination on whether a weighted
arithmetic relationship is appropriate. We have intentionally
constructed our scoring criteria such that we will have flexibility later
in the process to use a variety of arithmetic relationships to help
identify the most effective recommendations.
JHSAT - June 2006
15
Coordinating JHSAT US and International Efforts
US
Dataset
Selection
JSAT Process
Understanding
Refining
JHSAT
Analysis
May-Dec 06
Mar/Apr 06
Expert
Review/
Validation
Tech
Review
Jan 07
Jan 07
May/June 06
Identify
FCAA &
Military
Partners
Sept 06
Indoctrinate
in JHSAT
Process
TBD 06/07
Conduct
JHSAT
based
analyses
TBD 07
Mitigation
Recs
FCAA & Mil
First Set
US Mitigation
Recs
1Q 07
TBD 07
Mitigation
Recs - Global
4Q 07
JHSAT - June 2006
16
Coordinating JHSAT US and International Efforts
US Civil
Dataset
Selection
(1/2 of
helicopters
Worldwide)
Mar/Apr 06
JSAT Process
Understanding
Refining
JHSAT
Analysis
May-Dec 06
Tech
Review
Jan 07
May/June 06
Identify
FCAA &
Military
Partners
Sept 06
Train FCAA
&
Military
Partners
in JHSAT
Process
TBD 06/07
FCAA & Military
Partners
conduct
JHSAT
analyses
on their
accident data
Expert
Review/
Validation
Jan 07
Mitigation
Recs
FCAA & Mil
1Q 07
First Set
US Mitigation
Recs
4Q 07
Mitigation
Recs - Global
TBD 07
TBD 07
IHST
JHSAT - June 2006
17
Coordinating JHSAT US, International and Military Sources
Matt Rigsby from FAA-ASW will be acting as the contact point for
identifying overseas and military partners.
To date we have interest from:
Canada – may use Canada as a test case, good dataset,
strong interest, close proximity
EASA, UK, Australia, Chile, US Navy
Looking into:
Brazil, Japan
Any other entities that possess large helicopter accident datasets that
could be processed by the JHSAT method will be considered.
JHSAT - June 2006
18
Leveraging Existing Safety Reports
JHSAT is reviewing the recommendations of the following reports:
NASA - U.S. Civil Rotorcraft Accidents, 1963 Through 1997
NASA - Analysis of US Civil Rotorcraft Accidents from 1990 to 1996 and Implications for a Safety Program
NASA - ASRS Rotorcraft Incident Study - Draft Data Summary Aviation Safety Reporting System
NASA - Helicopter Accident Analysis Team
AMPA - A Safety Review and Risk Assessment in Air Medical Transport
CRS - Report for Congress - Military Aviation Safety
OGP - Safety Performance of Helicopter Operations in the Oil and Gas Industry - 2000 Data
TSB Canada - Lessons Learned from TSB Investigation of Helicopter Accidents (1994 - 2003)
Bell Textron - History of Helicopter Safety
Other NTSB and international reports may be considered.
Approach: Identify recommendations that call for systemic change via
oversight, policy, SOP development, investment areas, etc. Examples
are changes to safety culture across the industry, adopting SMS
approaches, foundational work for universal adoption of a new
technology. Recommendations calling for specific design changes,
specific training topics, specific changes to specific platforms or
operational missions, etc, will not be considered at this point. We will
allow the JHSAT process and NTSB data to guide us on the latter type
of recommendation.
JHSAT - June 2006
19
Issues for IHST Awareness/Concurrence
Dataset selection will be NTSB year 2000 accidents.
Team membership is light on small helicopter pilots/ops, may need help
from IHST to push for renewed commitment from some industry
members.
Scoring criteria adapted from that used in the CAST/JSAT.
General JHSAT program timing (slide 15)
Early work underway to identify international partners, will teach them
the JHSAT process and have them analyze their own datasets.
Reviewing existing safety reports for systemic change recommendations, if
any identified they will be passed to the IHST/JHSIT this summer/fall.
JHSAT - June 2006
20