Transcript Federal Aviation Administration Organization and Overview

The United States
Government’s USOAP
Experience
By: Michele Cappelle, National USOAP
Coordinator
Date: December 15, 2007
Federal Aviation
Administration
Safety oversight audit process
Pre-audit
Location
Montreal
Audit
On-site
Post-audit
Montreal
Sec Gen letter to State
Activities
Update
SAAQ &
CC
Team
meeting
Opening meeting
Final
report
Closing meeting
SPO handover to TL
Time Frame
12 months
days
9 months+
The United States Government’s USOAP experience
December 12, 2007
Federal Aviation
Administration
2
Preparation Recommendations
•
Select a Coordinator with strong team
building skills and knowledge of the
State’s organizational and legal structure
•
Ensure the Coordinator takes the ICAO
Regional Audit Preparation
Seminar/Workshop
– Outlines USOAP process and ICAO expectations
– Opportunity to meet others who have been audited
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
3
3
Preparation Recommendations
• Build a strong inter-agency working group
to respond to each of the audit areas
– Include all State entities involved with safety
oversight as well as representatives from the
appropriate offices in the CAA
– Ensure the legal offices are involved in the
coordination and development of State responses
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
4
4
Preparation Recommendations
• Complete the compliance checklists and the
State Activity Questionnaire in a timely
manner
– Offer clear, accurate and concise explanations
– Answers should represent the “official” government
position
– There are a lot of overlapping and repetitious
questions in the State Activity Questionnaire and in
the Protocols. Ensure that the answers are
compatible
The United States Government’s USOAP experience
December 12, 2007
Federal Aviation
Administration
5
5
Preparation Recommendations
• Develop answers to the audit protocols
questions in advance of the audit
–
–
–
–
–
For internal use
Coordinate answers
Identify and organize evidence to support answer
Identify the subject-matter-expert for each question
Identify the appropriate person to respond to the
auditor for each question
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
6
6
Preparation Recommendations
• Brief high-level government officials as well
as the CAA staff
– Ensure they understand the importance of the audit
– Explain the USOAP comprehensive systems
approach
– Prepare them for the extensive resource demands
for the pre-audit activities, on-site audit activities and
post-audit activities
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
7
7
Preparation Recommendations
• Brief industry representatives for ICAO
industry visits
– Explain USOAP process and purpose of the visit
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
8
8
Preparation Recommendations
• Learn to use and navigate audit tools on the
SOA website
• Learn how ICAO determines a State’s level
of compliance with the critical elements
– Bias toward less complex civil aviation oversight
systems
– Ensure high-level officials understand how this
number is developed
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
9
9
Preparation Recommendations
• Review audit results of other States
– Focus on audits which were conducted by the same
auditors who will audit your State
– Focus on audits of the most comparable oversight
systems to your State
– Become familiar with typical findings and begin to
evaluate whether they would be applicable to your
State as well
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
10
10
Preparation Recommendations
• Conduct practice audits for each audit area
– Use people who have served as ICAO USOAP
auditors of other countries
– Use compliance inspectors and train them on the
conduct of ICAO USOAP audits
– Identify officials who will meet with the auditors and
have them participate in the practice audits
– Identify weaknesses
– Ensure internal auditors are familiar with other
State’s audit findings
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
11
11
Preparation Recommendations
• Manage expectations by developing a
credible list of potential audit findings
– Ensure that the management team is aware of the
potential findings
– Avoid findings by correcting weaknesses where
possible
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
12
12
Lessons Learned
• The State’s USOAP team is critical to the
success of the audit
• Strong internal coordination is essential
• Develop good communication w/ USOAP
team
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
13
13
Lessons Learned
• Develop a detailed audit workplan
– ensure each auditor is scheduled to meet with the
appropriate officials for answers to all protocol
questions. This will avoid last-minute surprises and
unprepared or unavailable interviewees
• Insist on organized daily debriefs
– Support a clear understanding by all participants
– Prevent the development of inaccurate audit findings
– Helps manage the IOUs
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
14
14
Lessons Learned
• Use all the resources available to understand
and prepare for the audit
– ICAO SOA website, USOAP team leader, State
auditing expertise, audit results of other states
• Develop a strong internal communication
strategy
– Widely disseminate information concerning the USOAP
– Brief important officials
• Ensure they are familiar with potential audit findings
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
15
15
Lessons Learned
• Train interviewees
– On the USOAP process and expected attitude
– To stick to the subject
– To only answer questions which are in his/her
purview
– On the limits of their authority to respond to ICAO on
behalf of the State
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
16
16
Lessons Learned
• Test all components of the audit for
understanding and effectiveness
– Are the compliance checklists complete and accurate
– Do the State’s answers to the State Activity
Questionnaire completely reflect the State’s official
government response to each issue?
– Are the protocol answers fully coordinated and
supported by evidence?
– Are the interviewees conversant with the protocol
answers and evidence?
– Do you know with whom the auditors need to meet and
are those appointments scheduled?
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
17
17
U.S. Government USOAP
Point of Contact
Michele Cappelle
National USOAP Coordinator
Tel: 202-385-8887
Email: [email protected]
The United States Government’s USOAP experience
7/18/2015
December 12, 2007
Federal Aviation
Administration
18
18