Transcript ICS200 - National REP

G-131 & E/L-131: Exercise Evaluation and
Improvement Planning
Exercise Evaluation Overview
Kenneth L. Wierman Jr.
DHS/FEMA Headquarters
2010 National REP Conference
Course Goal
To familiarize individuals with exercise planning or
evaluation responsibilities with the Homeland Security
Exercise Evaluation Program’s (HSEEP) exercise
evaluation and improvement planning process, and to
provide those students opportunities to practice key
exercise evaluation skills.
FEMA Directive 123-15 16 January 2009
2010 National REP Conference
Target Audience
Individuals who manage, participate in, review, or
sponsor exercise evaluation and/or improvement
planning
 Federal, State, local, Tribal, Territorial government
 Private sector
 Non-governmental
2010 National REP Conference
Course Prerequisites
 IS-120.A: Introduction to Exercises
 IS-130: Exercise Evaluation and Improvement
Planning
 HSEEP Training Course
 Radiological Emergency Preparedness Program
(REP) and Planning Course (Recommended)
2010 National REP Conference
Course Objectives (1 of 2)
 Describe the need for a systematic approach to
exercise evaluation
 List the eight steps of the exercise evaluation and
improvement planning process
 Identify pre-exercise activities necessary for a
successful evaluation
 Explain the function of Exercise Evaluation Guides
(EEGs) and their relationship to the Target
Capabilities List (TCL)/Universal Task List (UTL)
2010 National REP Conference
Course Objectives (2 of 2)
 Practice evaluation observation, data analysis, and
report writing skills
 Describe post-exercise analysis activities
 Explain the purpose and format of the After Action
Report/Improvement Plan (AAR/IP)
 Describe how a Corrective Action Program (CAP)
translates exercise outcomes into continual
improvements to preparedness
2010 National REP Conference
Course Structure (1 of 2)
 Unit 1: Exercise Evaluation Overview
 Unit 2: Exercise Evaluation Process
 Unit 3: Planning and Organizing the Evaluation
 Unit 4: Observing the Exercise and Collecting Data
 Unit 5: Analyzing Data
 Unit 6: After Action Report and After Action
Conference
 Unit 7: The Corrective Action Program
 Course Summary (End of Two-Day Course)
2010 National REP Conference
Course Structure (2 of 2)
 Unit 8: Introduction to Evaluator Skill Practice
 Unit 9: Customizing EEGs from TCLs and Objectives
 Unit 10: Exercise Evaluation: Skill Practice #1
 Unit 11: Exercise Evaluation: Skill Practice #2
 Unit 12: Exercise Evaluation: Skill Practice #3
 Unit 13: Exercise Evaluation: Skill Practice #4
 Unit 14: Course Summary
2010 National REP Conference
Course Background
 Original course introduced by FEMA in 1992
 In 2002, DHS developed and introduced the
Homeland Security Exercise and Evaluation Program
(HSEEP)
 All-hazards
 Performance- and capabilities-based
 All levels of government
 DHS and FEMA have worked collaboratively to
standardize the language and concepts used in the
exercise evaluation process.
 This course updated to reflect HSEEP guidance
2010 National REP Conference
Activity: Defining an Exercise
What is an Exercise?
Why do we Exercise?
2010 National REP Conference
Defining an Exercise

A focused practice activity that places players in a
simulated situation

Conducted to support an overall assessment of a
capability

Primary benefits:


Individual and team training

System improvement
Critical component of preparedness cycle
2010 National REP Conference
Activity: Defining Exercise Evaluation
What is Exercise Evaluation?
2010 National REP Conference
Defining Exercise Evaluation
Exercise evaluation is the act of:

Reviewing or observing and recording exercise
activity or conduct

Assessing/analyzing behaviors and activities
against exercise objectives/standards

Noting strengths, weaknesses, deficiencies, or
other observations
2010 National REP Conference
Activity: Importance of Exercise Evaluation & Improvement Planning
Why is Exercise Evaluation and Improvement
Planning Important?
2010 National REP Conference
Importance of Exercise Evaluation & Improvement Planning (1 of 2)

Exercise evaluation


Assesses a jurisdiction or organization's
performance by identifying strengths and areas
for improvement
Improvement planning

Leverages the outputs of the evaluation process
by developing Improvement Plans (IPs)

Provides a disciplined process for implementing
corrective actions and socializing best practices
2010 National REP Conference
Importance of Exercise Evaluation & Improvement Planning (2 of 2)
 Exercise evaluation and
improvement planning are
critical components of the
preparedness cycle
 Implementation of
corrective actions is the
mechanism by which
exercises can inform and
improve other components
of the preparedness cycle
2010 National REP Conference
Activity: Customizing Exercise Evaluation Guides (EEGs)
Plan Ref
Task
Metric
[Name,
Pg/Sec#]
Order transportation services for
evacuation.
[From
plan]
[Name,
Pg/Sec#]
Coordinate with supporting agencies
[From
and prearranged providers to obtain
plan]
appropriate means of transportation for
those requiring transportation
assistance (e.g., buses, ambulances,
handicap-assisted vans).
[Name,
Pg/Sec#]
Implement plans for providing
alternative means of transport for
immobilized individuals or those
needing other special assistance in
transit.
[From
plan]
2010 National REP Conference
Yes
No
Time
Activity: Customizing Exercise Evaluation Guides (EEGs)
Activity Analysis
Observations (Each bullet will need a completed After Action Report Input form.)
Strengths
Areas for Improvement
Root Cause
Root Cause
Root Cause
Additional Observations:
2010 National REP Conference
Activity: Customizing Exercise Evaluation Guides (EEGs) – AAR Input
Form
Observation
□
[Insert a short, complete sentence that describes the general observation. Then select
from the lists below whether the observation is a strength or an area for improvement. A
strength is an observed action, behavior, procedure, and/or practice that is worthy of
recognition and special notice. Areas for improvement are those areas in which the
evaluator observed that a necessary task was not performed or that a task was performed
with notable problems. Lastly, identify the capability element that this observation falls
under.]
Noted Strength
□
Area for Improvement
□
□
□
Planning
Process
Training
Capability Element
□
Equipment
□
Organization
□
Personnel
References (Standards, Policies, or
Plans)
[List relevant plans, policies, procedures, laws, and/or regulations, or sections of these plans,
policies, procedures, laws, and/or regulations. If no references apply to the observation, it is
acceptable to list “N/A” or “Not Applicable.”]
Analysis
[Include a description of the behavior or actions at the core of the observation, as well as a brief
description of what happened and the consequence(s) (positive or negative) of the action or
behavior. If an action was performed successfully, include any relevant innovative approaches
utilized by the exercise participants. If an action was not performed adequately, the root causes
contributing to the shortcoming must be identified.]
Recommendations
[Insert recommendations to address identified areas for improvement, based on the judgment and
experience of the evaluation team. If the observation was identified as a strength without
corresponding recommendations, insert “None.”]
2010 National REP Conference
Pilot Courses
October 2009, February 2010
and April 2010
30 students
 Federal Agencies
 State and Local
 FEMA
 Chemical Stockpile
Emergency Preparedness
Program (CSEPP)
 National Exercise Division
(NED)
 Radiological Emergency
Preparedness Program
(REPP)
2010 National REP Conference
Upcoming HSEEP Exercises
Columbia Generating Station – 30 August 2010
Indian Point – 20 September 2010
Susquehanna Steam Electric Station – 5
October 2010
Pilgrim and Quad Cities – 15 November 2010
2010 National REP Conference
Summary
 What are the objectives of this course?
 What is exercise evaluation and improvement
planning?
 Why is exercise evaluation and improvement
planning important?
2010 National REP Conference
2010 National REP Conference