Transcript Crew Resource Management

LBFD
Managing Learning
LBFD TRAINING
JUNE 2005
1
Objectives


Review Means and Methods Employed to
Evaluate & Document LBFD Recruits.
Introduce Crew Resource Management
(CRM) as an Evaluation Tool.
2
Elements:
3

1. Cognitive / Head



2. Psychomotor / Hand



Validated / Peer Reviewed Exams
IP System
Realistic Manipulative Testing
IP System
3. Affective / Heart




Multiple Simulated Emergencies
Artificial Stressors
CRM
IP System
4
4. Data Collection
&
Continual Peer Review.
“If they’re not learning,
we’re not teaching.”
5
&
5. Student Feedback
6
What are Improvement Points
&
why do you need them?

Consequence based quantitative feedback
measurements to indicate the appropriate
level of improvement needed to achieve
competency.
7
Key points

IP’s measure degrees below competency
established by validated standards.


Do not describe as “failing a test”.
Develop “Major” and “Minor” sliding
scales.

Error tolerant
8
Key Points (cont.)

Regular individual status reports.


Not for public distribution.
Key tracking tool.



Individual
Topic
Academies
9
If disparity is noted:



? Curriculum
? Teacher
? Student
Minimize variables caused by the
first two.
10
1.
Cognitive / Head
Validated / Peer Reviewed
Exams
11
Exams

Peer Review During Development




Confirm Validity
Confirm Curriculum
Reference Check
Post Test Analysis


Critique
Item Analysis
12
The City of Long Beach
Fire Department
RECRUIT TRAINING MEMORANDUM
Date:
November 1, 2004
To:
Training Staff
From:
S. Raganold, Captain
Subject:
RECRUIT ACADEMY 2004 BLOCK 4 QUIZ & EXAM STAFF REVIEW
Enclosed within this package are three documents for your review:
 Block 4 Exam Reading List
 Quiz 8 Outcomes
 Block 4 Exam Outcomes
Please review, comment and note compliance on the table below:
Name
Date
Buchanan
Grego
Jackson, D.
Rexwinkle
Rohr
Samuelson
Segura
Sherrat
Initial
Only one copy of the Quiz and Exam exist at this time. They are to be considered as
confidential.
13
Name: ________________________________________________
LONG BEACH FIRE DEPARTMENT
RECRUIT TRAINING
Recruit Academy 2004A
PROBATIONARY EXAM 1
April 12, 2005
DO NOT USE TEST BOOKLET. NAME ONLY.
Answer Sheet Provided.
14
Post Test Analysis
Totals
Average
Median
StDev
#
Incorrect
Bayley, J.
Borders, I.
Chan, A.
Chang, M.
Demetropolis, A.
Farley, J.
Healy, B.
Hopkins, D.
Johnson, J.
Johnson, T.
Knapp, K.
Lawrence, M.
Lee, J.
Magana, M.
Marantz, N.
Marshal, C.
Oh, D.
Rindone, K.
Rose, K.
Rosenstein, J.
Van Egdom, M.
April 12, 2005
Proficiency 1
# Correct
Name
2004 A
%
113
111
112
114
112
117
113
113
109
115
112
114
113
112
110
111
116
114
112
117
118
P 1 Total
7
9
8
6
8
3
7
7
11
5
8
6
7
8
10
9
4
6
8
3
2
94.2%
92.5%
93.3%
95.0%
93.3%
97.5%
94.2%
94.2%
90.8%
95.8%
93.3%
95.0%
94.2%
93.3%
91.7%
92.5%
96.7%
95.0%
93.3%
97.5%
98.3%
120
2378
142
113.2
113.0
2.3
6.8
7.0
2.3
94.3%
94.2%
1.9%
15
?#
-#
Previous
Use
2004A Probationary Exam 1
Question Outcomes
Previous
?#
#
?#
-# Previous
Use
Use
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
0
0
0
0
0
0
0
0
1
0
0
0
0
0
2
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
P
B2/78
P
B2/78
P
B1/7
B1/56
P
P
P
B1/64
B1/58
B1/62
B1/65
N
B1/59
B1/60
B1/63
N
P
P
P
P
P
P
B1/22
B1/23
P
P
B3/104
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
9
0
2
1
1
0
0
0
0
1
1
1
1
0
1
0
0
0
5
1
0
7
0
0
0
0
0
9
0
N
B3/63
B1/35
Q5/6
P
Q5/11
Q5/2
Q5/7,B1/37
Q5/8
Q5/1,B1/34
P
B1/19
B1/15
B1/18
P
B2/74
B2/73
N
P
B2/75
B2/71
Q6/9
N
P
Q6/8
P
P
P
P
P
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
0
0
0
0
0
0
7
0
0
1
4
1
0
8
13
0
1
0
0
1
0
1
2
2
6
1
0
0
0
2
P
P
P
P
P
P
P
P
P
P
N
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
?#
-#
1
91.
0
92.
0
93.
0
94.
1
95.
1
96.
0
97.
2
98.
2
99.
100. 0
101. 1
102. 5
103. 0
104. 0
105. 0
106. 0
107. 0
108.
109. 1
110.
111. 0
112. 10
113.
114.
115.
116.
24
117.
118.
119.
120.
Previous
Use
P
P
P
P
P
P
P
P
P
P
P
N
P
P
P
P
P
P
N
N
N
Legend:
B2/78 = Previously used in 2004A Block 2 Exam, question number 78.
P = Previously used in past Probationary Exam(s).
N = New Question
Previously used questions may have been edited for this exam, if references were not accurate or if the answers
were poorly formatted The substance remained consistent with the original.
Red font denotes questions used in Item Analysis.
16
2005 A OPS 6
Day 37 - 11-7-05
25
20
# Students
2004A
Probationary Exam 1
Student Outcomes
# Correct #
+95. Fail
+96.
80%
+97.
+98.
+99.
+100.
+101.
+102.
+103.
+104.
+105.
+106.
+107.
+108.
+109.
1
+110.
1
+111.
2
+112.
5
+113.
4
+114.
3
+115.
1
+116.
1
+117.
2
+118.
1
+119.
+120.
15
10
5
9
5
3
2
0
1
0
27
26
# Correct
2
25
24
Fail
0
30
29
28
17
Partial Item Analysis
High student scores vs. high incidents incorrect answers.
Probationary Exam 1
2004A
Question # / Amount Incorrect
31
59
74
75
12
Student
Van Egdom
Farley
Rosenstein
Oh, D.
Johnson, T.
-9
+

+
+
+
-9
+
+
+
+

-8
+
+
+
+
+
-13

+
+
+

-10
+
+
+
+

Outcome
118
117
117
116
115
 - Incorrect / + - Correct
Quiz Review – Done individually
18
Total Wrong
EMT
EMT %
Total
Questions
Ops
Total Wrong
Ops
Ops %
Total
Academy
Questions
Total
Academy
Questions
Wrong
Academy %
Total
Test IP Old Total
Test IP New Total
Bayley, J.
Bohm, E.
Borders, I.
Chan, A.
Chang, M.
Dano, L.
De Hart, A.
Demetropolis, A.
Farley, J.
Filson, M.
Healy, B.
Hopkins, D.
Johnson, J.
Johnson, T.
Kepner, K.
Knapp, K.
Lawrence, M.
Lee, D.
Lee, J.
Magana, M.
Marantz, N.
Marshal, C.
Oh, D.
Rindone, K.
Rose, K.
Rosenstein, J.
Van Egdom, M.
White, M.
Total
Questions
EMT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Academy IP
Total
Name
#
57
54
50
38
83
28
111
33
50
4
72
86
53
42
113
32
41
112
81
53
62
43
34
74
76
65
63
138
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
336
20
11
17
14
21
44
18
18
12
4
34
17
25
13
24
17
15
52
14
9
21
15
12
10
12
14
9
31
94.0%
96.7%
94.9%
95.8%
93.8%
86.9%
94.6%
94.6%
96.4%
98.8%
89.9%
94.9%
92.6%
96.1%
92.9%
94.9%
95.5%
84.5%
95.8%
97.3%
93.8%
95.5%
96.4%
97.0%
96.4%
95.8%
97.3%
90.8%
736
736
736
736
736
35
504
736
736
195
736
736
736
736
470
736
736
504
736
736
736
736
736
736
736
736
736
470
59
42
58
54
78
3
43
49
28
2
86
70
64
50
59
38
40
61
68
82
51
69
44
68
104
62
32
91
92.0%
94.3%
92.1%
92.7%
89.4%
91.4%
91.5%
93.3%
96.2%
99.0%
88.3%
90.5%
91.3%
93.2%
87.4%
94.8%
94.6%
87.9%
90.8%
88.9%
93.1%
90.6%
94.0%
90.8%
85.9%
91.6%
95.7%
80.6%
1,072
1,072
1,072
1,072
1,072
371
840
1,072
1,072
531
1,072
1,072
1,072
1,072
806
1,072
1,072
840
1,072
1,072
1,072
1,072
1,072
1,072
1,072
1,072
1,072
806
79
53
75
68
99
47
61
67
40
6
120
87
89
63
83
55
55
113
82
91
72
84
56
78
116
76
41
122
92.6%
95.1%
93.0%
93.7%
90.8%
87.3%
92.7%
93.8%
96.3%
98.9%
88.8%
91.9%
91.7%
94.1%
89.7%
94.9%
94.9%
86.5%
92.4%
91.5%
93.3%
92.2%
94.8%
92.7%
89.2%
92.9%
96.2%
84.9%
0
0
0
6
6
13
0
0
0
0
6
6
6
0
21
0
0
18
9
21
6
0
0
0
48
0
0
52
8
4
8
13
15
18
5
6
1
0
16
13
12
7
17
3
4
31
19
21
13
10
3
14
42
9
2
45
218
359
Passed
Written IP Totals
Average
Median
StDev
56.57
53.00
17.30
336
336
0
16.14
15.00
5.84
95.20%
95.54%
1.74%
736
736
0
59.71
59.00
18.76
91.89%
91.98%
2.55%
1072
1072
0
75.86
76.00
21.01
92.92%
92.91%
1.96%
5.43
0.00
11.01
11.38
10.00
8.96
Fail
Totals
Average
Median
StDev
92.67
111.50
42.08
336
336
0
30.00
27.50
15.63
91.07%
91.82%
4.65%
453.17
487.00
228.29
49.83
51.00
29.00
88.86%
89.66%
4.76%
789.17
823.00
228.29
79.83
72.00
31.75
89.37%
88.52%
3.90%
17.33
15.50
19.16
20.00
17.50
15.75
19
Exercise 1
Write a Multiple Choice Question.
 Parameters:

– “Real World” Situational
– Valid
 To standards
 To student level
– Well Referenced
20
2.
Psychomotor / Hand
Realistic Manipulative Testing
 IP System

 Consequence
Based Sliding Scale
21
Real
22
Grade Sheet
OBJECTIVE:
Measurement of Recruit capability to complete inverted “L” cutting technique on a roll-up door and disable hinges with
a gasoline powered rotary saw and metal cutting blade.
SCENARIO:
Recruit is member of first arriving truck crew at a commercial building fire, one roll-up door and an outward swinging
conventional hinged door need to be rapidly failed to facilitate engine crew entry.
PROPS & EQUIPMENT NEEDED:
1. Roll-up door simulator with available material. (Each student will use similar material for the test.)
2. Available rotary saw, either K-12 or 950, inboard or outboard, with new blade. The student is to attach blade
and test the specific saw used in the test. (Note the occasional difficulty with the 950 flange.)
3. Three sections of heavy gauge flat-steel (4” vertically) attached at points next to roll-up door to simulate hinges
(ends exposed minimum of 2" horizontally).
4. Full P.P.E. for Recruits, including SCBA.
ESTABLISH TESTING ENVIRONMENT:
1. The entire inverted “L” maneuver is not done, but will be demonstrated after the manipulative phase of the test.
2. Available “Cut Lines” are identified by the instructor. The Student will identify the specific line used for the
test.
a. Horizontal cut minimum (5’) long above shoulder height.
b. Vertical cut minimum (4’)
3. “Hinge” cuts are then cut in order – lowest, middle, highest.
4. Start time when instructor tells Recruit to lock-in regulator
5. End time when Recruit stops saw and disconnects regulator.
6. Inverted “L” maneuver is explained to the instructor / drawn on dry erase board.
GRADING CRITERIA:
2-Improvement Points
1.
2.
3.
4.
Poor feed speed, RPM’s decrease
Horizontal cut below shoulder height
Each occurrence of poor control of saw. (i.e., bouncing, delay in starting cut)
Inverted “L” drawing incorrect (direction / numbering)
5-Improvement Points (AND FAILURE OF EVENT)
1.
2.
3.
4.
5.
6.
7.
8.
9.
Incomplete cut on roll-up door.
Incorrect hinge cutting sequence.
Lateral or side loading of blade. (Indicated by diagonal, notched or offset kerf or blade fragmentation.)
Using building / wall to stabilize self / saw during hinge cuts
Inability to accomplish task within 4-minutes
Unable to explain the inverted “L” process
Unsafe acts / inability to control saw.
Inability to start saw
Cut begins / extends 6” beyond indicated points.
23
TEST SEQUENCE
1._____ Instructor reviews criteria
2._____ Recruit in full P.P.E. - regulator not locked in
3._____ Recruit checks equipment, starts and stops saw after saw is warm.
4._____ Start time is when instructor tells student to lock in regulator
5._____ Recruit starts saw, makes (minimum 5’) horizontal cut at shoulder height
6._____ Recruit makes (minimum 4’) vertical inverted “L” cut
7._____ Recruit cuts 3 “hinges” (bottom, middle, top)
8._____ Saw is stopped by Recruit following completion of all cuts.
9._____ Regulator is disconnected from face piece. This is evolution stop time
10.____ Explanation (verbal/drawing) of inverted “L”
PASS/FAIL: ________________
TIME: _____________________
TOTAL IMPROVEMENT POINTS: _______________
EVALUATOR: ______________________________
COMMENTS: _______________________________
____________________________________________
_____________________________________________
Tape number and time:____________________________
24
Practice, Practice, Practice…….
25
Coaching & “Face Time”
26
Establish Testing Environment
27
Video
28
29
Outcomes
 Mid
–Course
&
 End Point
30
Day 17 -AM- 11/17/03 (SCBA)
Day 18 - AM - 11/18/03 (SCBA)
Day 18 - PM - 11/18/03 (SCBA)
Day 19 - AM - 11/20/03 TEST
83
122
109
102
102
Girard, Cory
73
110
89
93
74
Hogan, Joshua
100
120
111
92
92
Kopstein, Paul
105
206
132
118
137
Medina, Henry
135
190
127
132
117
Nicoll, Thomas
133
80
105
110
109
Scott, Kevin
78
118
113
115
112
Strecker, Daren
71
124
168
111
98
Wirtz, Mike
72
90
79
82
78
Mean
94.44
128.89
114.78
106.11
102.11
Standard Dev.
25.47
42.14
25.91
15.41
19.61
Median
83.00
120.00
111.00
110.00
102.00
POINTS
Day 16-PM - 11/14/2003 (No SCBA)
Barron, Stephen
2
2.00
All tests done with Sheet Curtatin segments. First introduction to skill - AM Rotation,
Day16. First timed evolution noted above. First rotation done to test format, Day 17-AM.
No PM Rotation on Day 17.
31
Exercise 2
Develop a manipulative test.
 Parameters:

– “Real World” Situational
– Valid
 To standards
 To student level
– Well Referenced
32
Grade Sheet
OBJECTIVE:
Measurement of Recruit capability to complete inverted “L” cutting technique on a roll-up door and disable hinges with
a gasoline powered rotary saw and metal cutting blade.
SCENARIO:
Recruit is member of first arriving truck crew at a commercial building fire, one roll-up door and an outward swinging
conventional hinged door need to be rapidly failed to facilitate engine crew entry.
PROPS & EQUIPMENT NEEDED:
1. Roll-up door simulator with available material. (Each student will use similar material for the test.)
2. Available rotary saw, either K-12 or 950, inboard or outboard, with new blade. The student is to attach blade
and test the specific saw used in the test. (Note the occasional difficulty with the 950 flange.)
3. Three sections of heavy gauge flat-steel (4” vertically) attached at points next to roll-up door to simulate hinges
(ends exposed minimum of 2" horizontally).
4. Full P.P.E. for Recruits, including SCBA.
ESTABLISH TESTING ENVIRONMENT:
1. The entire inverted “L” maneuver is not done, but will be demonstrated after the manipulative phase of the test.
2. Available “Cut Lines” are identified by the instructor. The Student will identify the specific line used for the
test.
a. Horizontal cut minimum (5’) long above shoulder height.
b. Vertical cut minimum (4’)
3. “Hinge” cuts are then cut in order – lowest, middle, highest.
4. Start time when instructor tells Recruit to lock-in regulator
5. End time when Recruit stops saw and disconnects regulator.
6. Inverted “L” maneuver is explained to the instructor / drawn on dry erase board.
GRADING CRITERIA:
2-Improvement Points
1.
2.
3.
4.
Poor feed speed, RPM’s decrease
Horizontal cut below shoulder height
Each occurrence of poor control of saw. (i.e., bouncing, delay in starting cut)
Inverted “L” drawing incorrect (direction / numbering)
5-Improvement Points (AND FAILURE OF EVENT)
1.
2.
3.
4.
5.
6.
7.
8.
9.
Incomplete cut on roll-up door.
Incorrect hinge cutting sequence.
Lateral or side loading of blade. (Indicated by diagonal, notched or offset kerf or blade fragmentation.)
Using building / wall to stabilize self / saw during hinge cuts
Inability to accomplish task within 4-minutes
Unable to explain the inverted “L” process
Unsafe acts / inability to control saw.
Inability to start saw
Cut begins / extends 6” beyond indicated points.
33
3.
Affective / Heart

Multiple Simulated Emergencies
Artificial Stressors
 “Non-Exam” Rubric

IP System
 Multiple Formal Counseling Sessions


Crew Resource Management
 Attitudinal
/ Behavioral Rubric
34
The “attitudinal standard”
is
“Introduction to Recruit
Fire Training”



Acts as a Training Contract.
Referenced throughout Academy.
Students tested early to establish level of
understanding.
35
The "offer of employment" letter sent to Recruits
in October of 1940 by Chief A. C. Duree contains
a description of professional behavior that rings
true today.
"You have been selected from the eligible
list for appointment to the position of firefighter
on this department. The fact that you passed
your examination and placed on the list shows
you were anxious to secure this position. I want
you to understand that it is up to you to make
your position here secure. You will be
furnished with the necessary knowledge
and material required for the duties of
firefighter, and it will be up to you to hold
the job. [Emphasis added]
36
Activities Log

Provides consistent exposure.
37
Nozzle
W - Saw
W - Saw
D- Sound
3 - Hydrant
D- Sound
E - Engineer
E - Engineer
H - F E Haligan
5 - Captain
5 - Captain
H - F E Haligan
Total Nozzle
Total Nozzle
R - F E Rotary
P - Propane
P - Propane
R - F E Rotary
T - Tower
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
V- Vehicle
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
S - Safety
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
S - Safety
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
T - Tower
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
D - Debris
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
D - Debris
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
V- Vehicle
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 - O'Cady
Totals
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 - O'Cady
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Positions
3 - Hydrant
Nozzle
Positions
38
“Non-Exam” Rubric


Provides descriptive consistency.
Consequence based sliding scale.
39
Point
Allocation
0
1
2
3
4
5
S
Safety
D
Delay
ET
Efficiency &
Team Work
Routine good
and safe
practice.
Actions cause
no delay in
any operation
Efficient and optimal
performance.
No potential
injury to self
or others, but
poor practice.
Potential for
minor injury,
to self or
others, if uncorrected.
Potential for
moderate
injury, to self
or others, if
not corrected
Potential for
major injury,
to self or
others. Some
intervention.
Instructor
intervention
required to
prevent injury.
Minor delay
Self corrected
Minor delay
Requires
Prompting
DF
Drill Format
Compliance
Competent the
grade sheet
parameters
Some minor
Efficient, but less then
elements self
optimal. Hesitation is
corrected during the
observed.
evolution.
Minimal efficiency.
Multiple minor
Occasional extra
elements major selfmovements. Poor
correction.
recognition of needs
of others
EF
EK
Error & Frequency Equipment
Prompt Response Knowledge & Use
No observable
errors. Requests
Feedback.
Few minor errors.
Student readily
recognizes errors.
Detailed
knowledge.
Good use of maint.
& Storage
Minor
specifications are
not known or
acted on
Noticeable minor
errors with quick
recognition.
General equip.
specifications are
not known or
acted on.
Major delay
Requires
Prompting.
Inefficient.
Wasted Movements.
Requires prompts.
Minor elements are
overlooked.
Potential for delay.
Multiple errors
Student requires
explanation to
recognize errors.
Poor usage of
equipment
maintenance,
storage
Major delay
No attempt to
correct.
Inefficient.
Delays in operation.
Requires prompts.
Major elements are
ignored. Potential of
objective failure.
Multiple errors,
explanation of the
underlying reason
Requires major
prompting to
operate & use.
Serious delay
and/or failure
of the
operation.
“Fumbling”, multiple
redirects. Acts in an
isolated manner.
Behavior that causes
Argumentative
the failure of the
Poor response to
evolution’s
prompting.
objectives.
Unable to operate
or potential for
damage.
40
Why?



Manipulative Exams are a Contrivance.
Needed Consistent Means to Measure
Other Events.
Rubric Provides:




Documentation Language
Context (Consequence Based)
Enhanced Peer Review Process
“A work in progress.”
41
Exercise 3
Use the “Non-Exam” Rubric
 Document Justification for Improvement
Points:

– Only Use a Few Sentences
– Set Environment
– Describe Consequences
– Allocate Points
– Base Example on Personal Experience
42
“Non-Exam” Rubric
Point
Allocation
0
1
2
3
4
5
S
Safety
D
Delay
ET
Efficiency &
Team Work
Routine good
and safe
practice.
Actions cause
no delay in
any operation
Efficient and optimal
performance.
No potential
injury to self
or others, but
poor practice.
Potential for
minor injury,
to self or
others, if uncorrected.
Potential for
moderate
injury, to self
or others, if
not corrected
Potential for
major injury,
to self or
others. Some
intervention.
Instructor
intervention
required to
prevent injury.
Minor delay
Self corrected
Minor delay
Requires
Prompting
DF
Drill Format
Compliance
Competent the
grade sheet
parameters
Some minor
Efficient, but less then
elements self
optimal. Hesitation is
corrected during the
observed.
evolution.
Minimal efficiency.
Multiple minor
Occasional extra
elements major selfmovements. Poor
correction.
recognition of needs
of others
EF
EK
Error & Frequency Equipment
Prompt Response Knowledge & Use
No observable
errors. Requests
Feedback.
Few minor errors.
Student readily
recognizes errors.
Detailed
knowledge.
Good use of maint.
& Storage
Minor
specifications are
not known or
acted on
Noticeable minor
errors with quick
recognition.
General equip.
specifications are
not known or
acted on.
Major delay
Requires
Prompting.
Inefficient.
Wasted Movements.
Requires prompts.
Minor elements are
overlooked.
Potential for delay.
Multiple errors
Student requires
explanation to
recognize errors.
Poor usage of
equipment
maintenance,
storage
Major delay
No attempt to
correct.
Inefficient.
Delays in operation.
Requires prompts.
Major elements are
ignored. Potential of
objective failure.
Multiple errors,
explanation of the
underlying reason
Requires major
prompting to
operate & use.
Serious delay
and/or failure
of the
operation.
“Fumbling”, multiple
redirects. Acts in an
isolated manner.
Behavior that causes
Argumentative
the failure of the
Poor response to
evolution’s
prompting.
objectives.
Unable to operate
or potential for
damage.
43
Review

Established academic parameters.


Established basic manipulative standards.


Head
Hand
Began operational capability assessment.

Heart?
44
This is sufficient for most
Students most of the time.
However…..
45
Attitudinal / Behavioral Rubric
46
Assertiveness
(AS)
Communication
(CM)
Decision Making
(DM)
Leadership (LD)
Mission Analysis
(MA)
Situational
Awareness (SA)
►Effective
utilization of pre and
post incident
analysis.
►Develops multiple
feasible courses of
action
►Maintains control
►Timely
assessment
►Multiple
information sources
►Monitors results
►Able to rapidly
alter tactics
►Utilizes multiple
communication
forms
►Recognizes and
overcomes
communication
filters
►Utilizes “OODA
Loop” in analysis
and implementation
of problem solving
strategies
►Acts as, and is
accepted as, a
functional /
situational leader
►Excels in fluid
environment
► Anticipates
problems
► Identifies change
and asks for
assistance
►Communicates if
comfort level is
exceeded
►Establishes
dialogue to offset
group mindset
►Active listener
►Paraphrases
feedback
►Succinct transfer
of complex
information
►Corrects
misunderstandings
►Verifies
information
►Complete
assessment of new
problem / situation
►Develops a
hypothesis
As a designated
leader:
►Decisive
►Professional
►Provides
feedback
►Leads by example
►Effective multielement planning
►Able to monitor
and respond to
changing mission
parameters
►Identifies actual
and potential
problems
Occasionally
succumbs to:
►Fatigue / Stress
►Task overload
►Group mindset
The willingness &
readiness to
actively participate,
state and maintain
a position, until
convinced by
alternate facts.
The clear and
accurate sending
and receiving of
information,
instructions, or
commands, and
providing useful
feedback.
The ability to use
logical and sound
judgment; to make
decisions based on
available
information.
Recognizes Error
Troika.
The ability to direct
and coordinate the
activities of others
and to encourage
the crew to work
together as a team.
The ability to
develop short term,
long-term and
contingency plans,
as well as to
coordinate, allocate
and monitor crew
and resources.
The degree of
accuracy by which
one's perception of
his / her current
environment mirrors
reality.
►Impulsive
►Resignation
►Avoids conflicts
►Overly courteous
►Limited feedback
►Poor conveyance
►Inappropriate
timing
►No recognition of
misunderstandings
►Dependant on
SOP’s
►Does not cross
check information
►Unable to define
rational for decision
►Passive
►Does not delegate
►Limited
information transfer
Does not routinely
utilize opportunities
for:
►Planning
►Preparation
►Pre & Post
incident briefs
Identifies situational
elements but is
unable to:
►Prioritize
►Assign values
►Respond to
changes
►Anti-authority
►Abusive / Hostile
►Invulnerability
►Macho
►No feedback
►Little or no
dialogue
►Inaccurate
►Inappropriate
modalities
►Dependant on
specific and clear
direction prior to
acting on simple
tasks.
►Blames others
►Divisive
►Overbearing
►Arbitrary
►Demeaning
►Lets others take
charge
►Places others at
risk
►Requires close
direction
►Not able to define
tasks or resource
allocation
►Ignores problems
►Misinterprets or
confuses key
situational
parameters
►Unable to
determine cause
and effect
►Places self at risk
Nominal
►Appropriate
advocacy of
alternatives.
►Plans ahead
►Provides useful
information
►Drives events
The ability to alter a
course of action
when new
information becomes
available.
Needs
Improvement
► Determine if SOP
is appropriate
► Offers and
advocates for
alternative solutions
►Quick recovery
after redirection
► Requires specific
prompting to develop
appropriate response
to changing
environment and/or
task.
Unsatisfactory
Optimal
Outstanding
Adaptability /
Flexibility (AF)
► Unable to respond
to changing
environment and/or
task.
►Does not respond
to redirection
47
Crew Resource Management
Explanation to Follow
48
Why?


“Head & Hand” Methods Well Established
Poorly Defined Acceptable Attitudinal /
Behavioral Parameters.


Uses:




“That was weird!”
Documentation Tool / No Points
Help Define “Intangibles”
Context for Peer Review
“A work in progress.”
49
4.
Data Collection
&
Continual Peer Review.
50
LBFD 2004 A Improvement Points
IP Collection (Student / Points)
Block
Written Tests
M & Q1 M2
M3 M4 & Q2
Day #
Day 1 Day 2 Day 3 Day 4
Day
Tues Wed Thurs
Fri
Date
09/07/04 09/08/04 09/09/04 09/10/04
Bayley, J.
57
Bohm, E.
54
Borders, I.
50
Chan, A.
38
Chang, M.
83
Dano, L.
28
4
4
De Hart, A.
111
Demetropolis, A. 33
Farley, J.
50
Filson, M.
4
Healy, B.
72
Hopkins, D.
86
4
Johnson, J.
53
Johnson, T.
42
Kepner, K.
113
Knapp, K.
32
Lawrence, M.
41
Lee, D.
112
4
4
Lee, J.
81
3
Magana, M.
53
Marantz, N.
62
Marshal, C.
43
Oh, D.
34
Rindone, K.
74
Rose, K.
76
Rosenstein, J. 65
Van Egdom, M. 63
White, M.
138
Points per Day
0
8
12
3
All Students
TOTAL:
1748
AVERAGE
62.43
MEDIAN
55.50
STDEV
30.10
Date
09/07/04 09/08/04 09/09/04 09/10/04
Day
Tues Wed Thurs
Fri
Day #
Day 1 Day 2 Day 3 Day 4
Block
EMT
Block 1
Block 2
M5
M6
M7
M8
M9+
Q3
M Final
Q4
Q5
B1
Q6
Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Day 11 Day 12 Day 13 Day 14 Day 15 Day 16 Day 17 Day 18 Day 19 Day 20 Day 21 Day 22 Day 23 Day 24 Day 25 Day 26 Day 27 Day 28
Mon Tues Thurs Fri
Mon Tues Thurs
Fri
Mon Tues Thurs
Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
09/13/04 09/14/04 09/16/04 09/17/04 09/20/04 09/21/04 09/23/04 09/24/04 09/27/04 09/28/04 09/30/04 10/01/04 10/04/04 10/05/04 10/07/04 10/08/04 10/11/04 10/12/04 10/14/04 10/15/04 10/18/04 10/19/04 10/21/04 10/22/04
10
1
1
1
1
3
1
1
5
4
1
1
1
5
1
3
5
2
5
1
1
6
1
9
3
6
1
10
10
15
5
1
3
3
4
1
10
9
1
10
4
5
4
5
4
6
10
10
2
2
10
1
1
9
3
1
15
6
2
1
2
2
1
1
4
5
5
1
6
16
1
10
1
1
4
4
6
5
1
2
10
1
9
1
3
4
1
2
1
1
1
1
1
1
4
3
1
7
5
5
1
2
1
3
1
1
4
10
6
1
7
2
6
2
3
8
0
0
0
15
0
0
150
0
0
7
1
6
2
0
0
113
27
21
10
25
49
14
5
Block 3
Block 4
Block 5
Grad Week
B2
Q7
B3
Q8
B4
Q9
B5
Day 29 Day 30 Day 31 Day 32 Day 33 Day 34 Day 35 Day 36 Day 37 Day 38 Day 39 Day 40 Day 41 Day 42 Day 43 Day 44 Day 45 Day 46 Day 47 Day 48 Day 49 Day 50 Day 51 Day 52 Day 53 Day 54 Day 55 Day 56
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Wed Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
10/25/04 10/26/04 10/28/04 10/29/04 11/01/04 11/02/04 11/04/04 11/05/04 11/08/04 11/09/04 11/11/04 11/12/04 11/15/04 11/16/04 11/18/04 11/19/04 11/22/04 11/23/04 11/24/04 11/26/04 11/29/04 11/30/04 12/02/04 12/03/04 12/06/04 12/07/04 12/09/04 12/10/04
25
4
3
7
9
4
13
3
1
2
2
7
9
3
2
10
1
7
9
3
2
6
6
12
2
2
3
24
5
15
1
27
1
9
1
13
1
6
22
6
11
17
12
4
6
6
12
25
9
18
39
303
2
2
2
3
1
5
1
2
2
5
3
2
2
6
3
9
2
3
4
2
2
5
5
2
5
2
5
5
2
1
0
4
16
61
5
31
2
20
10
16
28
10
15
5
6
17
18
16
51
15
8
35
19
22
12
14
12
8
25
18
4
35
445
5
2
1
9
3
2
1
2
4
2
0
To Day 29 - Successful
To Day 37 - Successful
Total
AV.
MEDIAN
STDEV
Total
AV.
MEDIAN
STDEV
0
40
2
1
2
1
3
1
6
1
1
31
2
9
5
2
Total
AV.
STDEV
MEDIAN
747
28.73
23
18.43
7
6
6
1
To Day 21 - Successful
297
11
10.01
9
6
1
14
15
0
6
2
18
5
3
4
3
7
5
2
7
5
4
7
7
7
9
4
19
6
14
10
2
2
8
151
2
8
0
To Day 44 - Successful To Day 45 - Successful
0
0
0
20
5
23
5
4
20
2
3
7
86
0
To Day 53 - Successful
0
0
To Day 56 - Successful
Total
Total
1331
955 Total
1108
1242 Total
1188
AV.
AV.
51.19
43.41 AV.
50.36
56.45 AV.
56.57
MEDIAN
MEDIAN
41
39.5 MEDIAN
47.5
53.5 MEDIAN
53
STDEV
STDEV
30.76
15.11 STDEV
15.72
16.89 STDEV
17.3
09/13/04 09/14/04 09/16/04 09/17/04 09/20/04 09/21/04 09/23/04 09/24/04 09/27/04 09/28/04 09/30/04 10/01/04 10/04/04 10/05/04 10/07/04 10/08/04 10/11/04 10/12/04 10/14/04 10/15/04 10/18/04 10/19/04 10/21/04 10/22/04 10/25/04 10/26/04 10/28/04 10/29/04 11/01/04 11/02/04 11/04/04 11/05/04 11/08/04 11/09/04 11/11/04 11/12/04 11/15/04 11/16/04 11/18/04 11/19/04 11/22/04 11/23/04 11/24/04 11/26/04 11/29/04 11/30/04 12/02/04 12/03/04 12/06/04 12/07/04 12/09/04 12/10/04
Mon Tues Thurs Fri
Mon Tues Thurs
Fri
Mon Tues Thurs
Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Mon Tues Wed Fri
Mon Tues Thurs Fri
Mon Tues Thurs Fri
Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Day 11 Day 12 Day 13 Day 14 Day 15 Day 16 Day 17 Day 18 Day 19 Day 20 Day 21 Day 22 Day 23 Day 24 Day 25 Day 26 Day 27 Day 28 Day 29 Day 30 Day 31 Day 32 Day 33 Day 34 Day 35 Day 36 Day 37 Day 38 Day 39 Day 40 Day 41 Day 42 Day 43 Day 44 Day 45 Day 46 Day 47 Day 48 Day 49 Day 50 Day 51 Day 52 Day 53 Day 54 Day 55 Day 56
EMT
Block 1
Block 2
Block 3
Block 4
Block 5
Grad Week
51
Class IP (With)
2004A with Written
41
45.00
39.00
50.36
47.5
56.57
45.00
AVERAGE
30.92
MEDIAN
16.89
17.3
15.69
21.81
10.01
22.84
18.43
24.76
26.49
11
27.00
9
30.1
STDEV
23
10
53
52.00
30.76
30
55.5
53.5
28.73
35.27
40
56.45
59.68
44.55
50
49.21
50.03
51.19
60
20
62.43
70
Day 53
Day 56
0
Day 21
Day 37
Day 29
Day 45
Day 53
Day 37
Day 56
Day 45
Total
Total
52
Class IP Comparison (With)
IP "With" - 2001A to 2004A
50.03
38.75
44.55
50
35.27
40
25.90
30.92
MEDIAN
20.81
21.52
23.03
21.81
22.84
24.76
10.57
STDEV
28.06
25.12
50.79
AVERAGE
31.00
21.00
19.59
26.49
19.96
18.00
48.50
41.00
45.00
45.00
39.00
27.00
20
49.00
52.00
53.00
30.46
30
10.45
45.75
49.21
59.68
60
51.44
60.31
70
10
9.79
7.00
7.00
0
21
DayDay
29 21
Day 37 Day 29 Day 45
Day Day
37 53
Day
56
Day 45
Total
Day 53
Day 56
Total
53
2004A - Without Written
70
35.27
45.00
50.67
24.88
16.04
13.84
14.2
9.7
MEDIAN
15.67
21.81
22.84
23.41
24.76
26.49
10.45
50.82
27.00
Day
53
Total
Day 56
7.00
7.11
0
AVERAGE
35.5
18.00
9
9.79
0.00
10
19.96
0.00
45.00
43
39.00
21
0.00 20
52
50
STDEV
24.4
0.00 30
52.00
50
30.92
25.12
0.00 40
49.21
43.62
0.00 50
44.73
44.55
50.03
0.00 60
54.29
59.68
0.00
Class IP (Without)
Day 21Day
21 Day 29
DayDay
2937
Day
Day
45 37
Day 53 Day
45Day 56
Total
54
Class IP Comparison (Without)
IP "Without" - 2001A to 2004A
55.25
70
49.00
43.50
49.21
44.55
35.27
9.34
10.45
25.12
30.92
Day 45
Day 53
21.81
Day 37
22.84
19.96
Day 29
18.00
27.00
24.76
Day 21
AVERAGE
STDEV
26.49
0
18.81
15.91
7.92
7.00
45.00
19.50
39.00
20.48
23.44
18.00
45.00
52.00
27.71
22.28
30.00
20
40.00
50.03
37.00
30
50.00
20.00
44.00
59.68
36.68
40
60.00
10
30.00
40.98
50
70.00
46.31
46.96
60
Day 56
MEDIAN
Total
55
Topic Comparison
56
IP Comparison as of Pre-Block 3 Exam
EMT Quiz
EMT Exam
Ops Quiz
Ops Exam
EMT Skills
Ops Skills Tests
Notes:
Characterized by
training phase and
primary function..
(i.e. SCBA donning
points accrued during
hose lays are typed as
a hose lay point.)
Knots
Knots Practice
Ladders
Ladder Practice
Hose lays
Hose lay Practice
SCBA
SCBA Practice
Force Entry / Tools
F E / Tools Practice
Roof
Roof Practice
Auto Ext
Auto Ext. Practice
Skills Misc. (Hose Ad.)
Operations
Notes:
Characterized by type
only.
(i.e. SCBA points
during a live fire are
typed as "PPE/SCBA".
Misc
Total
Knots
Ladders
Hose lays
PPE / SCBA
Force Entry / Tools
Roof
Auto Ext.
Misc.
198
54
12
0
42
0
115
411
44
0
31
0
134
12
35
0
10
0
103
22
0
0
20
66
0
12
23
21
0
5
0
5
15
661
8.17%
1.82%
0.00%
6.35%
0.00%
17.40%
62.18%
6.66%
0.00%
4.69%
0.00%
20.27%
1.82%
5.30%
0.00%
1.51%
0.00%
15.58%
3.33%
0.00%
0.00%
3.03%
9.98%
0.00%
1.82%
3.48%
3.18%
0.00%
0.76%
0.00%
0.76%
2.27%
2005A
2004A
2002B
Academy
Written if new used
Written
219
121
28
0
51
42
165
437
62
0
87
5
112
33
13
0
1
0
94
21
0
0
9
185
0
27
28
12
6
93
0
19
10
918
13.18%
3.05%
0.00%
5.56%
4.58%
17.97%
47.60%
6.75%
0.00%
9.48%
0.54%
12.20%
3.59%
1.42%
0.00%
0.11%
0.00%
10.24%
2.29%
0.00%
0.00%
0.98%
20.15%
0.00%
2.94%
3.05%
1.31%
0.65%
10.13%
0.00%
2.07%
1.09%
342
93
43
76
130
190
710
42
0
26
0
125
43
66
0
43
0
310
55
0
0
0
91
0
20
20
34
6
5
0
6
25
1,358
25.18%
6.85%
3.17%
5.60%
9.57%
13.99%
52.28%
3.09%
0.00%
1.91%
0.00%
9.20%
3.17%
4.86%
0.00%
3.17%
0.00%
22.83%
4.05%
0.00%
0.00%
0.00%
6.70%
0.00%
1.47%
1.47%
2.50%
0.44%
0.37%
0.00%
0.44%
1.84%
57
5. Feedback



Regular
Formal
Specific
58
Counseling

Regular Contact with Recruit.



Constant Review with Legal.


Benchmark Points @ 25, 50 & 75 points.
Audio Recording / Formal
Established relationship with City Attorney
Early Use of Referrals and Constructive
Action Plans.
59
Example - 75 Points
60
LONG BEACH FIRE DEPARTMENT
RECRUIT TRAINING
COUNSELING FORM
NAME: Recruit Xxxxx
DATE: November 15, 2002
Regular Counseling
LOCATION: Training Center
REASON FOR COUNSELING: Accrual of 75 Improvement Points
During the block III skills exam Recruit Xxxxx accrued 28 improvement points bringing his total to 93
improvement points. According to The Introduction to Recruit Fire Training, a formal counseling will take
place whenever 75 points are accrued.
The firefighter's job is one of the most hazardous of all occupations. Recruit firefighters are trained to
function in a safe and proficient manner consistent with the aggressive fire fighting nature of the Department.
Carelessness in any work routine, drill, or simulated emergency can result in serious injury or death.
Therefore, up to five improvement points will be assessed for each safety violation. Repeated failure to
comply with Department safety policies, practices and procedures, will result in a recommendation for
termination.
Recruit Xxxxx has been accumulating improvement points at a rapid rate since starting the operations
portion of the Recruit Academy. Recruit Xxxxx has received coaching sessions and formal counseling
sessions at 25, and 50 improvement points. In those sessions, among other things, we discussed safety as it
applies to both manipulative exercises and simulated emergency responses and attitude.
“As a Recruit you are expected to make mistakes while learning new skills. That is why you are in the
Academy. If you make a mistake, admit it, correct it and move on to the next challenge. But, mistakes
cannot go on forever.”
Recruit Xxxxx listens attentively during lectures and asks questions pertinent to the subject. He
accepts responsibility for his mistakes, listens attentively to corrective coaching and tries to apply the
techniques or information in subsequent evolutions.
Recruit Xxxxx is to be cautioned that the points he accrued today represent potential delays in
evolutions or safety concerns that jeopardize his safety, that of his crew or any potential victims that he may
encounter during emergency operations. The consequences of these actions have been emphasized during
classroom lectures, NIOSH reports, and “impact” videos. The rapid accrual of improvement points have raised
some serious concerns among the Training Staff as to Recruit Xxxxx’s ability to operate safely in the field.
Recruit Xxxxx is cautioned that 93 improvement points is a very significant amount at this stage in the
academy. Recruit Xxxxx is cautioned that safe, competent performance of skills and satisfactory academic
performance is necessary to successfully pass this academy. If 110 improvement points are accrued a
recommendation for termination will be forwarded to the Director of Training.
I HAVE BEEN COUNSELED
ON THIS DATE
COUNSELORS
SIGNATURE
61
After all of this, the most
effective tool is…..
62
Lunch.
63
Break ?
64
Attitudinal Measurement

Oxymoron


Can Measure Behavior, not Attitude.
Consider Crew Resource Management as
an effective model.


Genesis of Attitudinal / Behavioral Rubric
“Consequence based sliding scale.”
65
Crew Resource
Management
for
Long Beach Fire
Department
66
An Introduction
67
Definition-
An error management approach
defining behavioral strategies taught
as error countermeasures that are
employed to:



avoid error,
to trap errors committed
and to mitigate the consequences of
error.
68
“In the ten years it will take CRM to be
introduced nationally, we will attend 1000
firefighter funerals…
I can’t get that out of my mind.”
Gary Briese,
Executive Director
IAFC
69
Multiple Perspectives




Origin from Military Aviation over 30 years
ago.
Integrated to Commercial Aviation 20
years ago.
Discussed as a management tool in
medicine 10 years ago,
Fire Service text from 2004.
70
Basic Premises



Technology has greatly minimized “Tool
Error”.
Human Factors primary causative agent
for errors.
Rigid, hierarchal organizations especially
prone to this type of failure.
71
An Error Mitigation Troika



Training on how to avoid errors.
Potential errors are "trapped" before they
are committed.
Planning for mitigation of error
consequences.
72
Format 1
Naval Air
73

Although the following was designed for
Military Aviation, multiple terms are easily
transferred to our environment.

“Mission” or “Flight” = “Run”
74
Reference to your own
experience.
How do these descriptions compare to
your well functioning teams?
75
Crew Resource Management
Naval Air
https://wwwnt.cnet.navy.mil/crm/crm/stand_mat/seven_skills/sev_skills.asp
76
Basic Elements







Adaptability / Flexibility (AF)
Assertiveness (AS)
Communication (CM)
Decision Making (DM)
Leadership (LD)
Mission Analysis (MA)
Situational Awareness (SA)
77
ADAPTABILITY/FLEXIBILITY (AF)
WHAT IS ADAPTABILITY/FLEXIBILITY?
The ability to alter a course of action
when new information becomes available.
78
SITUATIONS THAT REQUIRE QUICK ADAPTATION
When:
Un-briefed Situations Arise
A Routine Mission Becomes an Emergency
Transitions Occur
A Crew Member is Incapacitated
Interactions are Strained
79
MAINTAINING ADAPTABILITY / FLEXIBILITY
Anticipate Problems
Recognize and Acknowledge any change
Determine if an SOP or Habitual Response is Appropriate
Offer alternative solutions
Provide and Ask for Assistance
Interact Constructively with Others
80
SETTING THE TONE FOR
ADAPTABILITY / FLEXIBILITY
Establish an open, professional atmosphere
Ensure the crew understands the mission
81
“AF” Example

“During live fire exercises, Recruit Jones
did not respond to instructor prompting
that the exposure to the debris fire was
smoking.”
82
Assertiveness (AS)
WHAT IS ASSERTIVENESS?
The willingness/readiness to actively participate,
state and maintain a position,
until convinced by the facts
that other options are better.
Requires the initiative and the courage to act.
83
COMFORT LEVEL
(Risk Homeostasis)
Comfort level is the degree to which you feel comfortable
with what is happening, while taking into account that
flying a mission can be dangerous and demanding.
Whenever comfort level is exceeded, "Speak Up".
84
BEHAVIOR CONTINUUM
PASSIVE
Overly
courteous
"Beats around
the bush"
Avoids
Conflicts
"Along for the
ride."
ASSERTIVENESS
OVER
AGGRESSIVENESS
Active Involvement
Readiness to take Domination
action
Intimidation
Abusive / Hostile
Provide useful
information
Makes suggestions
85
BARRIERS TO ASSERTIVENESS
Position of Authority
Experience
Rank
Lack of Confidence
Fear of Reprisal
86
“AS” Example


“Recruit Jones continues to stay at the
back of the group during field training”
“Recruit Jones has interrupted field
training with irrelevant comments several
times.”
87
Communication (CM)
WHAT IS COMMUNICATION?
Communication is the clear and accurate
sending and receiving of
information,
instructions, or commands, and providing useful feedback.
88
TYPES OF COMMUNICATION
Verbal (Involves Words)
Spoken
Written
Non-Verbal (Everything but Words)
Gestures
Voice Intonation
89
PROCESS OF COMMUNICATION
Sender Conveys ideas or information to others
Receiver hears or takes in information
and provides feedback
90
SENDER’S RESPONSIBILITIES
Communicate in Appropriate Mode
Verbal versus Non-verbal
Convey Information Accurately and Concisely
Provide Information at Appropriate Time
Request Verification or Feedback
91
RECEIVERS RESPONSIBILITY
Actively Listen
Active Vs. Passive Role
Take action as a result of Communication
Answer, or Respond, to Communications
Ask for Clarification of Unclear Communication
92
BARRIERS TO COMMUNICATION
Noise
Rank/Experience
Task Overload
Gender
Attitudes
Culture
93
“CM” Example


“Recruit Jones gives ladder commands is
an overly subdued manner.”
“During training evolutions, after
redirection is given, Recruit Jones often
looks away from the speaker and does not
respond.”
94
Decision Making (DM)
What is Decision Making?
Effective decision making refers to the ability to use
logical and sound judgment
to make decisions based on available information.
95
This includes:
Assessing the problem
Verifying information
Identifying solutions
Anticipating consequences of decisions
Informing others of decision and rationale
Evaluating decisions
96
A Decision Making Strategy for
Troubleshooting:
Identify all the symptoms
Make a hypothesis as to the possible cause
Test your hypothesis
Apply appropriate remedies
97
Another variation is
John Boyd’s OODA Loop
Observe
 Orientate
 Decide
 Act

98
99
Factors Which Promote
Good Decision Making:
Teamwork
Extra time to make a decision
Alert crew members
Decision strategies and experience
100
Barriers to Good Decision Making:
Barrier
Time
Inaccurate or
ambiguous info
How to Overcome
Use SOP's and select the best
decision using available information
Cross-check info
Pressure to perform
Evaluate the rationale for making a
decision
Rank Difference
Use assertive behaviors
101
Decision Strategy for Risk
Assessment:
(AESOP Model)
Once a hazard has been detected,
evaluate it to determine its potential effect on the planned
flight
by considering its impact on the:





Aircraft
Environment
Situation
Operation
People
102
The analysis should consider
the crew's relative ability
to cope with changes
in each of the five basic elements
listed in the AESOP model.
103
Remember:
Good decisions optimize risk management
and minimize errors,
while poor decisions can increase them.
Poor judgment or decision making
is a leading cause of failure
to complete missions and of mishaps.
Each decision affects your future options.
104
“DM” Examples

“Recruit Jones continued to open the
spreader on an auto extrication exercise in
spite of obvious indications that the
vehicle was becoming destabilized.”
105
Leadership (LD)
WHAT IS LEADERSHIP?
The ability to direct and coordinate the activities
of other crew members or wingmen,
and to encourage the crew to work together as a team.
106
TYPES OF LEADERSHIP
DESIGNATED LEADERSHIP
Responsible
Makes Final Decisions
Normal Mode of Leadership
FUNCTIONAL LEADERSHIP
Leadership by Knowledge or Expertise
Occurs when the Need Arises
107
RESPONSIBILITIES OF LEADERSHIP
Crew Performance
Direct Actions
Ask for Assistance
108
TRAITS OF AN EFFECTIVE LEADER
Respected
Open to Suggestions
Decisive
Builds Team Spirit
Delegates Tasks
Directs and Coordinates Activities
Provides Feedback
Maintains a Professional Atmosphere
Leads by Example
Keeps Crew Informed
Knowledgeable of how to do the Mission
109
“LD” Example

“As the assigned Truck Captain for the
Recruit Academy, Recruit Jones has been
noted to not assign tasks to other
Recruits. He attempts to accomplish
directives from the staff without
appropriate delegation.”
110
MISSION ANALYSIS (MA)
What is Mission Analysis?
Mission Analysis refers to the ability to develop
short term, long-term
and contingency plans, as well as to
coordinate, allocate and monitor
crew and aircraft resources.
111
PHASES OF MISSION ANALYSIS
include:
Preflight
In-flight
Post flight
112
PRE-MISSION ANALYSIS
involves:
Planning
Preparation
Briefing
113
CHARCTERISTICS OF AN
EFFECTIVE BRIEF
Professional
Stays Focused
Assigns Responsibilities
Interactive
Complete
114
IN-FLIGHT MISSION ANALYSIS
involves:
Short-term Planning
Monitoring Mission Progress
Identifying and Reporting Challenges or Changes
115
POST-MISSION ANALYSIS is:
Selective Review
Interactive
Timely
116
“MA” Example

“As the assigned Recruit Engine Captain,
Recruit Jones was unable to prioritize a
rescue task within the context of a live fire
exercise.”
117
SITUATIONAL AWARENESS (SA)
What is Situational Awareness?
Situational Awareness refers
to the degree of accuracy
by which one's perception of
his / her current environment
mirrors reality.
118
PERCEPTION VERSUS REALITY
View of Situation
Incoming information
Expectations & Biases
Incoming Information versus Expectations
119
FACTORS THAT REDUCE
SITUATIONAL AWARENESS:
Insufficient Communication
Fatigue / Stress
Task Overload
Task Under load
Group Mindset
"Press on Regardless" Philosophy
Degraded Operating Conditions
120
How does this relate to us?
121
Give Examples:
Insufficient Communication
Fatigue / Stress
Task Overload
Task Under load
Group Mindset
"Press on Regardless" Philosophy
Degraded Operating Conditions
122
“SA” Example


“Recruit Jones did not recognize or
respond to the kink in the supply line.”
“During the extrication simulation, Recruit
Jones stopped operation of the spreader
until the adjoining destabilized vehicle was
cribbed.”
123
Format 2
Crew Resource Management
For the
Fire Service
Okray & Lubnau 2004
124
Safety Culture



“Not another program!”
Individual Responsibility
Non-Punitive Culture and Policy to Error
125
Mission Analysis & Planning




Micro – Training Opportunities
Accountability
Risk Versus Gain Analysis
Risk Acceptance
126
Situational Awareness

Killer Equation

Reality times perception still equals reality.
A Enhancement Strategy:




Maintain Control
Timely Assessment
Multiple Sources
Monitor Results
127
Communications



Recognize different forms.
No assumptions
System Approach:




Inquiry
Advocacy
Monitoring (Situational Awareness)
Feedback
128
Effective Communication




Utilizes more than one form.
Both sender and receiver recognize
perceptions, influences and situations
(Filters) that affect the message.
Must have active listeners.
Communication loop (sending, receiving,
feedback)
129
CRM Leadership

Three Steps:




Introduction
Integration
Trust
Recognize formal and informal
(situational) leaders.
130
CRM “Followership”





Interaction
Listening Skills
Receiving, interpreting and following
instructions
Making decisions together.
Watching out for one another.
131
Hazardous Attitudes






Anti-authority
Impulsivity
Invulnerability
Macho
Resignation
Pressing
132
Decision Making




Define Problem
Generate a course of action
Evaluate a course of action
Carry out a course of action
133
Keys to Good Decision Making




Maintain good situational awareness.
Maintain technical proficiency
Know your resources
Evaluate, Evaluate, Evaluate
134
We have seen this stuff before!
135
HazMat & WMD







G – Gathering Information
E – Estimating Course and Harm
D – Determining Strategic Goals
A – Assessing Tactical Options & Resources.
P – Planning & Implementing Actions
E – Evaluating
R - Reviewing
136
Two Key Weaknesses
137
Debriefings & Critiques

Similar to CISD


“The Blue Line”
Template:





1.
2.
3.
4.
5.
Just the facts
What did you do?
What went wrong?
What went right?
What can be done?
138
“Mandatory refresher training wastes
the instructor's time, the firefighter’s
time and the citizen’s dollars. Saving
“a problem” for refresher training is a
disservice to our firefighters and the
public we protect.”
Page 254
139
Strategies for Implementation




Step 1 – Train to technical proficiency.
Step 2 – Train to CRM proficiency.
Train risk vs. gain.
Include CRM in department culture.
140
Strategies for Implementation

Step 1 – Train to technical proficiency.
Traditional perception of
“Training”.
We do this well!
141
Strategies for Implementation


Step 1 – Train to technical proficiency.
Step 2 – Train to CRM proficiency.
“Error Tolerant” Culture,
especially in Recruit
Training.
Assertiveness
142
Strategies for Implementation



Step 1 – Train to technical proficiency.
Step 2 – Train to CRM proficiency.
Train risk vs. gain.
“Fuzzy Logic”
Values are assigned to situational
elements.
Train “Experience”.
143
Strategies for Implementation




Step 1 – Train to technical proficiency.
Step 2 – Train to CRM proficiency.
Train risk vs. gain.
Include CRM in department culture.
?
144
Managing The
Unexpected
Jacksonville Florida
February 28, 2005
Presenters:
Karl Weick
Kathleen Sutcliffe
145
Rate Preoccupation
with Failure


Regard close calls and near misses as a
kind of failure that reveals potential danger
rather than as evidence of our success and
ability to avoid danger.
We treat near misses and errors as
information about the health of our system
and try to learn from them.
146
Rate Reluctance to
Simplify


People around here take nothing for
granted.
People are encouraged to express
different points of view.
147
Rate Sensitivity to
Operations


During an average day, people come
into enough contact with each other to
build a clear picture of the situation.
People are familiar with operations
beyond one’s own job.
148
Rate Commitment to
Resilience


There is a concern with building
people’s competence and response
repertoires.
People have a number of informal
contacts that they sometimes use to
solve problems.
149
Rate Deference to Expertise
 If something out of the ordinary
happens, people know who has the
expertise to respond.
 People in this organization value
expertise and experience over
hierarchical rank.
150
Mismanaging the Unexpected:
An Abrupt and Brutal Audit
151
Review
152
 Cognitive
/ Head
 Validated
/ Peer Reviewed Exams
 Psychomotor
 Realistic
 Affective
/ Hand
Manipulative Testing
/ Heart
 Multiple
Simulated Emergencies
 Artificial Stressors
 CRM
153
Feedback
and
Document !
154
Our Feedback

Exit Interview with Chief Officer.



They say “Thank You”
Multiple Post Academy Professional
Evaluations.
Graduates volunteer and come back to
help.
155
Exercise 3


Use Behavioral / Attitudinal Rubric
Describe Event to be used for counseling.





Use only a few sentences.
Describe Environment.
Discuss Potential Consequences.
Based on severity, what follow-up is
necessary.
Base example on personal experience.
156
Use “SOAPE” to
organize your document.
157
“SOAPE”
Format as below





1.
2.
3.
4.
5.
Subjective / Summary
Objective
Assessment
Plan
Evaluate
158
“SOAPE”
Develop Document as Below:





5.
1.
2.
3.
4.
Subjective / Summary
Objective
Assessment
Plan
Evaluate
159
Subjective / Summary
Introduction





Inform that there is a potential for discipline, if
applicable
Brief description of behavior, from your
perspective.
Distillation of the entire document in a few
sentences.
This section is presented first; however
written last.
160
Objective
Clear, critical description of behavior. (Anyone can
see it.)








No emotion
Chronologic
Frequency
Factual
Identify witnesses
Report the individual’s comment’s and responses
to the Counseling / Coaching.
Include actual negative consequences of action.
161
Assessment
(Explanation)
Reference the violation.






P & P’s
Standards
Common Practice
Crew Resource Management
Describe potential consequences of future
similar actions
162
Plan
(Administrative Consequences)
Potential Treatments:







EAP (Employee Assistance)
CAP (Constructive Action Plan)
Plan for Improvement
Discipline recommendation
Additional Training
Outline future potential progressive
discipline in general terms.
163
Evaluation
(Administrative Controls)
Remediation (Treatment) follow-up




Establish specific time frame
Establish specific behavior change(s) required.
Establish metrics
164
Assertiveness
(AS)
Communication
(CM)
Decision Making
(DM)
Leadership (LD)
Mission Analysis
(MA)
Situational
Awareness (SA)
►Effective
utilization of pre and
post incident
analysis.
►Develops multiple
feasible courses of
action
►Maintains control
►Timely
assessment
►Multiple
information sources
►Monitors results
►Able to rapidly
alter tactics
►Utilizes multiple
communication
forms
►Recognizes and
overcomes
communication
filters
►Utilizes “OODA
Loop” in analysis
and implementation
of problem solving
strategies
►Acts as, and is
accepted as, a
functional /
situational leader
►Excels in fluid
environment
► Anticipates
problems
► Identifies change
and asks for
assistance
►Communicates if
comfort level is
exceeded
►Establishes
dialogue to offset
group mindset
►Active listener
►Paraphrases
feedback
►Succinct transfer
of complex
information
►Corrects
misunderstandings
►Verifies
information
►Complete
assessment of new
problem / situation
►Develops a
hypothesis
As a designated
leader:
►Decisive
►Professional
►Provides
feedback
►Leads by example
►Effective multielement planning
►Able to monitor
and respond to
changing mission
parameters
►Identifies actual
and potential
problems
Occasionally
succumbs to:
►Fatigue / Stress
►Task overload
►Group mindset
The willingness &
readiness to
actively participate,
state and maintain
a position, until
convinced by
alternate facts.
The clear and
accurate sending
and receiving of
information,
instructions, or
commands, and
providing useful
feedback.
The ability to use
logical and sound
judgment; to make
decisions based on
available
information.
Recognizes Error
Troika.
The ability to direct
and coordinate the
activities of others
and to encourage
the crew to work
together as a team.
The ability to
develop short term,
long-term and
contingency plans,
as well as to
coordinate, allocate
and monitor crew
and resources.
The degree of
accuracy by which
one's perception of
his / her current
environment mirrors
reality.
►Impulsive
►Resignation
►Avoids conflicts
►Overly courteous
►Limited feedback
►Poor conveyance
►Inappropriate
timing
►No recognition of
misunderstandings
►Dependant on
SOP’s
►Does not cross
check information
►Unable to define
rational for decision
►Passive
►Does not delegate
►Limited
information transfer
Does not routinely
utilize opportunities
for:
►Planning
►Preparation
►Pre & Post
incident briefs
Identifies situational
elements but is
unable to:
►Prioritize
►Assign values
►Respond to
changes
►Anti-authority
►Abusive / Hostile
►Invulnerability
►Macho
►No feedback
►Little or no
dialogue
►Inaccurate
►Inappropriate
modalities
►Dependant on
specific and clear
direction prior to
acting on simple
tasks.
►Blames others
►Divisive
►Overbearing
►Arbitrary
►Demeaning
►Lets others take
charge
►Places others at
risk
►Requires close
direction
►Not able to define
tasks or resource
allocation
►Ignores problems
►Misinterprets or
confuses key
situational
parameters
►Unable to
determine cause
and effect
►Places self at risk
Nominal
►Appropriate
advocacy of
alternatives.
►Plans ahead
►Provides useful
information
►Drives events
The ability to alter a
course of action
when new
information becomes
available.
Needs
Improvement
► Determine if SOP
is appropriate
► Offers and
advocates for
alternative solutions
►Quick recovery
after redirection
► Requires specific
prompting to develop
appropriate response
to changing
environment and/or
task.
Unsatisfactory
Optimal
Outstanding
Adaptability /
Flexibility (AF)
► Unable to respond
to changing
environment and/or
task.
►Does not respond
to redirection
165
END
166