Transcript Evaluating Training Programs By Kirkpatrick

Kirkpatrick
The Four Levels
 Reaction
 Learning
 Behavior
 Results
All about Kirkpatrick
In 1959, Kirkpatrick wrote four articles describing
the four levels for evaluating training programs.
He was working on his dissertation for a Ph.D.
when he came up with the idea of defining
evaluation.
Evaluation, as according to Kirkpatrick, seems to
have multiple meanings to training and
developmental professionals. Some think
evaluation is a change in behavior, or the
determination of the final results.
All about Kirkpatrick
(continued)
 Kirkpatrick
says they are all right, and
yet all wrong. All four levels are
important in understanding the basic
concepts in training. There are
exceptions, however.
Kirkpatrick: Evaluating
Training Programs
 “What
is quality training?”
 “How
do you measure it?”
 “How
do you improve it?”
Evaluating
“The reason for evaluating is to
determine the effectiveness of a
training program.” (Kirkpatrick,
1994, pg. 3)
The Ten Factors of Developing
a Training Program
1.
2.
3.
4.
5.
Determine needs
Set objectives
Determine subject content
Select qualified applicants
Determine the best schedule
The Ten Factors of Developing
a Training Program
6. Select appropriate facilities
7. Select qualified instructors
8. Select and prepare audiovisual
aids
9. Co-ordinate the program
10. Evaluate the program
Reasons for Evaluating
Kirkpatrick gives three reasons ‘why’
there is a need to evaluate training:
1.“To justify the existence of the training
department by showing how
it contributes to the organizations’
objectives and goals.”
Reasons for Evaluating
2. “To decide whether to continue or
discontinue training programs.”
3. “To gain information on how to
improve future training programs.”
(Kirkpatrick, 1994, pg. 18)
The Four Levels
 Reaction
 Learning
 Behavior
 Results
“The Four Levels represent a
sequence of ways to evaluate
(training) programs….As you move
from one level to the next, the
process becomes more difficult and
time-consuming, but it also provides
more valuable information.”
(Kirkpatrick, 1994, pg. 21)
Reaction:
is the measuring of the reaction of
the participants in the training
program.
is “a measure of customer
satisfaction.” (Kirkpatrick, 1994,
pg. 21)
Learning:
is the change in the participants’
attitudes, or an increase in
knowledge, or greater skills
received, as a result of the
participation of the program.
Learning
The measuring of learning in any training
program is the determination of at least one
of these measuring parameters:
 Did the attitudes change positively?
 Is the knowledge acquired related and
helpful to the task?
 Is the skill acquired related and helpful to
the task?
Behavior
Level 3 attempts to evaluate how
much transfer of knowledge, skills,
and attitude occurs after the
training.
The four conditions Kirkpatrick
identifies for changes to occur:
 Desire
to change
 Knowledge
of what to do and
how to do it
 Work
in the right climate
 Reward
for (positive) change
When all conditions are met,
the employee must:
 Realize
an opportunity to use the
behavioral changes.
 Make the decision to use the
behavioral changes.
 Decide whether or not to continue
using the behavioral changes.
When evaluating change in
behavior, decide:
 When
to evaluate
 How
often to evaluate
 How
to evaluate
Guidelines for evaluating
behavior:
 Use
a control group
 Allow time for change to occur
 Evaluate before and after
 Survey/interview observers
 Get 100% response or sampling
 Repeat evaluation, as appropriate
 Consider cost versus benefits
Results
Level 4 is the most important and
difficult of all - determining final
results after training.
Evaluation Questions:
Increased production?
 Improved quality?
 Decreased costs?
 Improved safety numbers?
 Increased sales?
 Reduced turnover?
 Higher profits?

Guidelines for evaluating
results:
Use a control group.
 Allow time for results to be achieved.
 Measure before and after the program.
 Repeat the measurements, as needed.
 Consider cost versus benefits.
 Be satisfied with evidence if proof is not
possible.

Case Study #1
INTEL CORPORATION
Intel’s Compromise
of the Kirkpatrick Model
Intel uses the four-level model as
an analysis instrument to
determine the initial training needs
and design of its training program;
as well as using the model for
evaluations.
Intel’s Compromise
of the Kirkpatrick Model
Their uniqueness of using the model
is in the fact that the designers of
the training program worked
backwards in the analysis of the
training, starting with Level Four.
The Model
This implementation of the
Kirkpatrick Model stands as vivid
testimony to the versatility of the
model as a training tool, and in
developing fledgling training
programs.
The Model
It also reflects the open-mindedness
of the senior executives at Intel for
their infinite use of the model and
the use of the genius and visions of
Kirkpatrick.
How Intel applies the analysis
to their training program
Level Four …”Determine the
organizations’ structure and future
needs.”
Level Three. Change the
environmental conditions and
employee conditions to improve
business indicators.
How Intel applies the analysis
to their training program
Level Two. “Design a training
program that would ensure a transfer
of deficient skills and knowledge.”
Level One. Use a questionnaire,
according to their skill level, that
would instruct and inspire training
participants.
How Intel applies evaluation
to their training program
Level One - Questionnaire.
Level Two - Demonstrate competency,
create action plans through group
simulations.
Level Three - Follow-up to determine if
action plans were met (specific steps to
implement concepts of what was learned).
Level Four - Ongoing process of tracking
business indicators.
Case Study #2
ST. LUKE’S HOSPITAL
St. Luke’s is unique  Evaluation
of outdoor-based
training program, not classroom.
 Results analyzed statistically to
determine the significance of any
change.
 Evaluation led to recommendations
for future programs.
The New Questionnaire
Used before attendance in the program.
 Used 3 months after completion of the
program.
 Used again 6 months after completion of the
program.

(Communication showed statistically significant
improvement, and Group Effectiveness showed
statistically significant change.)
Kirkpatrick’s 4 Levels of
Evaluation are:
Level 1 - Reaction: how participants reacted
to the program.
 Level 2 - Learning: what participants
learned from the program.
 Level 3 - Behavior: whether what was
learned is being applied on the job.
 Level 4 - Results: whether that application
is achieving results.

Post-test Questions
(1) Name three ways evaluation results can
be measured.
(2) Do all 4 Levels have to be used?
(3) Do they have to be used in 1,2,3,4 order?
(4) Is Kirkpatrick’s method of evaluation
summative or formative?
(5) Which developmental “view” does
Kirkpatrick use? (discrepancy,
democratic, analytical, diagnostic)
“IF YOU THINK TRAINING IS
EXPENSIVE, TRY IGNORANCE.”
and, remember, the definition of
ignorance is
repeating the same behavior, over and
over, and expecting different results!