Transcript Program Evaluation Overview - Winston

E. D. Bell

Kapp & Anderson (2010) argue that the
program evaluation process has eight
steps that are fluid, dynamic, and
interdependent, i.e., they are not linear –
they influence and shape one another
throughout the evaluation process.
1.
2.
3.
4.
Identify the
evaluation question
Conceptualize the
evaluation question
Operationalize the
evaluation question
Pick and define the
method
5.
6.
7.
8.
Select the sample
Collect the data
Analyze and
manage the data
Utilize the
information to
improve program
performance
Kapp & Anderson (2010) make this point
The steps of the evaluation process need
to be framed with an appreciation of two
fundamental notions:
(1). The use of information for service
improvement is primary, and
(2). The context for evaluation is an
agency where evaluation must
compete with many enterprises for
attention. (p. 29)

Given these notions, the agency needs to
implement the following steps
concurrently with the implementation of
the program evaluation:
1. Define the program
2. Support users
3. Implement and report design of the
evaluation (Kapp & Anderson, 2010)
“The process of conducting research and
the evaluation of programs and the
outcomes associated with these activities
have the potential to be helpful or
harmful to individuals, groups,
organizations, and the broader society”
(Kapp & Anderson, 2010, p. 39).
So, if you were a participant in a study,
what would you want to know? If you
were a program director, how would
you want the evaluators to conduct
themselves? If the evaluation of your
program found a number of faults,
how would you want these to be
reported? (Kapp & Anderson, 2010, p.
59)

Guiding Principles for Evaluators
Systematic inquiry
 Competence
 Integrity/Honesty
 Respect for people
 Responsibilities for general public
welfare

“The
process of program evaluation
does not take place in a social and
cultural vacuum devoid of gender,
race, ethnicity, and other
consideration” (Kapp & Anderson,
2010, p. 107).
Kapp & Anderson (2010) describe four attributes:
1. Awareness and acceptance – that generates
respectful treatment and no prejudgment of
abilities, performance, and competency.
2. Self-awareness – that helps you understand
how your culture shapes what you see and
provides the ability to examine, control, and
change assumptions, false beliefs, and
stereotypes
Dynamics of differences – historical and
cultural differences influence how you see
and define reality, e.g., see Emmett Till.
This story that I saw in Jet magazine as a
young person still influences how I see and
understand reality. Click on the link “The
Picture of Emmett Till”.
4. Knowledge of individual cultures – of the
stakeholders who are involved with the
program being evaluated.
3.
5. Adaption of skills – The evaluator must be
able to determine if her or his methods are
culturally appropriate and respectful of the
persons and programs that are being
evaluated.

Review Kapp & Anderson (2010) pp. 111 – 131.
“With most evaluation projects, one of the
first things to address is the way the
program is supposed to operate” ( Kapp
& Anderson, 2010, p. 133).

Develop the logic model of how the
program is supposed to work (Program
Development and Evaluation, University
of Wisconsin – Extension for resource
tools)

Interview Stakeholders
 Board of Directors
 Clients
 Community Partners
 Funders
 Management
 Staff (Kapp & Anderson, 2010).



Use Logic Model Questionnaire (Kapp &
Anderson, 2010, p. 139)
Given the first draft of the logic model
for the agency what areas of the model
require additional information, i.e., what
are the potential evaluation questions.
Select one of the areas as the evaluation
question for your evaluation plan