Transcript EVALUATION PROGRAMS FOR GRANT PROJECTS Linda Mason, Ed.D. OSRHE [email protected] Evaluation Accountability measure  Insurance policy for the grant funder  Shows if you did what you said.

EVALUATION
PROGRAMS FOR
GRANT PROJECTS
Linda Mason, Ed.D.
OSRHE
[email protected]
Evaluation
Accountability measure
 Insurance policy for the grant funder
 Shows if you did what you said you would
do
 Shows if the grant funder’s money created
some good
Not acceptable:
“We are planning to evaluate….”

GRANT AGENCY CHOOSES
External Evaluation* - by someone not
connected with your institution
 Internal Evaluation – you or another
project staff member

Internal Evaluation
Collect your own data
 Simple annual evaluation process:
“What did you get in last summer’s
institute that you have been able to
implement?”
“What might we have done in the institute
that would have helped you even more?”

Internal Evaluation
Prepare your own evaluation format
 Use objectives and timeline
 Keep records of EVERYTHING!
 Photographs
 Testimonies
 Anecdotes
 Write proposal as if you were evaluating it

Internal Evaluation

Summative evaluation – Outcome Evaluation
Program effectiveness
Impact on the problem
Internal Evaluation
Summative evaluation:
What did you accomplish?
Were program objectives accomplished?
Was the target population changed?
Did you meet the timeline?
Was the program cost effective?
What new knowledge was generated?
Internal Evaluation
Summative evaluation:
Quantitative methods
Formal approach
Professional research procedures
Control and comparison groups
Compare to the big picture
Goal
Goal 1: xxx
xxxxxxxxxx
Responsible Benchmark
Party
Project
Director
Results
5 teachers 6 teachers
trained by presented
May 31, 2005
with QT
curriculum
workshop
by QT, Inc.
on May 1214, 2004
Internal Evaluation
 Formative
evaluation –
Use evaluation material to retool
Internal Evaluation
 Formative
evaluation – Process
evaluation
 Leads to changes mid-stream
 Generally use qualitative methods
 First year or first part
 Throughout the life of program
Internal Evaluation
 Formative
evaluation – Ask questions
during the project:
How are we doing with the timeline? Do
we need to change it?
What does the recruitment tell us about
group size?
What did the first of 3 workshops lead us
to determine about the next one? Any
changes needed in format or length?
Internal Evaluation
 Formative
evaluation – Don’t ask
questions about items you cannot or
will not change.
Internal Evaluation - Tips
Send thorough reports to grant agency
 Send more than requested
 Send in format of summary and full report

External Evaluator
Consider hiring one even if not required
 External evaluator adds credibility
 Consider being an external evaluator to
learn more about grant proposal
production
 Work with the evaluator at the proposal
design to be sure evaluation fits.
 Collect your own data

Evaluation - Types
You do not have to conduct fancy “twosample t-tests” or “analysis of variance”
 Pre- and post-tests
 Attitude surveys to participants
 Document interviews and focus groups
 Make charts of results

Evaluation
Ways to share results Research Fair
 Poster
 Videoconference
 Press release
 Follow-up session after project
 Project Results Meeting
 Prepare a document

Evaluation
Ways to share results Email news release
 Poster or brochure campaign
 Prepare a video or CD
 Go on the talk show circuit of TV or radio
 Present at school assemblies
 Present at faculty meetings
 Share with similar or larger projects

Evaluation
Traditional Ways to Share Results
Journal Article
 Conference presentation

Evaluation in Grant Proposal
Depends on Grant Agency
 Thoughtful examples of the plan
 Reasonable

Evaluation in Grant Proposal
Linkages between activities and outcomes
should be clear and explicit
Example: 80% of the 21 to 24-year olds
who participate in a comprehensive workstudy program will pass their GED exams.
 Show other support from larger or
partnering programs.

Evaluation - Tips
Don’t tack on the evaluation at the end—
Frontload it, build it in at the beginning.
 Explain what measures you are going to
use.
 Use the evaluation section to go into
detail.
 Show sustainability of the project as part
of the evaluation.

Evaluation - Tips
Negative findings are as useful as positive
findings
 Were there unexpected world, national or
local events that seemed to affect the
project?
 Could the project be replicated?

Evaluation Summary
You WILL evaluate
 Grant agency specifies internal or external
 External strengthens credibility
 Provide more data than requested
 Use a variety of sharing techniques
 Formative – changes during project
 Summative - outcomes

Evaluation Summary
Frontload evaluation plan
 Cost effective
 Replicable
 Renewed
 Sustained
