Evaluation: WVDE Innovation Zone and Drop-Out Grant Andu Meharie Office of Research

Evaluation

• • • It involves looking at your program and asking – Is what we’re doing working?

– – How do we know it’s working?

Under what conditions does it work best?

It is part of what you put in place beforehand to help you run your programs. – It should be about looking ahead not backwards.

It is part of the continuum of planning (rolling evaluation).

Evaluation

It is a systematic process of asking questions, collecting information, and using the answers to • • Measure progress and identify areas of improvement.

Set realistic goals and fine tune strategic program decisions.

• • • Identify training and technical assistance needs.

Be accountable and be credible to your constituents.

Motivate by providing documentation of your achievements.

• • • Guide budget and resource allocation.

Generate support for programs.

Be an efficient organization that works toward its mission.

Evaluation

• If (process) or then (outcome), which to evaluate?

Process evaluation – How are resources allocated and used for activities?

– How is the action plan implemented (fidelity), who participated, and what did they think about the activity? – – What obstacles were encountered?

How are preliminary evaluation findings used to improve implementation of activities?

Outcome evaluation – To what extent has the program achieved its desired results?

• • Is a control group possible?

To effect change (Theory of change logic model)

•

Evaluation

Are objectives S.M.A.R.T. with respect to the overarching goal/s of your project. – Specific? • What exactly do you want to accomplish (desired outcome)?

– Measurable?

• How will you assess your progress and accomplishments?

– Attainable?

• What types of resources do you need to attain your objective?

– Relevant?

• Does it align with other objectives and overarching goal?

–

Time-bound?

• When do you expect to see some outcomes?

Identifying Outcomes of your Objectives

• Take 5 minutes at your table: – Choose one objective from your grant application. Make sure to choose an Objective you think is S.M.A.R.T.

– With your team, answer the question: “What are one or more of the potential outcomes that would show your program has been successful in accomplishing this objective?”

Activity

Take 5 minutes to Identify one or more outcomes for an objective listed in your grant application.

Evaluation

• Is your outcome measurable? If your outcomes are measurable, you should be able to establish concrete criteria for gauging progress toward the attainment of each.

Activity

Take 5 minutes to Identify one or more measures/data sources for the outcome(s) you identified previously. Consider: 1. How will you know when the outcome is accomplished? 2. What indicators will you look for to measure progress and success? 3. Are data to measure your progress and success readily available? Or do you need to develop new measures, e.g., surveys, focus groups, etc…?

Evaluation

• Is your outcome time-bound? Outcomes should have starting points, ending points, and fixed durations.

– – When will you achieve this outcome?

When will you undertake activities to achieve our outcome? – When can you expect to see some short-term outcomes?

Take 5 minutes to Identify the Time Frame for each of the outcome(s) you identified previously. Think about time lines for the following: 1. Instrument development (if applicable) 2. Data Collection 3. Data Analysis 4. Reporting

Activity

1. Choose another of the objectives listed on your grant application. 2. Using the worksheet provided, identify an outcome, data sources; assign responsible parties for designing new measures (if necessary), data collection, data analysis, and reporting; and define a timeline.

Evaluation

Objective Outcome Measure/ Data Source Responsible Person (s)

Instrument development

Evaluation Timeline

Data collection Data analysis Reporting

Evaluation

• Put evaluation on your upcoming meeting agendas. Always take some time to talk about where you are in your project, what data you have collected, what it is telling you and what you still need to know.

Thank You

Andu Meharie Coordinator – Research and Evaluation WVDE Office of Research http://wvde.state.wv.us/research ph: 304.558.2546

fax: 304.558.1613

[email protected]