Evaluating Physical Activity Intervention Programs

Thomas Schmid, PhD Physical Activity and Health Branch CDC Atlanta, Georgia, USA

   Evaluation Definitions and Principles CDC Physical Activity Evaluation Handbook – 6 Steps Evaluation issues

The systematic examination and assessment of features of a program and its effects. Evaluation is designed to produce information that can be used by those who have an interest in the improvement or effectiveness of an initiative or program.



To assess the effects and value of a program

◦ ◦ ◦ ◦ ◦ for accountability for progress to identify best practice as a planning tool for research

    the program target groups and needs realistic achievable objectives is the program ready for evaluation?

 Identify needs  State goals and measurable objectives  Assess progress towards objectives

Efficacy Intervention tested under ideal conditions with optimal study design Effectivenes Field based intervention testing whether the intervention works as intended for a defined population in the real world Efficiency Results in comparison to the efforts required to obtain them – often cost or cost effectiveness

Formative

Develop program & resources

IMPACT

Measures short term objectives

PROCESS

Measures activities, program quality, reach

OUTCOME

Measures long term effects (goals)

     What is likely to work Best practices in health promotion Idea or concept development Developing and pilot testing materials Focus groups

   Focus groups and interviews with partners about their needs Reviewing best practices from other programs such as Agita Sao Paulo Pilot testing in one company

Process evaluation

◦ ◦ Program implemented as intended ?

Quality of program components ◦ ◦ Attendance rates ◦ Audience awareness of message Appropriate message learned ◦ Participant satisfaction

 Companies enrolled  Workshops held  Promoters trained  Companies participating in community events

Impact & Outcome Evaluation

◦ ◦ ◦ ◦ Short and long term effects of program Hierarchy of effects : what level of effects indicates program success Assessing the impact upon those who attend or participate Community effects

 Physical activity level of employees  PA level in community  Disease rates or costs

Program Evaluation Framework Physical Activity Evaluation Handbook

Program Evaluation Framework Physical Activity Evaluation Handbook

1. Engage Stakeholders

implementers, partners, participants, possible opponents and decision makers.

2. Clearly Describe the Program

The problem being addressed, descriptions of the planning, implementation and maintenance phases of the program, and finally, defining and communicating how all phases work together (eg. the "logic model").

Program Evaluation Framework Physical Activity Evaluation Handbook

3. Clearly Focus

3. Clearly Focus ensuring linkage to the problem being addressed. Includes clarifying process and/or outcome evaluations of interest 4. Develop and Implement Data Collection Design of data collection efforts, selection of appropriate control groups, designation of appropriate baseline values and decisions about how much time should be built in to adequately measure "change”

Program Evaluation Framework Physical Activity Evaluation Handbook

5. Develop Conclusions

5. Develop Conclusions

Data analyses and interpretation.

6. Communicate

Promotes the importance of tangible products from the evaluation, dissemination of those products, and follow-up (as appropriate).

www.iuhpe.org

20

    All physical activity interventions should be evaluated. Evaluation begins with program planning There are many good programs, but very few are well evaluated Involve stakeholders Community – University partnerships can be very useful

   Process as well as outcome evaluation Quantitative and qualitative   situation Evaluation is often undervalued compared with “hard science” to evaluation. 10% is recommended

    Evaluation results should be used to adjust programs and inform policy Share results with stakeholders Academic and practical training are needed The CDC Physical Activity Evaluation Handbook provides a good framework

 Gracias/Thank you