SOWK 6003 Social Work Research

Week 7 Designs for Evaluating Programmes and Practice – Experimental designs By Dr. Paul Wong

Purposes of Research

1. Exploration 2. Description 3. Explanation 4. Evaluation 5. Multiple purposes

Program evaluation refers to the purpose of research rather than to any specific research methods.

Purposes of program evaluation 1) to assess the ultimate

success

of programs; 2) to assess problems in

how programs are being implemented

; or 3) to obtain information needed in program

planning and development

1990s in the US: program evaluation as a prerequisite for approving grant applications

Two major types of evaluations

Summative – are concerned with assessing the

success

of programs Formative – are concerned on obtaining information that is

helpful in planning the program and in improving its implementation and performance

Politics

The evaluation, itself, can be dealt with quite straightforwardly. However, the POLITICS of it can be very difficult to handle.

Some common obstacles

• In-house vs. External evaluators • Utilization of the findings: – The implications may not always be presented in a way that nonresearchers can understand; – Evaluation results sometimes contradict deeply held beliefs. • Logistical and Administrative problems: – Learning as much as possible about the stakeholders; – Maintaining ongoing mutual feedback between them and the evaluator; – Tailoring the evaluation and its reportage to their needs and preferences as much as possible without sacrificing scientific objectivity

Types of Program Evaluation 1. Evaluating outcome and efficiency : – should strive to enhance

causal inference

by using the most internally valid experimental or quasi-experimental design possible – The assessment of efficiency asks whether program outcomes are being achieved at a reasonable cost and applies the principles of cost accounting

to calculate the ratio of program benefits to costs

2) Process Evaluation – aims to investigate the “means” , not the “ends”, and focus on identifying strengths and weaknesses in program processes and recommending needed improvements e.g., (pp.323) –

What types of individuals are not being reached by the service?

–

Are clients satisfied with services? Why and why not?

– Tends to rely heavily on qualitative methods

3) Needs assessment

 A diagnostic evaluation  Common techniques: − Key informants − Community forum − Rate under treatment approach − Social indicator − Community survey − Focus groups

• Randomized Controlled Trial – What is it?

In class activities

1) Break into small groups and design program evaluation studies regarding this class. How would they assess the need for it, its implementation, and its outcome in attaining its goals? 2) 1) Articles Critique: Cheung Chau Suicide Prevention Programme A local evaluation study using a RCT methodology