Transcript Instructions

Community Partnership Evaluation
for Injury Prevention
Susan J. Snelling, PhD
Health Promotion Field Support Specialist, Evaluation
Instructions
• Audio: You can listen and speak
through your computer or
phone – this is an option for
you to select in the “Audio
Mode”
2
Instructions
• Question: When it comes to
the Q & A session you can let
me know you have a question
by clicking the “Raise Your
Hand” button
3
Instructions
• Question: If you have a
question you would like to
type, you can do so in the
textbox provided
4
WEBINAR OUTLINE
• Partnerships
• What is a partnership?
• Why do we work in partnerships? What are the challenges?
• How do partnerships work and what makes a successful
partnership?
• Evaluating Community Partnerships
• What is evaluation?
• Options for evaluating partnerships: Strategies, evaluation
options, measures, methods, and tools
• Resources
5
WHAT IS A PARTNERSHIP?
6
Different Types of Inter-Organizational Work
• Networking: The exchange of information for mutual benefit.
• Coordination: Exchanging information and altering activities
for a common purpose. Match and coordinate needs and
activities. Limit duplication of services.
• Cooperation: As above plus sharing resources.
• Collaboration: As above plus, collaboration includes
enhancing the capacity of the other partners for mutual
benefit and a common purpose. Longer-term and more
deliberate efforts of organizations and groups to undertake
new, joint activities. Building interdependent systems to
address issues and opportunities.
Adapted from Labonte (2003) and Integrated Health Promotion (2003), Victoria, Australia.
7
What is a Partnership?
Community Partnership involves building collaborative
relationships between different groups in the community to
achieve a common goal of promoting health. Within the
partnership, decision-making is shared across partners. The
mutual action of the partnership has the potential to achieve
more than any one partner working independently.
- (Sears 2009 for Heart Health Resource Centres)
8
WHY DO WE WORK IN PARTNERSHIPS IN
INJURY PREVENTION?
9
Why do we Work in Partnerships?
•
•
•
•
•
•
•
Resource limitations; share workload
Reduce duplication
Increase influence and ‘reach’
Include a variety of perspectives
Learn from others
Develop shared messaging; reduce public confusion
Ontario Public Health Standards are built on a principle of
partnership and collaboration
• Social change goes beyond public health: see OPHS logic
models!
10
WHAT CHALLENGES DO YOU EXPERIENCE IN
INJURY PREVENTION PARTNERSHIPS?
11
What Challenges do we Experience in Partnerships?
• Processes: How we work together
•
•
•
•
Partnership building and maintenance takes time
Difficulty coming to consensus may impede current and future efforts
Power imbalances; contribution inequities
Changes in membership
• Outcomes: What we achieve together
• “Opportunity costs” of engaging in partnerships
• Focus on information sharing or endless planning rather than achieving
results
• Focus on long-term goals can lead to loss of focus and commitment in
the short term
Adapted from Partnerships: The Benefits. (2007) Institute of Public Health in Ireland.
12
HOW DO PARTNERSHIPS WORK AND WHAT
MAKES A SUCCESSFUL PARTNERSHIP?
13
Three Levels at Which Partnerships Operate
• Level 1: Partnership infrastructure, function, and processes
• Level 2: Partnership programs and interventions
• Level 3: Health and systems change outcomes
14
Five Conditions of Collective Success
• Common agenda
• Shared measurement systems
• Mutually reinforcing activities
• Continuous communications
• Backbone support organizations
- John Kania and Mark Kramer, “Collective Impact” (2011)
15
WHAT IS EVALUATION?
16
What is Evaluation?
The application of systematic methods to address
questions about program operations and results.
- Wholey, 2010, Handbook of Practical Program
Evaluation
17
10 Steps in Evaluation Process
Step 1: Clarify your program
Step 2: Engage stakeholders
Step 3: Assess resources for evaluation
Step 4: Select the type of evaluation
Step 5: Determine methods of measurement and procedures
Step 6: Develop workplan, budget and timeline
Step 7: Collect data
Step 8: Process the data and analyze the results
Step 9: Interpret and disseminate results
Step 10: Take action
18
Process Evaluation
• How do we do our work?
• Measures activity and implementation
• Sometimes known as tracking or monitoring
19
Outcome Evaluation
• Are we achieving the objectives we set out?
• Estimates success of program (or partnership)
at meeting goals and objectives
• Measures effects of activities at producing
change
• It can be difficult to measure long-term
outcomes
• Ii can be difficult (or impossible) to isolate the
contribution of your work to a societal result
20
OPTIONS FOR EVALUATING PARTNERSHIPS
21
Strategies for Implementing Effective Evaluation
in Partnerships
• Build in evaluation concepts from the start of the
partnership
• Establish principles: Here are a few suggestions:
•
•
•
•
No shame, no blame – do evaluation to learn and improve
Don’t evaluate something that can’t change
Commit to taking action based on the results
‘We’re all in this together’ (the problems and the solutions are not
the leader’s responsibility)
• Establish clear evaluation questions
• Have a plan for using the results of an evaluation
22
Options for Evaluating Partnerships
• Start with Evaluability Assessment
see Butterfoss: Are You Ready to Evaluate Your Coalition?
http://coalitionswork.com/documents/are_you_ready_to_evaluate_yo
ur_coalition.pdf
• Since there are three levels at which partnerships operate,
there are also three levels on which evaluation can focus:
• Level 1: Partnership infrastructure, function, and processes
• Level 2: Partnership programs and interventions
• Level 3: Health and systems change outcomes
23
Measures for Evaluating Levels of Partnerships
Level 1: Structures and Functions
• Number and type of partners
• Skills of partners
• Diversity
• Commitment, sense of ownership
• Partnership processes such as
meetings, workload
• Leadership style and
effectiveness
• Member satisfaction
• Perceived effectiveness among
members and community
Level 2: Program Implementation
• Implementation of plan
• Programs and services offered
• Community actions taken
• Reach; Media coverage
• Knowledge and behaviour change
• Improved or new policies or
practices (Level 2 or 3)
Level 3: Outcomes and Impacts
• Health status
• Community changes
• Community capacity
• Partnership sustainability
24
Methods and Tools for Evaluating Partnerships
Methods
• Active data collection: Surveys, interviews, focus groups, self-report
• Unobtrusive data collection: Observation, monitoring, statistics
• Descriptions of partnerships can be done through network mapping
Tools
• Coalition Member Survey; Coalition Effectiveness Inventory (Butterfoss)
• Vic Health Partnerships Analysis Tool
• Network Health Scorecard
• Mental Health Mapping Tool
• Partnership Effectiveness Tool
25
RESOURCES
26
Health Promotion Capacity Building Services
• Free to those working on Ontario-focused projects.
• Scope varies, depending on need:
•
•
•
•
•
training sessions;
brief, one-time advice;
consultations;
review your work or product;
links to other sources of information and resources.
• Consultation request form
http://www.thcu.ca/consultation/request_form.cfm
27
QUESTIONS?
28
Thanks to Ontario Injury Prevention
Resource Centre for supporting us on
this webinar!
29
Public Health Ontario wishes to
acknowledge and thank THCU staff and
many partners who contributed to an
earlier version of this document. THCU
(originally known as The Health
Communication Unit, started in
1993 at the University of Toronto) moved
to Public Health Ontario’s Health
Promotion, Chronic Disease and Injury
Prevention Department in 2011.