Transcript Defining and Measuring Gender at the Household Level

LESSONS LEARNED 1:
IMPORTANCE OF HAVING
CLEAR CONCEPTUAL
FRAMEWORK TO GUIDE
EVALUATION
SASA! Intervention Uganda
 Clear conceptual framework for intervention about
process of change trying to achieve in community
 Stages of intervention focus and associated activities
clearly articulated and translated into programming
 Used to inform M&E design
Phase 1:
Start
• Power
Within
• Precontempl
ation
Phase 2: •Power Over
Awarene •Contemplati
ss
on
Phase
3:
Support
• Power
With
• Preparatio
n for
Action
SASA! Phases / Process of Change
Phase 4:
Action
• Power To
• Action /
Maintenan
ce
SASA! Logic Model
Context
Levels of
SASA! Activities
reaching each
Circle of Influence
Sociodemographic
Factors
Sex
Age
Societal
Nat’l policy
makers, media
Income
Community
Education
Employment
Police, LCs,
HCPs, NGOs,
Ssengas, RLs,
Initial Outcomes
Knowledge
Skills
•Recognizing VAW a problem
•Types of VAW
•Consequences of VAW
•VAW/HIV linkage
•Response to women
experiencing violence
•Hold men accountable
•Promote balanced
power
•Support to activists
Awareness
•Root cause as imbalance of
power between women and
men
•Change can happen
Religion
Place of
residence
Socio-cultural
Factors
Relationship
Relatives,
elders,
neighbors
Critical Thinking and
Dialogue
•Public debate and discussion
•Personal reflection
Family
Characteristics
Individual
Participation
Social Support
Women, men,
youth, CAs
•Activists at grassroots, in
leadership, in institutions
Alcohol Use
Phase 1:
Start
• Power
Within
• Precontemplation
Longer term
outcomes
Intermediate Outcomes
Individual &
Collective
Capacity
•Supportive
environment
•Enhanced ability
to prevent and
respond to VAW
Action
•Intention to act
•Personal change:
balancing power
• Public change:
sanctions ag VAW
•Acceptability of
expanded gender roles
Reduced
social
acceptance
of VAW
Decrease in
experience /
perpetration
of VAW
Behaviors
•Balancing power
•Communication
with partner
•Decreased risk
behaviors
•Community
activism
Acceptance and
Influence
•Attitudes toward
power, gender, human
rights
•Silence broken
Connection
•Activists/leaders/profe
ssionals increasingly
connected and active
Phase 2: • Power
AwareOver
ness
• Contemplation
Impact
Improved
response to
women
experiencin
g violence
Sustained
Action
•Changed policies
•Organized
groups
•Changed practice
in relationships,
community,
institutions
Phase 3:
Support
• Power
With
• Preparation
for Action
SASA! Phases / Process of Change
Decrease in
HIV/RH risk
behaviors
Phase 4:
Action
• Power
To
• Action /
Maintenanc
e
Key elements in conceptual
framework
 What are the range of potential exposures
people may have to intervention?
 What is expected process of change?
 What are likely pathways of change?


Among individuals receiving the intervention
In the broader community
 What is a realistic timeframe for this change
to occur?
 How do we measure this?
LESSONS LEARNED 2:
POWER OF MULTIPLE FORMS
OF EVIDENCE (IMAGE STUDY)
Did IMAGE affect MFIs financial performance?
 50% reduction in centre “vulnerability” = attendance, arrears, savings
 Impact persisted 2 years after intervention completed
Vulnerable Centres
40
35
30
20
15
10
5
0
Au
g0
No 0
vFe 0 0
bM 01
ay
-0
Au 1
g0
No 1
vFe 0 1
bM 02
ay
-0
Au 2
g0
No 2
vFe 0 2
bM 03
ay
-0
Au 3
g0
No 3
vFe 0 3
bM 04
ay
-0
Au 4
g0
No 4
v0
Fe 4
b05
%
25
Similar impact on drop out rates.
IMAGE
Average
Past year experience of intimate partner violence
reduced by 55% (aRR 0.45 95% CI 0.23-0.91)
0.1
Past year experience of
physical/sexual partner
violence
0.45 (0.23-0.91)
- pushed you
- hit you
- forced sex
- scared to say no to sex
Pronyk et al. The Lancet Dec. 2006
Adjusted risk ratio
1
10
How was violence reduced?
Changes in women’s
relationships
“Now that we have money
we are able to say how we
feel without fearing that your
husband will stop supporting
you.”
“You can buy him cigarettes
from your profit. Because of
SEF’s money we are
experiencing fewer problems
in our households”
Women supporting women
“We are able to overcome
abuse when we are in SEF
because we get support from
the women in the groups.
When you engage yourself
with other women and listen
to their problems that will
help you to cope”
LESSON LEARNED 3:
COSTING IMPORTANT
PART OF INTERVENTION
EVALUATION
What does costing do?
 Document all inputs into intervention
 Capital & recurrent expenditure
 Ingredients approach to costing puts a
financial value to all inputs:


Monetary
Non-monetary
 Can be used to:



Look at break down of costs by activity
Estimate unit costs
Estimate cost-effectiveness
Scaling up IMAGE in South Africa
Pilot Study
Additional cost
= US $43/client
2001-2004
430 households
2005-2007
4500 households (30,000)
Scale-up
Additional cost
= US $13/client
2008-2010
15 000 households (80,000)
CHALLENGES 1. REPORTING
BIAS
Physical and/or sexual partner violence
against women at baseline – SASA! study
in past 12 months
INTERVENTION
Violent Behaviour
CONTROL
Women
Men
Women
Men
Slapped her or thrown something at her
that could hurt her
23.4%
20.5%
17.6%
23.1%
Pushed her or shoved her or pulled her
hair
11.5%
7.1%
9.5%
7.2%
Hit her with his fist or with something
else that could hurt her
11.2%
3.8%
8.4%
4.5%
Kicked her, dragged her or beat her up
10.2%
1.6%
8.4%
1.5%
Choked or burnt her on purpose
3.9%
0.0%
0.7%
0.0%
Threatened or actually used a gun, knife
or other weapon
2.0%
0.0%
2.6%
0.0%
Threatened or intimidated her into
having sexual intercourse even
when she did not want to
9.5%
0.6%
7.7%
0.9%
Physically forced her to have sexual
when she did not want to
10.5%
2.6%
9.5%
1.2%
Physical and/or sexual violence
29.1%
23.1%
24.5%
25.4%
The potential for reporting bias
about gender norms & violence
 Women tend to under-report experiencing
violence
 May be more willing to disclose following exposure
to intervention
 Could find higher disclosure in intervention
arms
 Men may not disclose perpetrating less socially
condoned acts of violence
 May be even less willing to report disclosure
following contact with intervention
 Could find higher disclosure in intervention
arms
Implications for evaluation
 Importance of choosing quantitative
outcome measures carefully
 Rely on women’s reports on
experiencing of violence, rather than
men’s reports of perpetration

If find reduction, effect less likely to be due
to reporting bias
 Use qualitative research to triangulate
CHALLENGE 2: CAPTURING
SOCIAL MOBILISATION
DIFFICULT AND
UNPREDICTABLE
IMAGE: community mobilisation led to a
diverse range of activities



Many focused on violence and
HIV
Also focused on other issues of
concern to women
Difficult to fully document all of
activities
Community mobilization:





40 village workshops
16 meetings with local leaders
5 public marches
2 partnerships with local
institutions
2 new village committees target
Crime and Rape
Timelines as part of M&E activities
in Uganda
 Periodically using timeline with key informants
 Discuss key actions and key events in the
community
 Aim to map out scale and geographical diversity of
activities over time
CHALLENGE 3: EVALUATION
DIFFICULT
Getting reliable evidence on
impact is not easy
 Getting strong quantitative data on impact difficult:
 RCTs gold standard study design, but expensive and technically
difficult
 Require partnership between intervention agencies & research
organisations
 May be rare that can show intervention impact on HIV or VAW
directly
 Seeking to evaluate complex interventions on complex issue
but:




Trial design generally focuses on limited number of impacts –
need to identify primary outcomes
Forms of social change may have multiple benefits
Limited evidence about what may be reasonable timeframes for
different forms of change
May be lack of recognition of importance of change in pathway
variables
The challenge of building a strong
evidence base

Data from control communities important to help attribute
impact





Difficult to fund evaluation research:




Challenge of getting sufficient number of study and control
communities
Difficult to identify and maintain control populations
May not be feasible if intervention national level
Difficult to identify and maintain control populations
IMAGE had 9 + donors
SASA! funding from 3 donors for baseline alone
GBV not primary focus of most research donors
Leads to under-powered studies

Less focus on statistical significance
CONCLUSIONS
The value of good M&E evidence
 Growing body of experience, but extremely limited evidence
about intervention impact on VAW & IPV
 Importance of M&E to provide insights about:
 Process & scale of project implementation
 Impact on recipients and broader community
 Resources / costs required
 Multiple forms of evidence important:
 Quantitative - scale of activity, magnitude of impacts
 Qualitative – acceptability, what happened, what means,
quality, role of key players, unexpected events
 Economic - forms of input, economic value, potential costs if
replicate
 Lessons from M&E important to inform
 ongoing programming
 replication locally, nationally & internationally
The challenge of building a strong
evidence base
 Fundamental challenge of how to move from documenting levels of
activity to level of change
 Getting strong data on impact difficult:
 RCTs expensive and technically difficult
 Require partnership between intervention agencies & research
organisations
 May be rare that can show intervention impact on HIV or VAW
directly
 Seeking to evaluate complex interventions on complex issue:
 Timeframe for change my be longer than evaluation funding
 Importance of capturing broad benefits of social empowerment
interventions
 Mixed methods likely to provide most reliable evidence

Draws on strengths of each approach
 Provides opportunities for triangulation
The way forward?

Document experience and lessons learned
 From broader field of evaluation research – community interventions, service
delivery, media interventions
 From current M&E of different models of GBV intervention
 Experience combining quantitative and qualitative methods
 About realistic timeframes and process of change

Lobby for strategic investment in multi-disciplinary evaluation research:
 Women’s empowerment
 School based programmes
 Community mobilisation
 Masculinities
 Alcohol
 Service provision
 Role of legal and policy reform

Recognise need for methodological work
 Use evaluation research to learn about pathways of change, and broader
lessons for M&E
 How to conceptualise, define and measure key variables
 Integrating participatory evaluation methods with quant evaluation frameworks