Transcript Slide 1

Evaluating
Work with Men
and Boys
Gary Barker
Instituto Promundo
Rio de Janeiro, Brazil
Why evaluate?
1.) Improve our programmes: the process
 How did the intervention go?
 Did we reach men and boys?
 Did they find it interesting and useful?
2.) Assess impact: the effectiveness
 What impact can we achieve in engaging men
and boys with our specific intervention?
 How long does it take? What is the “dose”
required to achieve that impact?
 How much does it cost?
What are we evaluating?
The What has to do with our specific goals for
engaging men and boys, and our approach
What can work with men and boys accomplish?
What do we propose with the work?
Examples: Knowledge change, attitude change,
behavior change, increase in users of a service,
increase in condom use or condom sales, etc.
PITFALL 1: We too often seek to measure things
that are outside or not contemplated in our
intervention
A Typology of Programmes for
Engaging Boys and Men
• Gender nuetral: No distinction between men &
women; men just another target group
• Gender sensitive: Recognize gender norms but
little attempt to transform them
• Gender norms transformative: Seek to promote
equitable relationships, somehow change gender
norms
How do we evaluate interventions that seek
to change gender norms?
What is engaging men and boys from a
gender norms transformative
perspective?
 Include
discussion of gender
norms and how these
influence behavior;
 Include deliberate public
debate + critical reflection
about these norms; and
 Measure changes in
attitudes toward gender norms
+ relevant behaviors.
Evaluating a Gender Norms Approach
1.) Formative research – mapping of masculinities,
gender norms. Define: What gender norms need to
be transformed? How best to promote change?
What are the “cracks” or existing gender-equitable
attitudes that can be built on?
2.) Intervention formulation – Logical framework,
setting objectives + outcome indicators. What
interventions will transform those norms?
3.) Implementation – Process evaluation: How many
men/boys? How did it work?
4.) Impact evalualtion: Did behavior + norms change?
1.) Formative research
• Qualitative and quantitative
• What are the prevailing norms related to
masculinities/gender in a given setting?
• How do these norms affect relations between
men and women?
• Where are the “cracks”, “voices of resistance” or
opportunities for change?
• Interview: men, women, boys, girls, gatekeepers
2.) Intervention Formulation:
Components to Change Gender Norms
3.) Implementation: Process
evaluation
• Number of sessions,
participants (attendance forms)
• Quality of sessions (interviews
with facilitators, field notes,
simple post-session
questionnaire)
• Number of participants at
community activities
(observation, field notes)
• Condoms distributed
4.) Impact Evaluation: Measuring
Gender Norms: the Gender Equitable
Men Scale – GEM Scale
The domains or major categories of themes:
 Home & child-care
 Sexual relationship
 Health & STI prevention
 Violence
 Homophobia & relations with other men
Examples of Items from the GEM
Scale:
• Men are always ready to have sex.
• There are times when a woman deserves to
be beaten.
• A woman’s most important role is to take
care of her home and cook for her family.
• I would never have a gay friend.
Percent w STI Symptoms
Association between traditional norms
and STI Symptoms*
Baseline:
50
45
40
35
30
25
20
15
10
5
0
Change in 1 year:
45
31
High Equity
Moderate Equity
Low Equity
More equitable young
men reported less STI
symptoms:
23
•In Bangu, 4 times
less
•In Maré, 8 times less
GEM Scale
*p < 0.001 – Mantel-Haenszel chi-square test, logistic regression
Traditional Views Regarding Gender by SelfReported Use of GBV – Young Men in Brazil
(n=225, 15-24, RJ)
50
45
40
35
30
25
20
15
10
5
0
High Equitable
Medium Equitable
Low Equitable
17
10
4
GEMS
*p < 0.001 - Chi-square test
B
A
N
G
U
6 months
Pre-test
N = 258
6 months
Post-test 1
N = 230
6 months
M
A
R
É
M
A
C
A
C
O
S
Pre-test
N = 250
6 months
Post-test 1
N = 217
6 months
Pre-test
N = 250
Post-test 2
N = 217
Post-test 2
N = 172
3 months
Post-test 1 (contr.)
Pre-test (int.)
N = 180
Post-test 1
(delayed int.)
N =89
Change in Selected GEM Scale Items
% Agree
Men need sex more
than women do.
37
I would be outraged if
my wife asked me to
use a condom.
11
Housework is the
mother's
responsibility.
62
52
22
16
17
15
24
29
26
31
23
21
0
44
43
43
20
38
40
baseline (Maré)
6 months (Maré)
1 year (Maré)
baseline (Bangu)
6 months (Bangu)
1 year (Bangu)
60
*p < 0.05 – t- test
80
Behavior Change: Condom Use at
Last Sex
100
90
80
70
60
50
40
30
20
10
0
79
84
77
69
58
Bangu
70
Baseline
6 Months
1 Year
Mare
*p < 0.05 - Chi-square test
Qualitative Results: Program H
Evaluation Study (Brazil)
 Reported changes in styles of interacting with other
men: movement toward more cooperative, less
aggressive interactions
 Ability to discuss sexuality openly (male and female)
 Recognition of women as having sexual rights,
sexual agency
 Worry about own health needs, HIV testing (male and
female)
 Delayed initiation of sexual activity with current
partner
% Agree
Attitudes
A man can hit a
woman if she cheats
on him
36
33
100
28
3
A woman should
tolerate violence to
keep family together
Harassment of Girls
% harassed any girl
hit
okay for a man
A manto
can
hithis
a
woman
if
she
if she refuses sex with
refuses sex
him*
33
9
There
are times
when
women
when
are times
women deserve to be
eserve to be beaten.
beaten
31
20
40
80
60
80
Pretest
Post test
40
20
14
0
Behavior
60
80 100
0
% Agree
Pre Test
Post Test
Pretest; n = 107
Other Outcome Indicators: Interventions
Engaging Men and Boys
• Descreased self-reported use of physical, sexual and
psychological violence in intimate relationship (Safe
Dates Programme, US; Soul City, South Africa)
• Increased contraceptive use (Young Dads Programme,
US; Together for a Happy Family, Jordan; Male
Motivation Campaign, Zimbabwe and Guinea; Involving
Men in Contraceptive Use, Ethiopia)
• Increased communication with spouse or partner
about MCH, RH(Involving Men in Maternity Care, India;
Together for a Happy Family, Jordan; Male Motivation
Campaign, Guinea; Soul City, South Africa)
Other Outcome Indicators: Interventions
Engaging Men and Boys - 2
• More equitable treatment of children
(Together for a Happy Family, Jordan)
• Increased use of SRH services by men
(Integration of Men RH Services in Family
Welfare Centers, Bangladesh)
• Increased condom use (Sexto Sentido,
Nicaragua; Program H, Brazil)
• Decreased rates of STIs (Program H, Brazil)
• Increased social support of spouse (Soul City,
South Africa).
How much evaluation do you need?
• Limited resources: Some formative + process
+ post-only focus groups
• Modest resources for evaluation.: Formative +
process + simple pre- and post-test
(quantitative) with no control group
PITFALL 2: Although qualitative research may be
less costly, analysis is more complicated and
should be treated as such. Collecting pre- and
post-test frequencies is easier although
sometimes more costly.
Higher level of resources:
 Extensive formative
 Multiple sites + control
group (or delayed
intervention group)
 Triangulation with
partners/women
 Qualitative and
Quantitative throughout
Conclusions
• More + longer-term evaluation often difficult, but
necessary
• Be realistic in selecting indicators; use methods
that are appropriate for the intervention
• Gather cost data: Need to show cost of achieving
change in gender norms
• Move from the individual to the structural level –
from gender norms to gender structures
• Use evaluation for advocacy but recognize that we
can’t prove it all .....