Transcript Brief Critique of IES Student mentoring evaluation
+
Jean Rhodes National Mentoring Partnership Professor Minnesota Mentoring Partnership October 24, 2010
For Which Youth is Mentoring Most Effective?
+
How effective is youth mentoring for all youth?
For which youth is it most effective?
Under what conditions is it most effective?
What are the implications for policy, practice, and research?
+
Recent Meta-Analysis of Youth Mentoring
Encompassed 73 independent evaluations (1999-2010). The overall effect size was .21*, collapsing across studies and outcomes The average follow-up effect size across the studies was .17.
DuBois, Portillo, Rhodes, Silverthorn, & Valentine (in press). Psychological
Science in the Public Interest.
+
Comparison of mean post-tx effects with school and community interventions with children/teens
Moderator
Attitudinal/Motivational
Mentor Meta-analysis Other Meta-analyses
0.19
0.23- 0.25
Social/Relational l Psychological/Emotiona Conduct Problem Academic/School Attitudes Grades Achievment tests Physical Health 0.17
0.15
0.21
0.21
0.19
0.24
0.18
0.06
0.15-0.26
0.10-.0.24
.02-0.41
0.11-0.27
0.14
0.22
0.11-0.24
0.08-0.41
+
Effect sizes
0.5
0.4
0.3
0.2
0.1
0 -0.1
Medium Effect Small Effect 0 1 2 3 4 5 6 7
Number of Practices
8 9 10 11 Empirically Based Practices Theory-Based Practices
+
Study level variables (moderators) associated with different effects
Youth, Mentor, Program Characteristics Effect Size
Problem Behavior Involvement Youth Gender Individual/Environmental Risk Evidence-based Mentor Training Yes: .29
No: .20
>50% Male: .25
<50% Male: .18
Low/High: .33
High/Low: .31
Below median: .19
Above median: .24
Mentor role function: Advocacy Yes: .26
No: .20
Matching based on shared interests Yes: .44
!!!
No: .21
+
How effective is youth mentoring for all youth?
For which youth is it most effective?
Under what conditions is it most effective?
What are the implications for policy, practice, and research?
+
Stronger effects when…
Youth with With moderate personal/environmental risk Who are male satisfactory, but not strong baseline relationships.
+
Effects of Mentoring on Youth with Different Relational Profiles
BASELINE Relationship s
Overall Academics Prosocial Effort Self-Esteem
Poor Relationships
.00
Satisfactory but not Strong Strong Relationships
.21*** .05
.04
.05
-.04
.19* .18* .07
.04
.00
-.01
(Schwartz, Rhodes, & Chan (2010). Developmental Psychology
+
Stronger effects when…
Mentors who Fit of background/ training with program goals Play an active, advocacy role Are sensitive to socioeconomic & cultural influences Have higher self-efficacy
Hold positive attitudes toward youth
+
Measuring mentors ’ attitudes Grossman et al., 2007 The scale asked mentors to rate how many “ kids in your community ” could be characterized by indicators of youth development: work hard at school respect adults are trouble-makers are fun to be around expect things to be handed to them try to do their best are interested in learning
+
Mentor attitudes and youth outcomes Karcher, Rhodes, Herrera, & Davidson (2010). Applied
Developmental Science
Vulnerable mentees who were paired with high school mentors with positive attitudes negative mentors about youth were more emotionally engaged with mentor than those paired with more Those who were paired with high school mentors with negative attitudes about youth were less emotionally engaged with and showed some negative outcomes.
Stronger effects when…
Relationships characterized by consistency closeness structure
appropriate meeting times duration
+
Does meeting time matter?…
Mentoring by Meeting Time
Afterschool/Lunch Academic Achievement Math Science Social Studies Reading Language Unexcused Absences Grades Scholastic Efficacy
*Note ES= Effect Size
After School During School .19* -.09
.18
.14
-.42** -.19
.11
.27* .21* -.35
-.20
.12
-.16
-.02
-.05
-.08
-.15
.13
Interaction Effect -.33* -.60** -.36* -.29
-.32
-.33* .21
-.51
.03
The role of duration
Grossman & Rhodes ( 2002). American Journal of Community Psychology
+
Re-matching?
+
Test of Time 2: Results
Only youth in matches lasting 24 wks or more benefited academically All mentored youth were less likely to skip school, regardless of match length After controlling for selection bias: Positive academic impacts observed only for youth with intact matches No academic impact for youth with early terminations
Negative academic impacts for rematched youth
Grossman, Chan, Schwartz, & Rhodes (in press). American Journal of Community
Psychology.
+
Stronger effects when…
Programs characterized by careful recruitment training monitoring Multi-modal
matching on interest
+
How effective is youth mentoring?
When are programs most beneficial?
How does mentoring promote positive youth development?
What are the implications for policy, practice, and research?
+
How effective is youth mentoring for all youth?
For which youth is it most effective?
Under what conditions is it most effective?
What are the implications for policy, practice, and research?
Pathways of mentor influence
Mentor
Mutuality Trust Empathy
Relationship mediator Parental/peer relationships Social-emotional development Identity development Positive Outcomes e.g
., reduced health risk, better psych.
outcomes Cognitive development Interpersonal history, social competencies, relationship duration, developmental stage, family and community context
Pathways of mentor influence
Scholastic Competence .25
.26
.08
.29
Grades Mentoring .22
-.28
Quality of Parental relationship .25
.26
Self-worth .19
.18
.09
School value .11
Skipping School Child Development, (2002), 1662-1671
Pathways of mentor influence
Mentoring .23
Quality of Parental relationship .18
.10
-.46
Self-worth -.04
.14
-.08
Substance Use Quality of Peer relationships Rhodes, Reddy, & Grossman (2005) Applied Development Science
Pathways of mentor influence
Academic Attitudes .25
.78
Quality of Mentoring .27
.13
Quality of Teacher relationship .53
.32
-.28
Quality of Parent Relationship Self-worth .27
.09
Grades .18
School Behavior Chan,Rhodes, Schwartz, & Lowe (2011)
+
How effective is youth mentoring for all youth?
For which youth is it most effective?
Under what conditions is it most effective?
What are the implications for policy, practice, and research?
+
Recommendations
Promote evidence-based practice include rigorous evaluation measured replication and dissemination Reward sustainability and quality over growth
+
Research to practice
Research
Implementation • Youth Serving Organizations Distribution • MENTOR/NMP Product Development.
+
Action steps
Develop and improve training and support around relationships Improve mentor retention Employee retention, qualitative studies Export to other youth-serving settings
+
Give Psychology Away
+
Credits
Big Brothers Big Sisters of America WT Grant Foundation Edna McConnell Clark Foundatin NICHD