Transcript Slide 1

BUILDING STRONG PROGRAMS: LESSONS
FROM WORKING IN THE SCHOOLS (WITS)
January 31st, 2014
DISCUSSION TOPICS
•
Introductions
•
Overview of WITS
•
Training, Expectations, & Strategies
•
Evaluation and Assessments
•
Small Group Discussions
WITS VIDEO
BRIDGING THE GAP
WITS believes that when mentors focus their
energy and time on the success of our city’s
students, their volunteer service directly
impacts the lives of young people and helps
support a better public school system.
WITS PORTFOLIO
Programs:
•
Mid-Day Mentoring
•
Workplace Mentoring
•
Early Childhood
•
Classroom Reading Tutors
•
Saturday Tutoring
•
Early Childhood Summer Program
•
WITSummer in the Parks
OF PROGRAMS
STAKEHOLDERS
Volunteers
(community,
corporate groups,
universities,
churches)
Schools
*
Students
POSITIONING
Frequency
and Quality
of Support
VOLUNTEERS FOR SUCCESS
Setting
Appropriate
Expectations
Modeling,
Coaching,
Reflection
STRATEGIES FOR WORKING WITH YOUTH
OVERCOMING OBSTACLES
Model Problem Solving
• Name the problem
• Talk through possible solutions
• Model appropriate problem solving
Encourage the Student
• Praise their process
• They aren’t alone and—neither are you!
The Goal: Students to see themselves as
capable problem solvers and valuable
individuals
BUILDING STUDENT FLUENCY
Model Fluency
• Reading expressively
• Use character voices
• Modulating speed and volume
Encourage the Student
• Help student choose books
• Re-read to build confidence
The Goal: students see reading as an
opportunity to explore their interests—not a
task to complete.
THE FIVE C’S OF POSITIVE YOUTH
DEVELOPMENT (PYD)
PYD: focus on the strengths of youth and the positive qualities
and outcomes we wish youth to develop
Competence
Confidence
Connection
Character
Caring
*
USING EVALUATION TO STRENGTHEN
PROGRAMS
Types:
• Formative
• Summative
Methods:
• experimental
• quasi-experimental
• observational
Considerations:
• What is the purpose of the evaluation?
• Who will use the evaluation results?
• How will they use the evaluation results?
• What do stakeholders need from the evaluation?
THE POWER OF OBSERVATION
(FORMATIVE EVALUATION)
Questions:
•
Are we meeting our program goals?
•
Is there consistency across programs?
•
What immediate improvements can we make?
•
What changes can/should we make in the future?
•
Use observation forms/standardize
WORKPLACE MENTORING LOGIC MODEL
Program Goals
Situation
-need for consistent
and caring mentors
-need for literacy
enrichment
programming in the
afterschool space
-scaffold student
self-efficacy
-scaffold student
attitude
Assumptions
Priorit
-all students are
ies
capable of becoming
better readers
-students are
motivated to
participate in WITS
programs
-volunteers are
competent
tutors/mentors
Inputs
Outputs
-human resources: staff;
corporate + university
volunteer time
investment
-effective 1:1 literacyfocused mentoring
programs
-partners:
teacher/principal time
investment; experts
-WITS Board;
Associates Board
-financial resources
-students and
volunteers form a
positive relationship
over time
-build knowledge base
-other: transportation;
bus chaperones; snacks;
maintain program
materials
-300 students from 13
schools participate
-19 corporate and
university groups
provide 600 volunteers
-curriculum
implemented
-volunteers coached to
provide student with
effective support
Outcomes
Student:
-improved attitude
toward reading
E
V
A
L
U
A
T
I
O
N
-increased selfefficacy
-improved reading
fluency
Volunteer:
- sustained
school/community
partnership
-high volunteer
satisfaction
-high volunteer
retention
identify external factors
WITS EVALUATION GOALS
•
mixed method approach
•
multiple data sources
•
triangulate data
•
balance formative/summative
•
make decisions on multiple findings
BEST PRACTICES
•
align data collection with program goals
•
think before you collect
•
weigh pros and cons of types of questions
•
balance formative and summative
•
pilot a new assessment
•
ask the hard/unpopular questions
•
consult peers
•
know your research domain
EXAMPLE: LEVERAGE EXTERNAL DATA
BENEFITS OF WORKPLACE VOLUNTEERISM
2011 Deloitte Volunteer IMPACT Survey
EXAMPLE: LEVERAGE INTERNAL DATA
STUDENT-REPORTED SATISFACTION
Youth Emotional Engagement Benchmark Survey adapted from: Herrera, C. (2004). School-Based Mentoring. A
Closer Look. Public/Private Ventures, 42.
EXAMPLE: LEVERAGE INTERNAL DATA
LENGTH OF PARTICIPATION IN
WORKPLACE MENTORING PROGRAM
Teacher Ratings of Student
Motivation/Attitude Toward School and
Reading
0 years
minimum of 2 years
100%
80%
60%
40%
20%
0%
Enthusiasm Toward
Reading
Positive Attitude Silent Reading
Positive Attitude Oral Reading
Positive Attitude School in General
COMMON EVALUATION ERRORS
Avoid:
•
•
•
•
failing to leverage the data you already have
collecting more data than you really need/can handle
using the same data points when a project has changed
poor survey/question design
o
o
o
o
o
o
o
wrong question type
vague wording/jargon
incorrect assumptions
leading questions
double-barreling
missing response options
incorrect use of scales
SMALL GROUP
DISCUSSION TOPICS
1. Discuss some challenges you have had when working with youth. What
changes did you make to your program structure, training or curriculum due
to those challenges?
2. How do you gain feedback from your volunteers/staff? Provide an example
of feedback your organization has received and programmatic changes you
have made due to the feedback?
3. Discuss manageable formative evaluation activities that you and your
organization can add to improve program quality.
4. Discuss realistic summative evaluation activities that you and your
organization can add to improve program quality.
5. How should an organization respond to negative evaluation results? How
should an organization respond to positive results?
KEY IDEAS
•
Set clear and appropriate expectations!
•
Constantly observe and troubleshoot
•
Think critically (but constructively)
•
Small organizations can easily take big steps
toward evaluating program quality