Transcript Youth Program Quality Improvement

Oakland
Planning with Data &
Kick-Off Webinar
September 1, 2011
Laenne Thompson
Weikart Center
A joint venture between the Forum and High/Scope.
Please press *6 to
mute and un-mute. If
you can hear it, so can
we.
© David P. Weikart Center for Youth Program Quality
2
Please press *6 to
mute and un-mute. If
you can hear it, so can
we.
This webinar is being
recorded. If you come late,
leave early or just want to
refresh your memory. You can
access the recording
afterwards.
© David P. Weikart Center for Youth Program Quality
3
Please press *6 to
mute and un-mute. If
you can hear it, so can
we.
This webinar is being
recorded. If you come late,
leave early or just want to
refresh your memory. You can
access the recording
afterwards.
Be Prepared: This
webinar will be
participatory. Be ready
to be called on.
© David P. Weikart Center for Youth Program Quality
4
Agenda






Quick Introductions
Using Data to Plan & Implement Program Improvements
Improvement Planning Overview
Submitting Improvement Plans Online (tutorial)
2011 - 2012 YPQI Kick-Off
Questions and Next Steps
© David P. Weikart Center for Youth Program Quality
5
Introductions
 Who’s here today?
 Each site, please share:
 The name of your organization
 Something new you’ve tried at your site that went well since
we last met
© David P. Weikart Center for Youth Program Quality
6
Youth Program Quality Improvement (YPQI)
ASSESS
PLAN
IMPROVE
Create
improvement plan
(YPQI Team)
Carry out
improvement plan
(Mgr coaches;
Stf do)
Conduct SA
(YPQI Team)
Conduct EA
(External assessor)
REPEAT
[Collect other data]
© David P. Weikart Center for Youth Program Quality
7
Improvement Planning Practice
 As a group, we’re going to have a practice improvement
planning meeting based on aggregate OFCY YPQA data.
© David P. Weikart Center for Youth Program Quality
8
Score Reports: Keep in Mind
 Observation scores represent a snapshot –this has
limitations and value.
 The overall story is more important than the individual
numbers.
 What you do with it (how you step up to action)
matters.
 Reference: ‘How to Interpret PQA Data’ handout
© David P. Weikart Center for Youth Program Quality
9
Some pointers about YPQA data
 Scores of all Fives
 Not useful for identifying areas of improvement.
 Extreme Variation
 Might signify a misunderstanding of what the tool is measuring.
 Typical trend
 This distribution common as you become more
comfortable with the process and the tool.
 Program self assessment scores may be lower
in subsequent years because raters are being
more honest or more critical as they develop
their reliability.
© David P. Weikart Center for Youth Program Quality
10
OFCY External Assessment* vs. National Sample**
*n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After
School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites
OFCY Average
National Average
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
OVERALL SCORE
I. SAFE
© David P. Weikart Center for Youth Program Quality
II. SUPPORTIVE
III. INTERACTION
IV. ENGAGEMENT
CULTURAL
11
Aggregate Data Graph
 Things to consider:






What does it mean?
What do you see here?
What does the data tell us?
Strengths?
Areas for growth?
What should we do as a response?
© David P. Weikart Center for Youth Program Quality
12
Aggregate Data Graph
 What scores seem lower than you would want?
 What item/domain do we want to focus on?
© David P. Weikart Center for Youth Program Quality
13
OFCY External Assessment* vs. National Sample**
*n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After
School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites
OFCY Average
National Average
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
OVERALL SCORE
I. SAFE
© David P. Weikart Center for Youth Program Quality
II. SUPPORTIVE
III. INTERACTION
IV. ENGAGEMENT
CULTURAL
14
Sample Improvement Plan
© David P. Weikart Center for Youth Program Quality
15
Sample Improvement Plan
© David P. Weikart Center for Youth Program Quality
16
Sample Improvement Plan
© David P. Weikart Center for Youth Program Quality
17
Steps to make an Improvement Plan
Set goal & address elements of quality (YPQA items)
Implementation: Develop action steps for each goal
A.
B.
Timeline
Evaluate
-
C.
(Mini) Self Assessment
Staff meeting check in
Observation-Reflection
Other?
**Don’t forget- Involve staff in
the process!
Discuss desired support from administrator
-
Training, information, etc…
Youth Work Methods courses & other further YPQA work
© David P. Weikart Center for Youth Program Quality
18
More to Consider: Youth Survey Data
 How to use this data in your Improvement Plans?
 What from the Youth Survey Data stands out as highest
priorities for improvement?
 What YPQA data does it correlate with?
 Does the Youth Survey data help you prioritize which YPQA
items to tackle in your Improvement Plan?
© David P. Weikart Center for Youth Program Quality
19
Discussion
 How do you feel about this process?
 Questions or potentially challenges about replicating it
at your program/site?
© David P. Weikart Center for Youth Program Quality
20
Submitting Improvement Plans Online (tutorial)
 You will receive your login information in an email
today from Leah Wallace of the Weikart Center
 If you have not received an email with those please
contact [email protected] as soon as possible.
 These logins will be used to enter both:
 The Spring 2011 Improvement Plan (due 9/30)
 The Fall 2011 YPQA Self-Assessment data
© David P. Weikart Center for Youth Program Quality
21
Login at scoresreporter.cypq.org
© David P. Weikart Center for Youth Program Quality
22
Why do I need to enter my plans?
 It allows you to track your goals and
your progress in reaching those goals
from year to year.
 It allows your program leader to tailor
professional development
opportunities
 It helps create an overall picture of
what organizations in your city are
prioritizing to improve the quality of
their programs.
© David P. Weikart Center for Youth Program Quality
23
YPQI TIMELINE
DATE
ACTIVITY
September 1, 2011
Planning with Last Year’s Data + Kick-Off for 2011 – 2012
September 30, 2011
Improvement Plans DUE
October 6, 2011
Youth PQA Basics Training
October 2011 –
January 2012
Conduct Self-Assessments
Late February 2012
TBA
All Self-Assessment Data DUE
March 1, 2012
Planning with Data
© David P. Weikart Center for Youth Program Quality
24
Next Steps & Expectations
 Meet with your program team and
choose goals for your
Improvement Plan
 Determine actions steps and
monitoring activities for each goal
 Finalize your Improvement Plan:
enter online by September 30
© David P. Weikart Center for Youth Program Quality
25
2011 – 2012
OFCY Grantee
Youth Program Quality Improvement
Kick-Off
© David P. Weikart Center for Youth Program Quality
26
YPQI Kick-Off: 2011 - 2012





Review: What is the YPQA?
What is the YPQI?
Self-Assessment
Next steps and Key Dates
Q/A
© David P. Weikart Center for Youth Program Quality
27
Quality Matters
Transfer: Application of skills in new settings
Time: Multiple sessions
Setting: Point of service
Quality of
Instruction
& Content
Youth
Engagement
© David P. Weikart Center for Youth Program Quality
Proximal skill
building
•Social emotional
•21st Century
•Cognitive
Short-term
outcomes
•Achievement
•Healthy behavior
•School retention
28
YOUTH
PROGRAM
QUALITY
ASSEMENT
(YPQA)
© David P. Weikart Center for Youth Program Quality
29
What is the YPQA?
The Youth PQA is a
research validated
instrument designed
to assess the quality
of youth programs
for the purposes of
accountability,
evaluation, and
program
improvement.
© David P. Weikart Center for Youth Program Quality
The instrument has
been used in a wide
variety of settings
including after-school,
community-based,
camp, drop-in, and
mentoring.
30
The Pyramid of Program Quality
 Youth are emotionally and
physically safe.
Plan
 Youth are supported by
staff and engaged in
hands-on skill building
 Youth work together and
experience a sense of
belonging
Make choices
Reflect
Lead and mentor
Be in small groups
Partner with adults
Experience belonging
Encouragement
Skill building
Active engagement
Reframing conflict
Session flow
Welcoming atmosphere
 Youth have access to high Psychological and emotional safety
level developmental
Program space and furniture
Emergency procedures
experiences: planning,
Healthy food and drinks
Physically safe environment
setting, goals, making
choices and reflecting
© David P. Weikart Center for Youth Program Quality
31
The YPQA is…
 …rooted in idea of Positive Youth Development
 …content-neutral
 …a measurement that guides learning and professional
development
 …a tool to measure what happens at the point-ofservice, where staff and youth interact
© David P. Weikart Center for Youth Program Quality
32
Higher scores at higher domains are associated
with higher levels of youth engagement
 Youth self-reports of:




© David P. Weikart Center for Youth Program Quality
Belonging
Interest
Challenge
Learning
33
High quality instruction provides youth with
opportunities to practice emerging social and
emotional skills…




Efficacy
Communication
Empathy
Problem Solving
… that support success
in adolescence and
early adulthood.
© David P. Weikart Center for Youth Program Quality
34
Lower scores are associated with youth disinterest
© David P. Weikart Center for Youth Program Quality
35
WHY THE YPQI?
YOUTH
PROGRAM
QUALITY
IMPROVEMENT
© David P. Weikart Center for Youth Program Quality
36
Theory of Change: Multiple Levels of Settings
This is the youth at your programs
Point of Service
Setting
Quality Instruction
& Proximal Child Outcomes
© David P. Weikart Center for Youth Program Quality
37
Theory of Change: Multiple Levels of Settings
This is your
organization and
staff
Organizational
Setting
Continuous Improvement
Practices for Site Teams
© David P. Weikart Center for Youth Program Quality
38
Theory of Change: Multiple Levels of Settings
Organizational
Setting
Point of Service
Setting
© David P. Weikart Center for Youth Program Quality
What you do here….
Will improve the
experiences for kids
here
39
Benefits of YPQI
 Creates a common language of quality
 Emphasizes positive youth development
 Provides a framework for professional development
© David P. Weikart Center for Youth Program Quality
40
What’s new for 2011 - 2012?
SELF-ASSESSMENT AND THE YPQI
© David P. Weikart Center for Youth Program Quality
41
This is the PQA Self Assessment Process
ASSESS
IMPROVE
Standardized
Training for Assessment
Instructional of Instruction
Skills
IMPROVE
Coaching and
Performance
Feedback
© David P. Weikart Center for Youth Program Quality
PLAN
Team-based
Planning
with Data
42
Assess
The PQA allows you to
observe what actually
goes on in your youth
programs?
How often do you have the
time or energy to simply
observe a program led by
another staff?
How often do you rely on
what you think happens
in your programs?
© David P. Weikart Center for Youth Program Quality
43
What does the Assess Stage look like?





Team Selection
Team Training
Planning the Data Collection
Collecting Data
Team Scoring Meeting
© David P. Weikart Center for Youth Program Quality
44
Team Selection
 Select 1-3 people from
your organization to be
on the self-assessment
team
 Make sure to include
frontline staff
 Attend a Youth PQA
Basics Training
© David P. Weikart Center for Youth Program Quality
45
Observations and Consensus
 Plan to observe 2-4 program
offerings
 Observe at least 15-20
minutes, but try to observe
the entire offering
 Come together with all your
data and come to consensus
scores for the entire program
based on that data.
 The conversation is the most
important part of this process!
© David P. Weikart Center for Youth Program Quality
46
Data Entry
You will enter your final scores
into an “online scores
reporter” data base.
This will allow you to make
reports and compare your
data form year to year.
© David P. Weikart Center for Youth Program Quality
47
Plan
Looking at the results of
your assessment gives
you a chance to make
concrete plans.
What are your program
strengths?
What do you want to
improve?
© David P. Weikart Center for Youth Program Quality
48
Planning with Data
This workshop will allow you to look at your assessment
data in depth and create improvement plans based on
it.
© David P. Weikart Center for Youth Program Quality
49
Improve
During the improve phase
you will implement your
plans.
What kind of professional
development do you
need?
What additional resources?
What changes to the
curriculum?
© David P. Weikart Center for Youth Program Quality
50
IMPROVE: Methods Workshops
This improvement process is supported with professional
development workshops
Aligned Youth Work Methods Courses
Voice & Choice (V-C, V-D, III-N)
Planning & Reflection (IV-P, IV-R)
Engagement
Interaction
Building Community (III-L)
Cooperative Learning (III-M)
Active Learning (II-H)
Scaffolding for Success (II-I)
Ask-Listen-Encourage (II-J)
Reframing Conflict (II-K)
Supportive Environment
Structure and Clear Limits
Safe Environment
Program Component: Homework Help
These 2- to 3-hour workshops are aligned with the items of the Youth PQA found in the pyramid.
Workshops may take place for a network or a program.
A joint venture between the Forum and High/Scope.
NEXT STEPS
© David P. Weikart Center for Youth Program Quality
52
YPQI TIMELINE
DATE
ACTIVITY
September 1, 2011
Planning with Last Year’s Data + Kick-Off for 2011 – 2012
September 30, 2011
Improvement Plans DUE
October 6, 2011
Youth PQA Basics Training
October 2011 –
January 2012
Conduct Self-Assessments
Late February 2012
TBA
All Self-Assessment Data DUE
March 1, 2012
Planning with Data
© David P. Weikart Center for Youth Program Quality
53
Any Questions?
© David P. Weikart Center for Youth Program Quality
54