Transcript Slide 1

Kentucky 21st Century
Community Learning Centers
Fall 2011:
Quad-State Conference
Program Director’s Meeting
State Results and Next Steps
Center for Evaluation & Education Policy
Indiana University
© David P. Weikart Center for Youth Program Quality
1
Presentation Overview
1. Introduction
2. 2009-2010 National Comparison Results
3. Spring 2011 Site Visit Results
4. Self Assessment and Next Steps
© David P. Weikart Center for Youth Program Quality
2
National Comparison: Attendance
Regular Program Attendance: Kentucky vs. All States
Kentucky
84%
90%
All States
80%
67%
70%
60%
50%
40%
25%
30%
16%
20%
8%
0%
10%
0%
Less than 100 Students
100-200
200+
Regular Participants Served Per School Year
© David P. Weikart Center for Youth Program Quality
3
National Comparison: Attendance
Program Attendance: Special Services/ Program Classification
Regular Student Attendees: Kentucky vs. All States
73%
Kentucky
80%
All States
61%
70%
60%
50%
40%
30%
16%
20%
10%
12%
10%
2%
0%
Students with Limited
English Proficiency (LEP)
Students Eligible for
Students with Special
Free/Reduced Lunch Price
Needs or Disabilities
(FRPL)
Special Services or Program Classification
© David P. Weikart Center for Youth Program Quality
4
National Comparison: Grades
English/Language Arts Grade Changes
Among Regular Attendees During the 2009-2010 School Year:
Kentucky vs. All States
56%
60%
Kentucky
52%
All States
50%
40%
30%
25%
27%
19%
23%
20%
20%
12%
10%
0%
Increased Grade
Decreased Grade
No Change
Achieved HGP
ELA Grade Change Status
© David P. Weikart Center for Youth Program Quality
5
National Comparison: Grades
Math Grade Changes
Among Regular Attendees During the 2009-2010 School Year:
Kentucky vs. All States
56%
60%
51%
Kentucky
All States
50%
40%
27%
27%
30%
23%
22%
17%
20%
12%
10%
0%
Increased Grade
Decreased Grade
No Change
Achieved HGP
Math Grade Change Status
© David P. Weikart Center for Youth Program Quality
6
Spring 2011 Site Visits
Who? What?
•
•
37 visits to Cycle 7 KY 21st CCLC programs
February through April 13, 2011
How?
1. Site coordinator interview
2. School day teacher interview
3. Standardized observation protocol for academic and
enrichment activities
© David P. Weikart Center for Youth Program Quality
7
Site Visit Rating System
Rating System:
•
12 Items
(rated on a scale of 1 to 4)
1 = Must Address and
Improve
2 = Some Progress Made
3 = Satisfactory
4 = Excellent
•
48 Possible Points
© David P. Weikart Center for Youth Program Quality
8
Site Visits (CEEP)
Purpose of 2011 Site Visits
Elementary/Middle
School Programs
High
School Programs
1. Activities geared toward
rigorous academic enrichment
1. Activities promote academic
growth, remediation, and
development
2. Links to the regular school day
2. Links to the regular school day
3. Individual support and
opportunities for positive
interactions for youth
3. Participants contribute ideas,
make choices, and having
positive experiences
4. Relationships with schools,
parents, and other community
constituents
4. Establish partnerships and
employ successful recruitment
strategies
© David P. Weikart Center for Youth Program Quality
9
Elementary and Middle School Site Visit Results
Focus Area 1 & 2:
Activities Geared Toward Rigorous Academic Achievement
Links to the School Day
Homework Help
Focus
Area
1
3,0
Supplemental Academic Enrichment
2,9
Active Learning
3,0
School Day Curriculum
Focus
Area
2
2,7
School Personnel Involved
3,6
Activities Tailored to Student Needs
2,9
1
© David P. Weikart Center for Youth Program Quality
1,5
2
2,5
3
3,5
4
10
Elementary and Middle School Site Visit Results
Focus Area 3 & 4:
Opportunities for Individual Support and Positive Interactions
Relationships with Schools, Parents, and Community Organizations
Individual Support
Focus
Area
3
3,1
Positive Interactions with Adults
3,4
Positive Interactions with Peers
3,0
Well-Integrated with School
Focus
Area
4
3,6
Relationships with Parents
3,4
Partners with CBOs
3,8
1
© David P. Weikart Center for Youth Program Quality
1,5
2
2,5
3
3,5
4
11
High School Site Visit Results
Activities Offered by Sites
100%
86%
100%
90%
80%
70%
60%
50%
40%
30%
14%
20%
10%
0%
Credit Recovery
© David P. Weikart Center for Youth Program Quality
Homework Help/Tutoring
Goal Setting/Career
Development
12
High School Site Visit Results
Focus Area 1 & 2:
Activities Promote Academic Growth, Remediation, and Development
Links to the School Day
Homework Help/Tutoring
Focus
Area
1
3,2
Credit Recovery
3,0
Goal Setting/Career Development
3,1
School Day Curriculum
Focus
Area
2
3,0
School Personnel Involved
3,9
Activities Tailored to Student Needs
3,0
1
© David P. Weikart Center for Youth Program Quality
1,5
2
2,5
3
3,5
4
13
High School Site Visit Results
Focus Area 3 & 4:
Participants Contribute Ideas, Make Choices, and Have Positive Experiences
Establishes Partnerships and Maintains Successful Recruitment Strategies
Interest Based Choices
Focus
Area
3
Focus
Area
4
3,4
Youth Engaged in Program Decisions
3,3
Opportunities for Positive Interactions
3,3
Intentional Recruitment and Retention
3,3
Well-Integrated with School
3,7
Partners with Parents and CBOs
3,9
1
© David P. Weikart Center for Youth Program Quality
1,5
2
2,5
3
3,5
4
14
Areas for Improvement
1. Supplemental Academic Enrichment (ES/MS Programs)– what
activities does your program provide that are explicitly linked to
grade-specific standards?
2. Academic Activities Tailored to
Individual Student Needs (All
programs) – how does your
program identify students in need of
academic support and use school
data to plan programming?
© David P. Weikart Center for Youth Program Quality
15
Areas for Improvement (Cont’d)
3. Links to the School Day (All Programs)– how are the programs
connected to the school-day content without being “more school after
school?”
4. Credit Recovery (HS programs) –
what activities does your program
provide to give students the
opportunity to engage in credit
recovery and remediation?
© David P. Weikart Center for Youth Program Quality
16
st
21
Kentucky
CCLC
2011-2012
Action Plan
© David P. Weikart Center for Youth Program Quality
17
Implications for Kentucky’s 21st CCLC Program
1. States must demonstrate that systems are in place
for quality control and technical assistance.
2. Programs must demonstrate
that data are being used for
continual improvement.
3. Programs must demonstrate
increased ability to achieve
results.
© David P. Weikart Center for Youth Program Quality
18
What is program quality?
Program
staff
students
???
outcomes
training
youth program
Another way to say it…
What do we want to see in high quality youth programs?
© David P. Weikart Center for Youth Program Quality
19
YPQA: The Pyramid of Program Quality
Plan
Make choices
Engagement
Reflect
Lead and mentor
Be in small groups
Partner with adults
Experience belonging
Interaction
Encouragement Reframing conflict
Supportive
Skill building
Session flow
Active engagement Welcoming atmosphere
Environment
Psychological and emotional safety
Program space and furniture Emergency procedures
Safe
Healthy food and drinks
Physically safe environment
Environment
Youth Voice and Governance
Professional Learning Community
© David P. Weikart Center for Youth Program Quality
20
What is the YPQA?
1. A validated instrument
designed to assess the quality
of youth programs and identify
staff training needs.
Engagement
Interaction
Supportive Environment
2. A set of items that
Safe Environment
measures youth access
to key developmental experiences.
3. A tool which produces scores that can be used for
comparison and assessment of progress over time.
© David P. Weikart Center for Youth Program Quality
21
Kentucky 21st CLCC
Building Capacity for a Quality Improvement System (QIS)
STEP 1
STEP 2
Workshops
Site teams
offered for
schedule and
program and conduct selfsite leaders to assessment of
learn about the two program
selfofferings (one
assessment
academic and
process
one
enrichment
offering)
February
February/
March
© David P. Weikart Center for Youth Program Quality
STEP 3
STEP 4
Site teams
score selfassessment
instruments
and enter
results into the
Scores
Reporter
database
Site teams use
results of the
selfassessment
process (and
other data
sources) to
develop a
Program
Improvement
Plan
February/
March
May
STEP 5
STEP 6
Site teams
Implement
Program
Improvement
Plan
Site teams
report on
progress made
toward meeting
objectives in
Program
Site teams
Improvement
conduct YPQA Plan by
selfcompleting a
assessment
mid-year and
annually
end-of-year
progress report
Summer and December
forward
and May
22
Quality Improvement System (QIS)
During the 2010-2011 school year . . .
 A total of 134 KY 21st CCLC sites participated in
the Quality Improvement System or selfassessment process
 Represented roughly 80% of all program sites
© David P. Weikart Center for Youth Program Quality
23
Quality Improvement System (QIS)
Self Assessment Data
 A self assessment team (usually a program director and
site coordinator) observed two program activities.
 Activities observed included an academic enrichment and
social/cultural enrichment activity.
 Each site entered data from self assessment forms into
the online system Scores Reporter.
 Sites used a self-assessment form for younger youth or
older youth, depending on the grade levels of students
served.
© David P. Weikart Center for Youth Program Quality
24
Quality Improvement System (QIS)
Self Assessment: Aggregate Data
Younger Youth (N=69)
5,00
4,00
3,00
2,00
1,00
I. Safe Environment
© David P. Weikart Center for Youth Program Quality
II. Supportive Environment
III. Interaction
IV. Engagement
25
Quality Improvement System (QIS)
Self Assessment Data
 Observation scores represent a snapshot – this has
limitations and value.
 These are aggregate scores from multiple observations.
 The overall story is more important than the individual
numbers.
 What you do with the data matters most.
© David P. Weikart Center for Youth Program Quality
26
Quality Improvement System (QIS)
External Assessment Data
 External assessments were conducted for 17 sites during
the spring 2011 site visits.
 Sites in Cycle 7 who participated in the (QIS) during the
2009-2010 school year received an external visit.
 Assessments were conducted to give sites the opportunity
to compare self-assessment results to those conducted by
someone external to the program.
© David P. Weikart Center for Youth Program Quality
27
Quality Improvement System (QIS)
External Assessment Data
Keep in mind…
 External assessment scores are always lower than
self assessment scores.
 Observation scores represent a snapshot – this has
limitations and value.
 The overall story is more important than the individual
numbers.
 What you do with the data matters most.
© David P. Weikart Center for Youth Program Quality
28
Quality Improvement System (QIS)
Areas for Improvement: YPQA
1. Interaction—how does your program provide children with
opportunities to practice leadership, develop a sense of belonging,
participate in small groups, and interact positively with adults?
2. Engagement—what activities does
your program provide to give
children opportunities to set
goals/make plans, reflect on
activities, and make choices based
on their interests?
© David P. Weikart Center for Youth Program Quality
29
2012 YPQA Kentucky 21st CCLC
Self-Assessment Process: Tentative Timeline
MONTH
ACTIVITIES
•
YPQA Basics training – introduce all Cycles to the tool and timeline for the YPQA.
All new program directors and site coordinators who have not attended this
training in the past are required to attend. Location, Louisville. Date: TBD.

Self Assessment Team (Program Director and Site Coordinator), observes one
academic enrichment and one personal enrichment offering led by internal staff
and completes one YPQA assessment for each offering observed.
Early March 2012

Self Assessment Team attends a check-in/Scores Reporter webinar
End of March 2012

All scores must be entered into online system Scores Reporter

All self-assessment teams (except for those teams who participated in 2011) are
required to attend the Planning with Data workshop held in May.
Location, Louisville. Date: TBD.
Self-assessment teams will complete a Program Improvement Plan and submit for
review.
Early February 2012
February 14th-March
15th 2012
May 2012
November 2012;
May 2013


Self-assessment teams will complete interim and final reports to document
progress made towards goals and objectives included in their Program
Improvement Plans.
© David P. Weikart Center for Youth Program Quality
30
Quality Improvement System (QIS)
What Happens Next?
 Year 2011 Participants:
 Required to enroll additional program sites in the 2012 QIS
Cycle and train additional members of the Self Assessment
Team for the new site.
 Those who have not yet participated in the QIS
 Required to enroll a Self-Assessment Team in the 2012 QIS
Cycle. Team members typically include the Program Director
and Site Coordinator.
© David P. Weikart Center for Youth Program Quality
31
Kentucky 21st Century
Community Learning Centers
Fall 2011:
Quad-State Conference
Program Director’s Meeting
State Results and Next Steps
Center for Evaluation & Education Policy
Indiana University
© David P. Weikart Center for Youth Program Quality
32