Transcript Quality Matters Presentation.ppt

Quality Matters:
Building Capacity and Investment
in Youth Program Quality
The Center for Youth Program Quality
1
Objectives
• Understand how quality defined and measured
by the Youth Program Quality Assessment and
other quality assessment tools
• Learn about quality improvement systems
currently implemented in numerous statewide
and place-based networks
2
Systems for Quality
Leveraging existing “change” resources
Professional
Development
Local
Evaluators
Accountability
Measurement
POS Quality
Access to key
developmental
and learning
experiences
3
Quality & Reach…
Looking into the developmental white space
Outcome
Areas
Developmental
“White Space”
Ages
school
afterschool
Times of
Day/Year
At best, schools fill only a portion of
developmental “white space.”
Who fills the rest? And what is the
“locally appropriate mechanism for
monitoring the availability,
accessibility, and quality of
programs…” in school and out?
Saturating communities with
“ample programs” requires
improving the quality and reach
of all the systems, settings and
programs that touch young
people’s lives. Selective
replication and improvement are
not sufficient.
4
The Landscape of Youth Programs
Children’s Services in LA County
5
SOURCE: Margaret Dunkle
The Ready by 21™ Quality Counts Framework
Domains
Elements
Strong Policy /
Leadership
Horsepower
Decision-maker engagement & coordinating structures
Aligned policies for quality accountability and improvement
Shared vision, strong demand, active family/youth involvement
Strong, Stable
Program Base
Healthy program landscape (distribution & focus)
Cross-system program data base/info source
Cross-system convening/coordination mechanisms
Capacity to
Assess &
Improve
Programs
Buy in re definitions, quality standards, accountability requirements
Widely adopted assessment and monitoring procedures
Adequate assessment and improvement training/TA capacity
Capacity to
Recruit, Train,
Retain
Workforce
Cross-system provider networks and communications
Accurate data on workforce (skills, supports, recruitment, retention)
Professional development opportunities/incentives
6
What is program quality?
inputs
inputs
???
outcomes
inputs
youth
program
Another way to say it:
•What do we want to see in high quality youth programs?
7
Defining Quality
Several ways to organize:
• Readin’ + ‘Ritin + ‘Rithmatic (old-school)
• Affect + Active Learning + Metacognition (Education)
• Relatedness + Autonomy + Competence (Psychology SDT)
What kids need…
Relationship + Task + Increasing Complexity
Content
Therapeutic
process
What adults should do…
Our Quality Construct:
The Pyramid of Program Quality
Plan
Make choices
Engagement
Reflect
Lead and mentor
Be in small groups
Partner with adults
Experience belonging
Interaction
Encouragement Reframing conflict
Supportive
Skill building
Session flow
Active engagement Welcoming atmosphere
Environment
Psychological and emotional safety
Program space and furniture Emergency procedures
Safe
Healthy food and drinks
Physically safe environment
Environment
9
Viewing Quality in a Systems Context
Engage
Interaction
POS
Point Of Service
Support
Safety
PLC
Professional
Learning Community
SAE
System Accountability
Environment
10
Defining Quality
What can management do?
Community
Continuity
Daily supervision
& support by site coordinator
Pre-session planning
Frequent staff meetings
Learning
Features
Data & Information
Performance
Feedback
External monitoring
Lesson & curriculum review
Focusing
Features
Youth engagement
& content relevance
Management
Priorities/Values
Purposeful relationships
Youth voice structures
11
Defining quality at the system level
Current Regulatory Models Miss the POS
Source: (1998).The NSACA Standards for Quality School-Age Care.
There are thirty-six keys of quality and 144 total standards
Engagement
6 standards
Interaction
4 standards
Supportive Environment
33 standards
Safe Environment
49 standards
Youth Organizational Voice
1 standard
Professional Learning Community
19 standards
Other
32 standards
12
The Youth Program Quality Assessment
(YPQA)
– Validated observational
assessment tool
– Measures quality at the
point of service
– Assesses frequency
and access to key
developmental
experiences
– Can be used to assess
progress over time
– Content-neutral for use
across settings, ages,
systems
13
Structure of the Youth PQA
Form B
Organizational Interview
Form A
Observation
Ask questions, write, score
(2 hours)
Watch, write, score (3 hours)
Program Offering 1
Program Offering 2
Organization
Program Offering 3
Program Offering 4
14
Sample Item from the Youth PQA
“Domain”
III. Interaction
“scale”
III-L. Youth have opportunities to develop a sense of belonging.
Note: Structured refers to the quality of being intentional, planned, and/or named; it does not refer to informal conversation.
Indicators
1 Youth have no
opportunities to get to
know each other (beyond
self-selected pairs or
small cliques).
Supporting Evidence
3 Youth have informal
opportunities to get to know
each other (e.g., youth
engage in informal
conversations before, during,
or after session.
5 Youth have structured opportunities to
get to know each other (e.g., there are
team-building activities, introductions,
personal updates, welcomes of new
group members, icebreakers, and a
variety of groupings for activities)
“item row”
The Youth PQA consists of
7 Domains (4 in A, 3 in B)
30 Scales (18 in A, 12 in B); 103 item rows (60 in A, 43 in B)
15
What POS Quality Looks Like on the Ground
Occurred For All
Occurred For Some
Did Not Occur
• Sample of nearly 600 different youth workers
• Parallel findings in schools research
8/13/2008
The Center for Youth Program
Quality
16
Does it Work? Findings from Several
Samples
• POS quality-outcomes findings:
– Supportive environment related to: Attendance
– Interaction related to: Interest in program
– Engagement related to: Sense of challenge, sense of growth,
school-day reading, school-day suspension
– Note: No offerings get to high engagement without high support
and high interaction
• Quality Improvement (YPQI) Findings
– Scores increase from pre to post
– Scores increase in the targeted areas more
– Management practices are related to quality change (Vision,
Feedback, Continuity)
How we think about DDCI
- People change not programs
Prochaska, J.O., & DiClemente, C.C. (1982). Transtheoretical therapy toward a more integrative model of change. Psychotherapy:
Theory, Research and Practice, 19(3), 276-287.
Maintenance
Repeat
cycle
Action
POS
Point Of Service
Implementation
& coaching
PLC
Professional
Learning Community
Preparation
Planning with data
Contemplation
Quality assessment
Individual Change
Model
8/13/2008
SAE
System Accountability
Environment
Organizational
Context
The Center for Youth Program
Quality
18
Competencies for
Youth Development Practitioners
Emergence of POS quality culture
Spend time planning & prepping activities
Improve practice systematically
"Team focused on youth experience" mentality
Intentionality in POS quality practice
Icebreakers and inclusion
Cooperative grouping strategies
Targeting learning edge – scaffolding, engaging youth in planning &
reflection, active learning
Providing opportunities for youth voice and leadership
19
Competencies for Management
– Knowledge/understanding of positive youth
development methods individually and integrated as
the High/Scope participatory learning approach;
– Conduct reliable performance assessment based
on observation and validated measurement rubrics;
– Conduct performance coaching based on strengths
based transmission of performance data;
– Lead a staff team through a data driven quality
improvement process
20
System Level Outputs
– Accountability policy with high returns in staff
buy-in and learning;
– Common language and terminology
supporting focus on quality at the point of
service;
– An integrated professional development
investment that is integrated over time as a
sequence of PD experiences, integrated
across levels of organization and professional
roles, and integrated with ultimate program
purposes in positive youth development;
21
Outcomes
–High quality programs
–Better staff retention and retention
of better staff
–Greater impact on youth
development and learning
22
Systems for Quality
Accountability Policies in Places
• YPQA is part of state and county accountability policies:
–Cross sector (DHS& DOE) snapshots: Iowa, Washington, Arkansas
–Statewide 21st Century: Michigan, Maine, Minnesota, Rhode Island, New Mexico,
–Cities and Counties: Rochester, Detroit, Grand Rapids, Palm Beach
Rochester
Grand
Rapids
Minneapolis
Washington*
Chicago
New York
etroit
Iowa
Indianapolis
mbus
Georgetown
Divide
Columbus
St. Louis
Oklahoma
Rhode
` Island
`
`
Kentucky
Nashville
l
Austin
West Palm Beach
County
23
Designing Quality Improvement Systems (QIS)
Mostly Mangers
System
External quality
Capacity
assessment (a)
Managers and direct staff
Use of on-line
dashboards and
training (d)
External Quality Report with Norms
Quality
Advisor (e)
TOTs for quality
assess, coaching,
and youth work
methods (f,g)
Coaching & Training
Self-Assessed Quality Report
Program
SelfStaff Skill assessment
& Knowl
of Quality (b)
Use of on-line
dashboards and
training (d)
Planning
with Data (c)
Phase 1: Readiness &
Capacity
Targeted youth
work methods
training for
direct staff (h)
Quality
coaching by
managers (i)
Phase 2: Impact &
Sustainability
24
Defining the Purpose of Your QIS
Lower Stakes
Higher Stakes
(the creative middle)
Program
Self-Assessment
External
Assessment
Rough data to
get staff thinking and
discussing program
quality in the context
of best practice
Precise data for
internal and external
audiences for evaluation,
monitoring, accountability,
improvement, reporting
Less time
Less money
Impact internal
audiences
More time
More money
Impact internal and
external audiences
25
Columbus Indiana
Phase 1: Building Local Capacity
POS
Point
Of Service
SAE
System
Accountability
Environment
STEP 1
Decide to
build system
PLC
STEP 2a
Selfassessment
STEP 2b
Professional
Learning
Community
STEP 3
STEP 4
STEP 5
Plan for
improvement
Carry out
plan
Measure
change
External
assessment
August 25
Quality Matters
Presentation
August 26
Annually
Youth PQA Basics
October 8
Opt Phase 2
January 27-28
Planning with Data
Improvement Plan
Method Workshops
Quality Coaching
Ext Assessment
Program SA
Ext Assessment
Observe-Reflection
Planning with Data
Questions about…
Purpose?
Process?
Pilot?
Next Steps…
27