Transcript ECE Finance: Framing Our Discussion

QRIS Standards Learning Table
Session #3: Efficiency in Monitoring:
Streamlining Documentation
National Center on Child Care Quality Improvement
Introductions and Updates
• Introduce the state team (Name, title, agency)
AL, CA, CT, GA, HI, NV, OR, VI
• Describe what your state team has been doing
with regard to your QRIS since our last call. It
could be related to the homework or other points
of interest in your work.
• Share strategies your state is using to bring these
concepts and materials back to workgroups
within the state. (agendas, topics, etc.)
• If a certain resource or idea has been particularly
helpful, tell us about that.
2
Key Point
Even a QRIS that appears simple
can become complex and
expensive to administer unless
steps are taken to streamline the
documentation procedures for
standards and sources of
evidence.
Sources of Evidence
For each standard you must:
• Clarify if/when documentation is required
Example: If you’ve already seen a source
of evidence in the past, do you need to
see it each year?
• Specify what documentation can be accepted
to verify compliance.
Efficiency Opportunity: Current
Assessment Tools as Source of Evidence
• Some Program/Classroom Assessment tools
measure the same content.
• Some Program/Classroom Assessment tools
measure criteria included in a state’s QRIS.
• Thus, a QRIS could use an Assessment tool –
such as ERS or PAS as the source of evidence.
Do the common tools measure the same concepts?
ECERS-R
FCCERS-R
CLASS
PAS
BAS





General Cognition
Social & Emotional
Development

Approaches to Learning

Heath/Physical Development




Business Practices
Family Involvement

Internal Communication

Leadership/Management
?






Efficiency Opportunity: Self-Report
• What standards are most appropriately
verified by self-report?
• What are effective procedures for validating
self-reporting? Is random sampling
appropriate?
• What documentation needs to be available for
review?
Efficiency Opportunity: Automation
• How can automation streamline the monitoring
process?
– Links to data-bases for licensing, registry, CACFP,
subsidy, accreditation
– Electronic scoring/reporting of ERS, CLASS, PAS/BAS
– Director portals or on-line applications (in real time)
that enable programs to upload evidence
– Provider-focused platforms that include downloadable
tools/templates to support compliance
Automation: Learning Table States
Results from both sessions:
• Links to Registry: AR, DE, NH, OR, GA
(OK, CA, PA developing)
• Links to Licensing: DE, KY, NM, OK, TX, OR, PA
• Links to PreK Monitoring: NM
• Links to Head Start Performance Review: AR, DE, OK
• On-line Upload of Documentation: AR, NM
Case Study: Maine
Michel Lahti, PhD
University of Southern Maine
QUALITY FOR ME – THE BASICS
• Licensing compliance
• Membership in MRTQ Registry
• Online application based upon a self-evaluation
Once the on-line application is submitted, the
provider immediately receives feedback from the
Quality for ME system regarding the anticipated
Step level
• Portfolio of documentation (random)
• On-site Observations (random)
The General Approach
• Web-based application
• Linkage to licensing database and PD Registry
– Relieves burden for all applicants
– Improves data quality in QRS application
– Feedback loop also improves data quality in linked database
• Criteria cross-walked with Accreditation criteria
• Self-report on remaining items
– About 50 specific questions if no Accreditation
– Reduced to just 5-10 questions depending on Accreditation
• Immediate feedback on how to move to next step in
each area
• Individual and aggregate reports shared with R&R
centers to facilitate technical assistance
QRS Step in Each of Eight Areas:
• compliance history/licensing status
• learning environment/developmentally appropriate
practice
• program evaluation
• staffing and professional development
• administrative policies and procedures
• parent/family involvement
• family resources
• authentic assessment
Criteria for achieving steps cross-walked with standards for the
following:
• NAEYC Accreditation
• NAEYC Candidacy
• National Association of Family Child Care Providers Accreditation
• National After School Association Accreditation
• American Montessori Society Accreditation
• Head Start: Zero Non-compliance Issues at Last Review / All
Non-compliance Issues at Last Federal Review Resolved
Key Data Linkages
Quality Rating
System
Univ of Maine, Orono
program license #
self-reported data
calculated data
…
Program
Licensing
MeDHHS, Augusta
program license #
contact info
capacity
license status
license expiration
type of program
…
Maine Roads To Quality (Prof Dev Registry)
Univ of Southern Maine, Portland
provider ID
provider education
provider training record
license # of program where provider employed
…
Maine Roads To Quality (Prof Dev
Registry)
Univ of Southern Maine, Portland
program license #
accreditation
…
Improves Data Quality at
Linked Databases
Final Step rating of 3, a program would have to be on Step 3 or 4 across
all eight areas.
Immediate Feedback to Applicant
If you have any questions regarding how or why Step values are
determined, please contact Allyson Dean with the Quality for Maine
program.
Section Name
Steps
Compliance History/Licensing
Status
1
Learning
Environment/Developmentally
Appropriate Practice
2
Program Evaluation
1
Staffing and Professional
Development
1
Administrative Policies and
Procedures
4
Parent/Family Involvement
2
Community Resources
4
Child Observations
2
Overall the Program is at Step 1
Recommendations
Immediate Feedback to Applicant
Figure xx. Example of detailed automatic immediate feedback to provider on how to achieve
next steps in each area of QRS evaluation (abbreviated).
Recommendations
Compliance History / Licensing Status
Current step is #1.
In order to move to step #2:

Your facility must have no substantiated serious violations in the past year.
Learning Environment / Developmentally Appropriate Practice
Current step is #2.
In order to move to step #3:

At least 50% of lead teachers (per program site) working with children ages 3-5 must have completed
the training on implementing curriculum based on Maine’s Early Childhood Learning Guidelines.
Program Evaluation
Current step is #1.
In order to move to step #2:

Your program must provide an opportunity to identify strengths and weaknesses that is inclusive of
staff, families, and administrators

Staff must be given feedback regarding the yearly self assessment
and Professional
Development
(specificStaffing
recommendations
for
each of 8 areas)
Current step is #1.
In order to move to step #2:
Data Usage…
• Monitor Enrollments and Characteristics of
Programs
• ERS Scores – Focus on Areas of Strength and
Improvement
• Monitor Program Progress through Step Levels
• Monitor Supports to Programs
• Infrastructure for Evaluation Projects:
– Comparing QRIS to non-QRIS Sites
– Investigate QRIS Standards: Use of Child Level
Assessments
– Validation Study
Lessons Learned from Maine
• Intention is to Build a
System, an Infrastructure
to Help Align ECE
Programming
• System Operation
Requires Ongoing
Attention - Keep it Valid
and Reliable
• Develop Working
Partnerships with State
Program Administrators
and University Research
Staff
• Importance of
Translating Data from
QRIS Monitoring into
Information for
Decision-making
BENEFITS TO JOINING
QUALITY FOR ME…
• Ability to accept Child Care Subsidy and receive a
payment differential based upon Step Level
• Assistance in paying for Accreditation fees and
cohort supports (some facility improvement grants)
• On-site technical assistance
• Scholarships to pursue early childhood education
degrees
• Tax credits for parents and providers
Automation of QRIS Implementation
Results from both sessions and other states
included:
• WELS (FL, NY, MS)
• MOSAIC (MI, CA)
• BRANAGH (LA)
• State - Developed Systems (AZ, ME, GA, PA)
Georgia’s Online System
Background
 Design - 2011-2012 Launched 1/2012
 Equal emphasis on Process Quality (ERS) and Structural
Quality (Program Portfolio
 In-house design and development of online system to manage
all of Quality Rated from process to data
Application
Training/Technical Assistance – Registration to tracking
Portfolio Submission – CQI Plans
Incentives Management
Resources
Reports and Data
Communication
Quality Rated Components
Research Questions – Data Dictionary
• Validation and research guided development
of online system
• With TA support from FPG
– Developed logic model
– Developed validation and evaluation model
– Created data dictionary
– Created reports
Validation Plan by Phase
Phase
Validation
Phase 1
Completed
12/2011
Content Validation (standards)
Phase 2
9/1/12
> = to 30
portfolios
submitted
Inter-rater reliability (portfolio/ERS)
Distribution of program standards and criteria
Correlating components
Portfolio accuracy - elements verified at ERS visit (long and short form)
ERS distribution
Phase 3
3/2013
Distribution of rating levels- rating distributions by program type (HS, rural vs. urban
types of children served, etc.)
Testing of various scoring structures/weighting and cut-offs
Phase 3
2014
Meaningful differentiation of quality levels with another outside variable i.e. CLASS
Evaluation Plan
Type of evaluation
questions
Year 1
2012-2013
Participation
x
Retention
x
Motivation
Quality Supports (TA,
Training, Incentives,
Year 2
2013-2014
Year 3
2014-2015
Year 4
Year 5
2015-2016 2016-2017
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Tiered Reimbursement,
Bonus Packages)
DECAL Resources
and Processes
Statewide Quality
Improvement
x
x
Logic Model
Mission Statement
Long Term
Goals
Years 5 and beyond
Intermediate Goals
Years 3-4
Short Term
Goals
Years 1-2
Output
Quality Rated will improve the quality of early education and school‐age care programs through
aligning and coordinating system-wide initiatives.
Families choose higher rated programs
O
n
g
o
i
n
g
V
a
l
i
d
a
t
i
o
n
a
n
d
E
v
a
l
u
a
t
i
o
n
25% of children receiving subsidy are in rated
programs
Families choose Quality Rated Programs
20% of children receiving subsidy are in a
Quality Rated program
Some programs increase quality by at least 1 level
51% of Georgia counties have 51% or more of eligible programs participating
Quality of ECE programs statewide has improved
For programs rated within the last 4 years:
• Enrolled programs continue to participate (measure)
For programs rated within the last 2 years:
• Retention at 75%
• Some programs increase quality by at least 1 level
Recruitment:
• 25% of registered FCC homes participating in Quality Rated
• 40% of licensed GDCH and Centers participating in Quality Rated
• 50% of programs that receive CC subsidy are participating in Quality Rated
Families have access to Quality Rated Programs Recruitment:
• 10 % of registered FCC homes participating in Quality Rated
10% of children receiving subsidies are
• 15 % of licensed GDCH and Centers participating in Quality Rated
participating in a Quality Rated Program
• 75 % of programs participating are rated
• 25% of programs serving subsidized children are participating in Quality
Rated
Develop a process to recruit and serve exempt but eligible entities (school
districts w/GA Pre-k, Head Start, DOD)
CHILDREN and FAMILIES
PROGRAMS
# of programs participating
# of programs rated with quality improvement plan
# of programs receiving support
Activity
Programs volunteer to participate in Quality Rated, ratings assigned, supports to improve quality (TA , PD, bonuses, incentives, tiered
reimbursement, public awareness campaign)
Input
Stakeholder developed Quality Rated standards, public and private funding, research findings
Online Site Users
•
•
•
•
•
•
•
Quality Rated staff
Technical assistance staff
Resource and referral agencies
Programs enrolled in QR
Incentive partners
Research team
Parents in 2013 – will see levels
Quick Tour of the Site
Lessons Learned in Georgia
•
•
•
•
Keep it simple
Resources make all of the difference
Transparency
INVOLVE THE RESEARCHERS
Web-Based Supports for Providers
The Raise Quality Tab on ECESharedResources.org:
http://national.ecesharedresources.net/index/
•
•
•
•
•
SharedSource PA,
Child Care Tennessee,
New Mexico Early Learning Alliance
Oregon
Maine
Efficiency Opportunity:
Multi-Site Centers
• How is documentation streamlined for multisite centers?
• What information can be gathered from the
central office?
• What must be gathered at each site?
• What standards might be revised given a
multi-site management framework?
QRIS Administration with
Multi-Site Centers: State Examples
• New Mexico – Verify documentation at central
office
• Oklahoma – Head Start participation
• Georgia – Cohort structure
Verification: Effective and Efficient?
Standards Think Tank Participants thought the
most effective and efficient verification methods
were:
• Objective Third Party Observation/Assessment
• Electronic Link to Licensing, Registry or other
Official Database
• Self-Report with Verification of Random
Sample
Questions, Reflections, Comments?
Next Session - Homework
1. When you consider your state’s initial QRIS standards or in the early stages of implementation,
what types of considerations are your team discussing?
• Ease of administration of the QRIS
• Standards that provide administrative or research data (e.g., must enroll in the state
Professional Development Registry)
• Ease of participation for early care and education programs in QRIS (e.g., Do the standards
begin very low to entice enrollment? How rigorous is the highest level?)
• Research base for the standards
• Standards that address emerging issues (e.g., diversity, child assessment, reflective practice)
• Alternate pathways for various provider types (under what circumstances for which types of
providers)
• Other
2. What data or research did you use to guide your selection of QRIS standards and what type of
data are you collecting to guide future revisions?
• Information from participants in the QRIS (programs, providers, parents)
• Data from your QRIS management system
• State Research
• National Research
• Other
Thank You
National Center on Child Care Quality Improvement
NCCCQI does not endorse any non-Federal organization, publication, or resource.
Follow-up Contacts:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
www.qrisnetwork.org