Transcript Evaluation of the Child Care Voucher Eligibility Reassessment

Evaluation of the Child Care Voucher
Eligibility Reassessment Administrative
Changes in Massachusetts
A policy research partnership between:
MA Department of Early Education and Care (EEC) ● Jennifer Louis (On-site PI)
Brandeis University ● Pamela Joshi (PI), Erin Hardy (Co-PI)
Boston University ● Yoonsook Ha (Co-PI)
Other partners: Nancy Marshall (Wellesley College); Rennie Center for Education
Research & Policy
About the Partnership
• This project establishes a new research
partnership between the CCDF-lead agency in
Massachusetts (EEC) and a team of policy
researchers from three Massachusetts
universities.
• The objective of the partnership is to study the
effects of state-specific CCDF administrative
changes that have immediate relevance to local
stakeholders and to child care researchers and
policymakers nationally.
Key Policy Challenges in MA Context
National
• A major challenge facing CCDF is the high rate of discontinuity in
subsidy receipt.
• CCDF’s high level of administrative burden for providers and families
can contribute to subsidy instability and hamper access to subsidized
care over time.
Massachusetts
• Past research finds that Massachusetts faces high turnover (median
duration of subsidy receipt is 6 months over 2 year period) and other
access issues due to high administrative burden (i.e. long waits and
language barriers).
• Massachusetts has implemented a series of administrative changes to
make CCDF program administration more family-friendly.
Research Motivation
• While state policymakers increasingly aim to
implement family-friendly CCDF policies, they
are faced with limited research regarding the
efficacy of administrative reform models.
• The proposed study of this new research
partnership is a mixed-methods evaluation of
a Massachusetts administrative change in the
eligibility reassessment process.
Key Definitions
Massachusetts is one of 13 states nationally that delivers child care subsidies to
families through two mechanisms:
• Voucher: Subsidy in the form of a voucher that can be used at any child care
provider that accepts vouchers.
• Contracted Slot: Subsidy in the form of a contracted “slot.” Certain
providers hold a contract with EEC for the provision of subsidized slots.
Therefore, providers can be:
• Voucher providers:* Child care providers that serve children receiving
subsidies in the form of vouchers.
• Contracted providers:** Child care providers that hold a contract with EEC
and that serve children receiving subsidies in the form of contracted slots.
*A “non-contracted” provider is a provider that serves children through
vouchers only.
**It is important to note that a child care provider can serve both children
utilizing contracted slots and children utilizing vouchers.
About the Administrative Changes
• In January 2012, EEC initiated administrative changes that:
o shifted the location and responsibilities for voucher
reassessment from 7 regional Child Care Resource and
Referral (CCR&R) centers to contracted providers who care
for income eligible families with vouchers
o involved the development and implementation of a new
web-based Voucher Management Application that allows
contracted providers to conduct voucher reassessment
electronically and mirrors the voucher reassessment
application used by CCR&Rs
o were based on the results of a pilot program from March
2010 to February 2011.
o Coincided with CCR&R budget cuts, including decreased
funds for voucher reassessment activities.
Populations Affected
• The changes only affects “voucher children” who use
their vouchers at “contracted providers” (providers
who accept vouchers and also have contracted slots)
o Prior to the change all voucher children were
required to visit the CCR&R to recertify
o Since the change, voucher children who use
contracted providers can recertify directly with
their providers
o Change affected 50% of the income eligible
voucher caseload (e.g. does not affect children in
families that receive TANF or participate in the child
welfare system).
Overview of Administrative Changes
Available Subsidy Types of Providers
that can be Chosen:
Types:
Non-Contracted
Provider
Voucher
Recertification
Entity:
CCR&R
(vouchers only)
Contracted
Provider
AFTER
Contracted
Provider
(vouchers & slots)
Contracted
Slot
Contracted
Provider
(vouchers & slots)
Contracted
Provider
*Notes: Administrative change depicted in red; Does not include DTA and DCF families
Reassessment Work Flow Before and
After Administrative Changes
NEW
.
R&R prepares original voucher
1 year later VR begins
CP:
.
CP has voucher
Calls parent
R&R:
.
R&R prepares VR
list
CP:
.
CP contacts family
on VR list
Parent:
.
Parents sign and
data letter
R&R:
CP inputs date
letter signed
.
R&R:
.
R&R either approves
or rejects VA. Voucher
is printed then faxed
to CP
Parent:
.
Parent comes in to sign
voucher process complete
NEW:
CP VR for families with one CP
CCRR for families with multi CPs
.
R&R prepares original voucher
1 year later VR begins for all
families
Single CP & CCRR Multi CP:
.
Families with only one CP & IE voucher: CP
conducts entire VR process
Families with multiple CPs: & DCF or DTA
vouchers: CCRR handles reassessment in VR
process
Parent:
.
Parent(s) sign & date
Voucher Application
and Fee Agreement
Single CP
CCRR Multi CP:
.
Enters data into system
Prints new voucher for
parent
Parent:
.
Parent(s)
sign & date
voucher
Single CP & CCRR Multi CP:
.
Attendance and billing records for all vouchers
updated by CP online for all vouchers (IE, DCF,
DTA) from initial placement, billing sent to
CCRR for processing. CCRR pays CP for vouchers
Why are these changes important
for families and providers?
Intended benefits Potential costs
• Change in location
• Quality tradeoffs: Lose
reduces travel burden
benefits of recertifying
• Families recertify with with CCR&Rs? (more
versed in child care
more familiar/known
options/quality)
staff
• Increased number of • Is new administrative
“access points” should “load” for providers too
reduce wait times and heavy, leading to hidden
costs and inefficiencies?
speed process
Overview of Evaluation Design
Mixed Methods Approach: Main Study Components
IMPACT
STUDY
IMPLEMENTATION
STUDY
SUBGROUP
ANALYSES
Research Questions by Component
IMPACT
STUDY
• What are the impacts
of the change on:
1. Continuity of
voucher receipt ?
2. Stability of care
arrangements ?
• How does
implementation
variation help explain
results?
• Spatial analysis: Is
location change
driving impacts?
IMPLEMENTATION
STUDY
SUBGROUP
ANALYSIS
• What key
• Is there variation
components of the
in impact by
recertification
selected focal
process changed?
sub-groups,
• Was the change
including Hispanic
model delivered as
families, families
intended?
living in suburban
• How did families and
vs. urban areas,
providers experience
and in different
the change
CCR&R regions?
(scheduling time,
paperwork, staff)?
• Improvements?
Quasi-experimental research design
Study Sample:
Voucher
Voucher
children
(who begin receiving
subsidies in 2012)
Study &
Recertification
Comparison Groups Entity at Month 12
Study group:
Voucher children
using contracted
providers
Study group:
Contracted
Provider
Comparison:
Voucher children
using non-contract
providers
Comparison:
CCR&R
Outcomes
(Measures)
• Continuity of
voucher receipt
(Monthly receipt
of vouchers)
• Stability in child
care
arrangements
(Monthly
measures of
arrangements)
Families voluntarily
select providers
Note: The observation period starts 1/1/2012 (effective date of change) and
culminates in a 24 month period (project year 1) and a 36 month period (project year 2)
Impact Study: Data and Methods
Data sources:
• MA CCDF administrative data (CCIMS)
• Contextual data from Census, American Community Survey, etc.
Key covariates:
• Recertification months
• Variables representing administrative burden (e.g., transportation
costs, distance traveled for recertification)
Empirical strategies:
• Spell analysis; discrete-time event history modeling to examine the
continuity of voucher use
• Mediation effect modeling to examine the stability of care
arrangements
Potential research questions for Year 3 & 4
• Impact on parental choice on care arrangements
• Impact on the child care market (e.g., the supply of child care)
• Other prioritized policy research issues
Impact Study: Spatial Analysis
Spatial analysis will be used for 3 primary purposes:
• Spatial descriptive work will inform impact study design and
result in policy-relevant maps for EEC
• Geocoding of administrative data will allow us to link to
contextual datasets and to create spatial measures that will
inform key research questions
o Spatial methods used to create key covariates, e.g.
neighborhood-level transportation access, foreign-born
presence
o Spatial variables required to assess a key research question
of whether the change in recertification location explains
observed impacts
• Lastly, we will consider testing our impact models in a spatial
regression framework to examine the role of spatial effects
(e.g. clustering/regional variation) in explaining impacts
Implementation Study:
Data and Methods
Multiple data sources:
• Key informant interviews with the designers and implementers of the
administrative changes
• Site visits to purposively selected CCR&Rs
• Provider interviews and recertification observation for the study group
(voucher children using contract providers) and the comparison group
• Parent interviews and focus groups – exposed and not exposed to
administrative changes; oversample Latino families
Empirical strategy:
•
•
•
•
Formulate a logic model of the administrative changes
Map work flow before and after the administration changes
Track implementation fidelity to the new model
Create qualitative database, coding and analysis of interviews
Future research (Year 4):
• In-depth analysis of Latinos and immigrant families subsidy use and
barriers
• Evaluation of other identified family-friendly practices
Practical Implications
National
• Contributes to the evidence base about practical steps to help improve
CCDF administrative processes that are associated with increased
stability of subsidy receipt and continuity of care for vulnerable
children.
• Informs federal guidance, technical assistance and dissemination
efforts to other regional/state CCDF initiatives.
Massachusetts
• Evaluates the effectiveness of the MA administrative changes, the
specific pathways of impact, and variation in pathways of the impacts.
• Identifies differential impacts for subgroups to inform tailoring of the
administrative changes for diverse service populations.
• Provider, parent and policy interviews will provide feedback on
changes, identify any hidden costs and benefits, help explain and
interpret impact results and give suggestions for future improvements.
Thank you / Request
We value your feedback and would
like to interview a subset of Board
members for key informant
interviews.
Questions?
Please contact:
Pam Joshi ([email protected])
Kate Giapponi ([email protected])