Transcript Document

Update on Family Outcomes Work
Don Bailey and Melissa Raspa
ECO Advisory Board Meeting
March 7, 2012
Goals for Today
• Provide update on family outcomes APR data
• Introduce ECO family outcomes and
experiences measurement system selfassessment tool
• Discuss future of family outcomes
Early Childhood Outcomes Center
Part C
Measurement Approaches
• Tools
– NCSEAM survey (n = 25)
– ECO Family Outcomes Survey (n = 24)
• FOS-R (n = 3)
– State-developed survey (n = 6)
• Methodology
– Census (n = 44)
– Sampling (n = 11)
Early Childhood Outcomes Center
3
Part C
Population Surveyed
Timing
All families
Families > 6
months
Total
Point in time
25
12
37
At exit
3
6
9
At IFSP meeting
3
2
5
IFSP or exit
2
1
3
Total
33
21
54
Early Childhood Outcomes Center
4
Part C
Response Rates
• Overall response rate = 36%
• Tool
– ECO: 41%
– NCSEAM: 31%
– State: 31%
• Methodology:
– Census: 37%
– Sample: 34%
Part C
Response Rates
• Distribution
– In person: 46%
– Multiple methods: 37%
– Mail: 27%
• Return
– In-person: 75%
– Multiple methods: 36%
– Mail: 27%
Early Childhood Outcomes Center
6
Part C
Representativeness
Criteria
States (N)
States (%)
Race/ethnicity
41
75
Geography (district, county)
17
31
Gender
14
25
Child’s age (at referral, at survey)
13
24
Disability/eligibility category
6
11
Length of time in services
6
11
Income
3
5
Early Childhood Outcomes Center
7
Part C
APR Trend Data
100
90
80
70
60
50
40
30
20
10
0
84 86
80 81
85 86
81 83
90
87 88 90
FFY 06-07
FFY 07-08
FFY 08-09
FFY 09-10
4A
Know rights
Early Childhood Outcomes Center
4B
Communicate
needs
4C
Develop and learn
Part C
Trend Data – Indicator 4A
30
25
> 50%
50-59%
60-69%
70-79%
80-89%
90-100%
20
15
10
5
0
FFY 06-07
Early Childhood Outcomes Center
FFY 07-08
FFY 08-09
FFY 09-10
9
Part C
Trend Data – Indicator 4B
35
30
> 50%
50-59%
60-69%
70-79%
80-89%
90-100%
25
20
15
10
5
0
FFY 06-07
Early Childhood Outcomes Center
FFY 07-08
FFY 08-09
FFY 09-10
10
Part C
Trend Data – Indicator 4C
30
25
50-59%
60-69%
70-79%
80-89%
90-100%
20
15
10
5
0
FFY 06-07
Early Childhood Outcomes Center
FFY 07-08
FFY 08-09
FFY 09-10
11
Part C
APR by Survey Tool
100
90
80
70
60
50
40
30
20
10
0
88
82
89
91
89
91 89 91
80
ECO
NCSEAM
State
4A
Know rights
Early Childhood Outcomes Center
4B
Communicate
needs
4C
Develop and learn
12
Part B
Measurement Approaches
• Tools
– NCSEAM (n=43)
– Adapted NCSEAM or ECO (n=8)
– State (n=8)
• Methodology
– Census (n=21)
– Sampling (n=35)
Early Childhood Outcomes Center
13
Part B
Response Rates
• Overall response rate = 24%
• Distribution
– Multiple methods: 30%
– Mail: 17%
• Return
– In-person: 75%
– Multiple methods: 36%
– Mail: 27
Early Childhood Outcomes Center
14
Part B
APR Trend Data
Early Childhood Outcomes Center
15
Next Steps
• APR analyses for FFY 2010-2011
• Examine data quality
– APR by response rates
• Develop template for states to share family
outcomes data
• Provide TA to states on as-needed basis
Early Childhood Outcomes Center
16
FAMILY OUTCOMES AND
EXPERIENCES MEASUREMENT
SYSTEM
SELF ASSESSMENT TOOL
Early Childhood Outcomes Center
17
Purpose
• Provides a common language and organizing structure
for discussing family outcomes and experiences
• Assist states in developing a measurement system that
captures various aspects of family outcomes and
experiences, such as
–
–
–
–
benefits that families receive from program participation
family satisfaction with services
perceived helpfulness of the services
family involvement with the service delivery system
Early Childhood Outcomes Center
18
Framework and Self-Assessment
• FRAMEWORK
– Set of components and quality indicators
– Provides the structure for the self-assessment
• SELF-ASSESSMENT
– Scale that provides criteria for levels of
implementation within each quality indicator
– Rating assigned based on level of
implementation within each indicator
Early Childhood Outcomes Center
Partner States
•
•
•
•
Connecticut
Illinois
Minnesota
Texas
Early Childhood Outcomes Center
20
Framework
• Components
– 7 key areas of a measurement system
• Quality indicators
– 16 specific items that further describe each of
the components
• Elements
– Various examples of what constitute quality at
the indicator level
Early Childhood Outcomes Center
21
Quality Indicators
Components
Elements
Purpose
1.
State has articulated purpose(s) of COMS.
Data Collection and Transmission
Purpose
Data
Collection
and
Transmission
Analysis
Reporting
Using Data
Evaluation
2.
Data collection procedures are carried out efficiently and effectively.
3.
Providers, supervisors, and others involved in data collection have the
required knowledge, skills, and commitment.
4.
State's method for entering, transmitting, and storing data is effective and
efficient.
Analysis
State identifies accountability and program improvement questions related to
child outcomes.
Local programs identify accountability and program improvement questions
related to child outcomes.
5.
6.
7.
State agency analyzes data in a timely manner.
8.
Local programs analyze data in a timely manner.
9.
State agency ensures completeness and accuracy of data.
Reporting
10. State agency interprets, reports, and communicates information related to
child outcomes.
11. Local programs interpret, report, and communicate information related to
child outcomes.
Using Data
12. State agency makes regular use of information on child outcomes to improve
programs.
13. Local programs makes regular use of information on child outcomes to
improve programs.
Evaluation
14. State evaluates its COMS regularly.
Cross-system Coordination
a.
b.
c.
d.
e.
f.
g.
h.
State has…
State has…
State agency..
Representative..
State agency…
State ……
State provides…
State has..
15. Part C and 619 coordinate child outcomes measurement.
CrossSystem
Coordination
16. Child outcomes measurement is integrated across early childhood (EC)
programs statewide.
17. Child outcomes measurement is aligned with state’s early learning
guidelines/standards.
18. State has a longitudinal data system to link child outcomes data from EC
program participation to K–12 data.
22
Self-Assessment
• Provides guidance to states on what constitutes
a high quality family experiences and outcomes
measurement system.
• Assists states in setting priorities for improving
their measurement system
• Provides information to assist states in
advocating for resources for systems
development
23
How Does it Work?
• Each quality indicator has multiple elements
• Evidence for the extent of implementation for each
element is provided
• Each element is rated as
•
•
•
•
NY = Not Yet or Don’t know
IP = In Process
IF = Fully Implemented
NA = Not applicable (only where applicable)
• The quality indicator is given a rating based on the ratings
of the elements
Early Childhood Outcomes Center
24
Self-assessment Scoring
Implementation of Elements
Quality
Indicator
Score
All elements are fully implemented
7
Nearly all elements are fully implemented and the rest are in process
6
Most of the elements are fully implemented and the rest are in process.
5
At least one element is fully implemented and the rest are in process
4
All of the elements are in process
3
Some of the elements are in process
2
None of the elements are yet in process
1
25
Challenges with Family Framework
and Self-assessment
• Going beyond the APR: Family outcomes
and experiences
• Balancing expectations: Reality vs. ideal
• Scoring system
– Difficulty in reaching 7
– Not applicable items
Early Childhood Outcomes Center
26
Next Steps
• Finalize framework and self-assessment
tool and post on ECO web site
• National conference call to introduce to
broader audience
• Provide TA to states on as-needed basis
Early Childhood Outcomes Center
27
WHAT IS THE FUTURE OF
FAMILY OUTCOMES?
Context
• Family-centered (FC) practices are endorsed
by many professional organizations as central
to high-quality practice
–
–
–
–
–
Institute of Medicine
American Academy of Pediatrics
American Hospital Association
American Speech-Language Hearing Association
Division for Early Childhood
We know….
• Meta-analyses of research provides
strong evidence that FC practices lead to
many benefits for families and children
Dunst, Trivette, & Hamby (2007)
• Meta-analysis of 47 studies of family-centered
practices in a variety of settings
• Higher use of FC practices was associated with:
– Satisfaction with services*
– Self-efficacy beliefs*
– Parenting behaviors
– Personal and/or family well-being
– Social support*
– Child behavior and functioning
*Strongest associations
Pitrowski, Talavera, & Mayer (2009)
• Meta-analysis of 13 articles evaluating Health Steps, a
pediatric care model based on
– strength-based primary prevention
– the assumption that child well-being and family-centered care
are necessarily linked
– supporting the role of parents
– positive, caring, and enduring relations with parents.
• Parents in HS were
–
–
–
–
More satisfied with services
Exhibited more desirable parenting practices
More satisfied in their role as parents
Interacted more sensitively and appropriately with child
Broom & Enriquez (2009)
• Reviewed 9 randomized trials of familycentered interventions in treating children
with Type 1 diabetes
• Family-centered practices were associated
with
– Improved health outcomes for children
– Reduced diabetes-related conflict in families
– Improved family relations
Two studies using the National Survey of
Children with Special Health Care Needs
• When families perceived a strong
partnership between the child’s family and
the child’s service provider
– children’s health outcomes were improved
– children received more services
– families were more satisfied with services
Also
• Our NEILS analyses and our ECO papers
show that family-centered practices are
related to family outcomes
Epley, Summers & Turnbull (2011)
But research also shows that FC
practices are not widely implemented
• Medical Expenditure Panel Survey and National
Survey of CSHCN children found that only about
2/3 of families reported getting family-centered
care
• A hospital survey found that only about half used
FC rounds in pediatric in-patient settings
• Minority families generally report fewer FC
services and less satisfaction
Dunst (2012) argues that EI has not
evolved in tandem with what we know
So, if…..
• Professional organizations widely endorse
FC practices, and…
• Research shows that FC practices make a
difference…
• What should we do about the future of FC
practices and family outcomes in early
intervention and preschool programs?
Use of Family-Centered Practices
• We don’t know at a state or national level the
extent to which programs are actually using familycentered principles and practices
• We don’t know what factors predict or are
associated with variability in implementation of F-C
practices
• NEEDED: A comprehensive description of the
status of F-C practices in EI and preschool
programs, and a clear understanding of why some
programs are more family-centered than others
Family-centered principles, when integrated across
all program activities, ought to lead to family
outcomes.
Family-Centered Principles








Shared philosophy
Families as partners
Focus on strengths
Family choice of goals
and services
Collaboration and
coordination of service
Effective communication
Flexibility
Community-based
Program Activities








Referral and intake
Determining eligibility
Child assessment
Family assessment
Team meetings and
decision-making
Service provision and
coordination
Parenting practices
Community support
Family Outcomes





Understand child’s
abilities and special
needs
Know rights and
advocate effectively
Help child develop and
learn
Have support systems
Access the community
What is it about F-C practices that leads
to better outcomes?
• With the exception of the NEILS study and other
smaller studies, we don’t know the full range of
outcomes that F-C practices can affect
• The pediatric literature has far more studies of this
than EI
• NEEDED: Sophisticated large-scale studies
modeling how and which F-C practices influence a
variety of outcomes
The desirability & applicability of F-C
practices & family outcomes in preschool
• Nothing is known about F-C practices in preschool
programs for children w/disabilities
• The ECO family outcomes have never been
accepted or seriously discussed for preschool
• We have some evidence that family outcomes are
less likely to be achieved with older children
• NEEDED: Stakeholder opinions and consensus
regarding the desirability of FC practices &
appropriate family outcomes in preschool
F-C practices and child outcomes
• Is helping families a good outcome in and of itself, OR
must we show that this results in benefits for children?
• We have danced around this topic for 20 years, with
no resolution
• NEEDED: Either we need consensus on the argument
that family benefit is a worthwhile outcome OR we
need to show that F-C practices and family outcomes
are important to maximize child benefit
Potential Year 5 Family Activities
• Help states use framework to improve
systems for family outcomes assessment
• Targeted TA to states with low family
helpfulness ratings
• Collect, compile and analyze data from states
that have FOS data that can be linked with
child outcomes data
• Explore with one or two states or programs
use of FOS in preschool programs