Transcript Slide 1

The Results are In!
Child Outcomes for OSEP EI and
ECSE Programs
Donna Spiker
Early Childhood Outcomes Center
at SRI International
October 13, 2011
(CCSSO-SCASS Meeting)
San Francisco, CA
What we will cover
• Quick review of the reporting requirement and
state approaches
• Share the national data
• Describe how the national data were computed
• Discuss the quality of the national data
• Discuss next steps for states
• Framework and self-assessment (briefly. …)
Early Childhood Outcomes Center
2
OSEP Reporting Requirements:
Child Outcomes
• Positive social emotional skills
(including positive social
relationships)
• Acquisition and use of
knowledge and skills (including
early language/ communication
[and early literacy])
• Use of appropriate behaviors
to meet their needs
Early Childhood Outcomes Center
3
OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer
to functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged
peers but did not reach it
d. Improved functioning to reach a level comparable to
same-aged peers
e. Maintained functioning at a level comparable to sameaged peers
3 outcomes x 5 “measures” = 15 numbers
Early Childhood Outcomes Center
4
Illustration of 5 Possible Paths
70
60
Score
50
40
30
20
10
0
1
6
11
16
21
26
31
36
41
46
51
56
Age in Months
Maintained functioning comparable to age peers
Achieved functioning comparable to age peers
Moved nearer functioning comparable to age peers
Made progress; no change in trajectory
Did not make progress
The Summary Statements
1. Of those children who entered the program below
age expectations in each outcome, the percent who
substantially increased their rate of growth by the
time they turned 3 years of age or exited the
program.
2. The percent of children who were functioning within
age expectations in each outcome by the time they
turned 3 years of age or exited the program.
Early Childhood Outcomes Center
6
State Approaches to Outcomes Data
Approach
Part C
Preschool
(56
states/jurisdictions)
(59
states/jurisdictions)
COS 7 pt. scale
41/56 (73%)
37/59 (63%)
One tool statewide
7/56 (13%)
9/59 (15%)
Publishers’ online
analysis
3/56 (5%)
6/59 (10%)
Other
5/56 (9%)
7/59 (12%)
Estimated Data for Part C, 2009-10
40%
35%
30%
25%
20%
15%
10%
5%
0%
a
b
c
d
e
Social relationships
1.7%
18.0%
18.6%
29.4%
32.4%
Knowledge and skills
1.5%
20.0%
24.8%
36.9%
16.8%
Action to meet needs
1.5%
17.5%
21.4%
37.1%
22.6%
Note: Based on 29 States with highest quality data
Early Childhood Outcomes Center
8
Estimated Summary Statement Data for Part C, 2009-10
80%
70%
60%
50%
40%
30%
20%
10%
0%
SS1
SS2
Social relationships
71.0%
61.8%
Knowledge and skills
74.2%
53.8%
Action to meet needs
75.6%
59.7%
Note: Based on 29 States with highest quality data
Early Childhood Outcomes Center
9
Estimated National Data for Early Childhood Special
Education, 2009-2010
40%
35%
30%
25%
20%
15%
10%
5%
0%
a
b
c
d
e
Social relationships
1.7%
11.5%
28.2%
34.3%
24.3%
Knowledge and skills
1.8%
13.4%
32.5%
34.4%
17.9%
Action to meet needs
1.6%
10.8%
20.9%
35.6%
31.0%
Note: Based on 33 States with highest quality data
Early Childhood Outcomes Center
10
Estimated National Summary Statements for Early
Childhood Special Education, 2009-2010
100%
80%
60%
40%
20%
0%
SS1
SS2
Social relationships
82.6%
58.7%
Knowledge and skills
81.5%
52.3%
Action to meet needs
81.9%
66.7%
Note: Based on 33 States with highest quality data
Early Childhood Outcomes Center
11
Criteria for States with Quality Data
1. Low percentage of
missing data
2. No odd patterns in “a” or
“e” categories
3. Did not use
questionable data
collection methods
Early Childhood Outcomes Center
12
Problem with Missing Data
• We don’t know how well the data the state
has represent the entire state.
• If the data are representative, the
percentages for the a to e Progress
Categories and the Summary Statements
won’t change as data are added on more
children.
Early Childhood Outcomes Center
13
Can we trust
these data?
Early Childhood Outcomes Center
14
Pattern checking for validity
• Checking across years
–How do the 2009-10 compare to the
data for 2008-09?
• Checking across methods
–How do the data for all states
compare to states with highest
quality data?
Early Childhood Outcomes Center
15
Part C, Outcome A:
Social Relationships
SS1: % who Increased
Growth Rates
80
70
71
70.2
64.5
SS2: % who Exited at
Age Expectations
80
70
64
67
61.3
60
60
50
50
40
40
30
30
20
20
10
10
0
0
All states
19 best
FFY 08-09
All states
29 best
FFY 09-10
64.7
61.8
All states 19 best All states 29 best
FFY 08-09
FFY 09-10
Part C, Outcome B:
Knowledge and Skills
SS1: % who Increased
Growth Rates
76.6
80
70.4
SS2: % who Exited at
Age Expectations
74.2
68.1
70
80
70
60
60
50
50
40
40
30
30
20
20
10
10
0
All states
19 best
FFY 08-09
All states
29 best
FFY 09-10
59
53.9
55.8
53.8
0
All states 19 best All states 29 best
FFY 08-09
FFY 09-10
Part C, Outcome C:
Meets Needs
SS1: % who Increased
Growth Rates
80
70
76
69.7
SS2: % who Exited at Age
Expectations
75.6
68.1
80
70
60
60
50
50
40
40
30
30
20
20
10
10
0
All states 19 best All states 29 best
FFY 08-09
FFY 09-10
64.6
60.5
61.5
19 best
All states
59.7
0
All states
FFY 08-09
29 best
FFY 09-10
Part B Preschool:
Social Relationships
SS2: % who Exited at Age
Expectations
SS1: % who Increased
Growth Rates
90
80
82.7
77.2
79.7
82.6
70
90
80
70
60
60
50
50
40
40
30
30
20
20
10
10
0
All states 15 best All states 33 best
FFY 08-09
FFY 09-10
61.7
58.7
62.1
58.7
0
All states 15 best All states 33 best
FFY 08-09
FFY 09-10
Part B Preschool:
Knowledge and Skills
SS1: % who Increased
Growth Rates
90
80
82.7
76.2
78.3
SS2: % who Exited at Age
Expectations
81.5
90
80
70
70
60
60
50
50
40
40
30
30
20
20
55.8
51.2
55.5
52.3
10
10
0
0
All states
15 best
FFY 08-09
All states
33 best
FFY 09-10
All states
15 best
FFY 08-09
All states
33 best
FFY 09-10
Part B Preschool:
Meets Needs
SS2: % who Exited at Age
Expectations
SS1: % who Increased
Growth Rates
90
80
81.6
75.3
78.1
81.9
90
80
70
70
60
60
50
67.8
67.2
66.7
66.7
50
40
40
30
30
20
20
10
10
0
All states 15 best All states 33 best
FFY 08-09
FFY 09-10
0
All states 15 best All states 33 best
FFY 08-09
FFY 09-10
What do these data tell us?
• Nationally, a high proportion of children
who receive Part C and ECSE services
are showing greater than expected
progress.
• Nationally, many (over half) are exiting the
program functioning like same age peers
in at least one of the outcomes.
Early Childhood Outcomes Center
22
Using data at state level……
• Never intended to compare states
• OSEP wants states to begin by comparing
own data to past data
• States set targets to compare yearly
performance to those targets
• States implement program improvement
activities
• OSEP looking at data in monitoring visits
Early Childhood Outcomes Center
23
Linkages to other state EC
efforts……
• Some states are beginning to develop
linkages between these OSEP data and other
assessment in their states, for example:
– Colorado’s Results Matter
– Results Matter in Nebraska
– Kentucky Early Childhood Data System (KEDS)
Early Childhood Outcomes Center
24
Colorado’s Results Matter
Early Childhood Outcomes Center
25
Supporting States in Building a Child
Outcomes Measurement System
(just briefly………)
Framework and Self-Assessment
• FRAMEWORK
– Set of components and quality elements
– Provides the structure for the self-assessment
• SELF-ASSESSMENT
– Scale that provides criteria for levels of
implementation within each quality element
– Rating assigned based on level of
implementation within each element
Early Childhood Outcomes Center
27
Early Childhood Outcomes Center
28
Additional Information & Questions
Additional
information:
Questions
www.the-ecocenter.org
Early Childhood Outcomes Center
29