2012 OSEP Leadership Conference

Leading Together to Achieve Success from Cradle to Career Child Outcomes for Early Intervention and Preschool Special Education: The National Picture Kathy Hebbeler ECO at SRI Lynne Kahn ECO at FPG Presentation # EB7

2

Overview

• • • National numbers and the quality of the state data • The findings for 2010-11 Findings for the last 3 years State variation and data quality Early Childhood Outcomes Center

3 OSEP Reporting Requirements: Child Outcomes • Positive social emotional skills (including positive social relationships) • Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) • Use of appropriate behaviors to meet their needs Early Childhood Outcomes Center

If you have not seen it….

4 Early Childhood Outcomes Center

State Approaches to Outcomes Data, 2010-11

Approach

COS* 7 pt. scale

Part C (N=56)

43/56 (77%) One tool statewide Publishers’ online analysis 7/56 (13%) 3/56 (5%) Other * Child Outcomes Summary Rating 5/56 (9%)

Preschool (N=59)

36/59 (61%) 9/59 (15%) 6/59 (10%) 8/59 (14%)

6

Child Outcomes: What we know

• • • States are in various stages with regard to building their child outcomes measurement systems Some phrased in Some changed approaches Early Childhood Outcomes Center

3 Methods Methods for Calculating National Estimates 1. States with the highest quality data weighted by child count to represent all states* 2. All states weighted by child count 3. All states averaged (each state weighted as 1) 7 *The data we will be presenting for the national picture Early Childhood Outcomes Center

8

Identifying the States with Highly Questionable Data

• • • Criteria for eliminating states: Not reporting data on enough children Odd patterns in the data Review of method reported suggested the data were of questionable quality Early Childhood Outcomes Center

9

Continued increase in the number of states included

Number of States that Met Criteria for Inclusion in the National Analysis Pt. C Part B Preschool

08-09 09-10 10-11

19 29 39 15 33 36 Early Childhood Outcomes Center

OSEP Reporting Categories

10 Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

Illustration of 5 Possible Paths 70 60 50 40 30 20 10 0 1 6 11 16 21 26 31 36 Age in Months 41 46 51 56 Maintained functioning comparable to age peers Achieved functioning comparable to age peers Moved nearer functioning comparable to age peers Made progress; no change in trajectory Did not make progress

12 • And now the data….

And now the data….

Early Childhood Outcomes Center

13 *Based on 39 states with highest quality data

*Based on 36 States with the highest quality data.

14

15 The Summary Statements 1. Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program.

2. The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program.

Early Childhood Outcomes Center

*Based on 39 states with highest quality data 16

*Based on 36 States with the highest quality data.

17

Part C: Trends over Time

18 Early Childhood Outcomes Center

Part C: Trends over Time

19 Early Childhood Outcomes Center

Part B Preschool: Trends over Time 20 Early Childhood Outcomes Center

Part B Preschool: Trends over Time 21 Early Childhood Outcomes Center

Looking for Patterns

22 Early Childhood Outcomes Center

Early Childhood Outcomes Center 23

% 24 Early Childhood Outcomes Center

Early Childhood Outcomes Center 25

26

Part C: States continue to report on more children

Total Number of Children States Included in the Child Outcomes Data

Number of States and Jurisdictions

Number of children reported

2007-08 (N=56) 2008-09 (N=56) 2009-10 (N=54) 2010-11 (N=54)

99 or less 100-499 500-999

13 25 6

1,000-1,999

9

2,000+

3 Early Childhood Outcomes Center 6 16 13 11 10 5 9 10 10 20 4 10 7 13 22

27 Early Childhood Outcomes Center

Part B Preschool: States continue to report on more children

Number of children reported Total Number of Children Included in Progress Data Number of States and Jurisdictions

FFY 2007 (N=58) FFY 2008 (N=58) FFY 2009 (N=58) FFY 2010 (N=59)

10 or fewer 10-99 100–499 500-999 1000–1999 2000–2999 3000–4999 5000-8999 9000+

5 3 1 1 11 14 10 8 5 1 7 6 12 7 10 11 3 1 9 7 1 1 6 4 10 9 11 1 7 2 8 5 12 13 9 2 Early Childhood Outcomes 28

Early Childhood Outcomes Center 29

% 30 Early Childhood Outcomes Center

Maximum 75 th Percentile 50 th Percentile 25 th Percentile Minimum Outlier

Variation across states percent “a” : Part B 619 2010-11

Variation across states percent “b” : Part B 619 2010-11

Variation across states percent “c” : Part B 619 2010-11

Variation across states percent “d” : Part B 619 2010-11

Variation across states percent “e” : Part B 619 2010-11

Variation across states in greater than expected growth (Summary Statement 1) : Part B 619 2010-11

Variation across states in exits at age expectations (Summary Statement 2) : Part B 619 2010-11

Variation across states percent “a” : Part C 2010-11

Variation across states percent “b” : Part C 2010-11

Variation across states percent “c” : Part C 2010-11

Variation across states percent “d” : Part C 2010-11

Variation across states percent “e” : Part C 2010-11

Variation across states in greater than expected growth (Summary Statement 1) : Part C 2010-11

Variation across states in greater than expected growth (Summary Statement 2) : Part C 2010-11

46 Part C Improvement Activities

# states 11 5 3 3 2 4 2 2 2 7 Topic/theme

Training/TA on evidence based practices, effective practices, family centered services, DAP TACSEI/CEFEL Training/TA on social emotional development (not TACSEI/CEFEL specific) Training/TA on more functional IFSP outcomes Revised IFSP format to enhance practices Training/TA on Literacy Training/TA on RBI Training/TA on ASD Training/TA on PSP Other topics: coaching, instruction, self-help, transition practices, Special Quest, assistive technology, relationship based services in NE Early Childhood Outcomes Center

47

Conclusions

• • • The data show good outcomes for many children who attend Part C and Part B, Preschool.

The data quality continues to get better.

The data will be a powerful tool for program improvement when – The data are consistently good at the local level – States and local programs consistently use it.

Early Childhood Outcomes Center

You might be interested in…..

48 Sponsored jointly by ECO and NECTAC Early Childhood Outcomes Center

For more information….

49 www.the-eco-center.org

Early Childhood Outcomes Center

50 Early Childhood Outcomes Center

Identifying States with Quality Data 51 Early Childhood Outcomes Center

Calculating Missing Data for Part C

Proxy for missing data = Number with data for C3/ Exiting Data (618) • • Do not expect this number to be 100% ..but we don’t expect it to be 10% either 52 Early Childhood Outcomes Center

Percentage of Exiters Reported by Part C Continues to Increase 53 Early Childhood Outcomes Center

Calculating Missing Data for 619

Proxy for missing data = Number with data for B7/ Child count • • Do not expect this number to be 100% ..but we don’t expect it to be 10% either 54 Early Childhood Outcomes Center

55 Percentage of Child Count Reported by Part B Preschool Continues to Increase

Pt. B Preschool: % of Child Count Reported in Child Outcomes Data

10 8 6 4 2 0 18 16 14 12 08-09 Early Childhood Outcomes Center 09-10 10-11 <10% 10-20% 21-30% 31-40% 41-50% >50%

56 Many states reporting a low percentage for “a” but still have a few outliers.

Number of States by % Reported in “a,” 2010-11 <2% 2 to <5% 5 to <7% 7 to <13% 28 to 49%

Part C

32 11 5 0 3

Part B Preschool

30 11 3 5 0 Early Childhood Outcomes Center

Reason Part C state was excluded

State is sampling No outcomes data reported Reported outcomes data on less than 28% of reported exiters Had at least one outcome with category a greater than 10% or category e greater than 65% Reported outcomes data on less than 28% of reported exiters AND Had at least one outcome with category a greater than 10% or category e greater than 65% Questionable data quality based on review of SPP/APR and knowledge gained through TA

States included in the analysis Number excluded

3 0 3

Percent (n=51)

6 0 6 4 8 1 1

39

2 2

76

Early Childhood Outcomes 57

Reason Part B state was excluded

State is sampling No outcomes data reported Reported outcomes data on less than 12% of child count Had at least one outcome with category a greater than 10% or category e greater than 65%

Number excluded

4 1 2 4 Reported outcomes data on less than 12% of child count AND Had at least one outcome with category a greater than 10% or category e greater than 65% Questionable data quality based on review of SPP/APR and knowledge gained through TA No child count data available Reported Summary statements but no progress categories

States included in the analysis

0 2 1 1

36 Percent (n=51)

8 2 4 8 0 4 2 2

71

Early Childhood Outcomes 58

59 Producing National Estimates Using the States with the Highest Quality Data 1. Divide the states into those with large, medium, and small child counts.

2. Identify the states that appear to have the best data.

3. Weight by child count and calculate the overall OSEP percentages using the “best data” states in each category.

4. Use the OSEP percents and total child count for each category to get the national estimates.

Early Childhood Outcomes Center

Variation across states in Outcome 1: Part B 619 2010-11

Variation across states in Outcome 2: Part B 619 2010-11

Variation across states in Outcome 3: Part B 619 2010-11

Variation across states in Outcome 1: Part C 2010-11

Variation across states in Outcome 2: Part C 2010-11

Variation across states in Outcome 3: Part C 2010-11