Transcript ECO Longitudinal - OSEP Leadership Mtng

Using Child Outcomes Data
Integrating Outcomes Learning Community
June 12, 2013
1
Using Data for Improvement
• Linking outcomes data to other data
• Helping locals look at data
• Examining practices and how they relate to
outcomes
2
Linking Outcomes Data
to other Data
• Linking child outcomes data to...
•
Child/Family characteristics, e.g.
•
•
Service data, e.g.
•
•
Race/ethnicity, income, disability type
Amount of services, type of services provided
Program data
•
Geographic location or specific program characteristics
What kinds of questions might these answer to
inform program improvement efforts?
3
Child/Family Characteristics
% Children that Exited the Program Within Age
Expectations in Social Relationships
by Income Categories
64
62
• In your state, what
child/family
characteristics data are
you linking to your
child outcomes data?
60
58
• What have you
learned?
56
54
52
50
low income
medium income
high income
• What could you
potentially link?
4
Child/Family Characteristics
% Children that Showed Greater than Expected Growth
in Knowlede and Skills by Racial Categories
80
70
60
• In your state, what
child/family
characteristics data are
you linking to your
child outcomes data?
50
• What have you
learned?
40
30
20
10
0
White
Black/African
American
American Indian
Asian
Hawaiian/Other
Pacific Islander
• What could you
potentially link?
5
Service Data
% Children that Showed Greater than Expected Growth
(SS1), by Amount of Services
95
• In your state, what
service data are you
linking to your child
outcomes data?
85
• What have you
learned?
75
65
55
• What could you
potentially link?
45
35
25
<8 hrs
8-19 hrs
20+ hrs
6
Program Data
% Children that Exited the program within age
expectations (SS2) in Knowledge and Skills, by Program
Quality
70
• In your state, what
program data are you
linking to your child
outcomes data?
60
• What have you
learned?
50
40
30
• What could you
potentially link?
20
10
0
low quality
medium quality
high quality
7
• What other data are you linking to your
child outcomes data?
• What are your data telling you?
8
Helping Local Programs
Look at Child Outcomes Data
• Examples of states...
• What efforts/initiatives are going on in your
state to help local programs look at their child
outcomes data?
9
Tools linking outcomes and practices
Local Contributing Factors Tool
Relationship of Quality Practices to
Child and Family Outcome
Measurement Results
10
Local Contributing Factors Tool
• What do you like about the tool?
• What do you think is useful about the tool?
• How might you use the tool in your state?
11
Relationships
• What do you like about the tool?
• What do you think is useful about the tool?
• How might you use the tool in your state?
12
REMEMBER!
• You want to trust your data
BEFORE
• You draw conclusions and take action
based on that data!
13
High Quality Child Outcomes Data
Looking at statewide data quality:
• Completeness of the data (% of children in
the data)
• Patterns in the data
14
Completeness of the Data
• Denominator
• Children exiting in the reporting period,
with at least 6 months of service
• Numerator
• Of the children in the denominator, the
children with entry and exit outcomes data
GOAL: 100%
What % is your state currently reporting?
15
Completeness of the Data
ECO National Analysis (proxy)
• Part C: outcomes data available on at least
40% of children who exit Part C
Annually, more and more states are meeting
the standard.
16
Looking for patterns in the data
ECO National Analysis
• Odd patterns in the data
• E.g.
•
•
State % in ‘a’ is not overly high (> 5%)
State % in ‘e’ is not overly low (< 5%) or high
(> 65%)
Annually, more and more states are meeting
the standard.
17
Looking for patterns in the data
What might be other ‘odd’ patterns in the a-e
data?
•
•
•
State % in ‘b’ is not overly low (< 5%) or high
(> 50%)
State % in ‘c’ is not overly low (< 5%) or high
(> 50%)
State % in ‘d’ is not overly low (< 5%) or high
(> 50%)
Does your state have any ‘odd’ patterns in
the data?
18
Looking for patterns in the data
States’ capacity to do more pattern checking
than nationally available
•
E.g.
•
•
•
•
•
•
By entry scores
By exit scores
By disability
By geography
By age at entry
Other
19
Looking for patterns in the data
The quality of the child outcomes data are
established by a series of analyses that
demonstrate the data are showing predictable
patterns.
Pattern Checking Table
http://www.fpg.unc.edu/~eco/assets/pdfs/
Pattern_Checking_Table.pdf
20
High Quality Child Outcomes Data
Looking at LOCAL data:
• Completeness of the data (% of children in
the data)
• Patterns in the data
21
Data Quality Efforts?
• What are you doing related to checking data
quality?
• What strategies are you planning to use to
ensure data quality over time?
• How could integrating outcomes into the
IFSP or IEP help with data quality efforts?
22
Improving Data Quality
• Training - e.g. initial/orientation, refresher,
regular opportunities
•
ECO resources –
•
•
professional development
self-directed learning
• TA – e.g. ongoing support, mentoring,
supervision with feedback
• Guidance materials – e.g. manuals, early
learning guidelines
23
Improving Data Quality
• Data analysis and use – e.g. pattern checking at
state and local levels
•
ECO resources
•
•
•
Pattern checking document (and new one coming!)
PD – “Looking at Data”
Calculators and Graphing templates
24
How is my state’s data with regard
to...
• Completeness of the outcomes data?
•
•
•
•
•
Do we have the ability to identify the true
denominator for this calculation?
What % of exiters with 6 mo service have child
outcomes data?
Are we looking at ‘missing data’ issues for local
programs/districts?
What, if anything, do we need to work on?
What strategies have we used to reduce missing
data?
25
How is my state’s data with regard
to...
• Data patterns for a-e?
•
•
•
•
•
•
Do we have ‘a’ greater than 5%?
Do we have ‘e’ greater than 65%?
What other ‘odd’ patterns in the data have we
examined?
Are we looking at ‘data patterns’ for local
programs/districts?
What, if anything, do we need to work on in my
state?
What strategies have we used to identify and
investigate odd patterns in the data?
26
Additional Questions/Comments?
27