Transcript Slide 1

Idaho’s
Early Childhood Outcomes
System
(Idaho ECOS)
Measuring Early Childhood Outcomes
Idaho Infant Toddler Program
Idaho Early Childhood Special Education Program
1
Agenda
• Measuring Child Outcomes:
– Why, What, When, Where,
– Who & How
• Review Child Outcome Summary Form,
Tools & Resources
• Practice Exercises
•
2
Federal Accountability
and
Early Childhood Outcomes
U.S. Congress
Office of Budget and
Management
U.S. Department of Education - Office of Special
Education Programs
State Education
Agencies and
State Part C
Agencies
Local Education
Agencies and
Early Intervention
Programs
3
Training Objectives
Participants will be able to :
• Demonstrate the ability to accurately
identify, record, and report a child’s
outcome data using the Idaho Child
Outcome Summary Form (Idaho
COSF)
4
Idaho’s goal
Document our program’s
impact for children
with IFSPs or IEPs
5
Measuring outcomes make
sense for Idaho
– Support DEC/NAEYC best practice
guidelines
– Examine and refine current assessment
practices
– Demonstrate value of
• Early Intervention (Part C),
• Early Childhood Special Ed (Part B) programs
– Meet Federal Reporting Requirements
6
What are the
OSEP’s Early
Childhood Outcomes?
7
OSEP’s outcomes focus on
measuring the results of
our service system
8
OSEP’s 3 Child Outcomes
1. Children have positive socialemotional skills (including positive
social relationships)
2. Children acquire and use knowledge
and skills (including early language
/communication)
3. Children use appropriate behaviors to
meet their needs
9
OSEP’s 3 Child Outcomes
1. Children have positive socialemotional skills (including
positive social relationships)
• Such as: responding to others,
expressing emotions, turn-taking,
using appropriate social greetings,
play skills or having friendships with
same-age peers, etc.
10
OSEP’s 3 Child Outcomes
2. Children acquire and use
knowledge and skills
(including early language
/communication)
• Such as: expressing thoughts and
ideas, listening to and enjoying stories
and books, or learning new ways to do
things, etc.
11
OSEP’s 3 Child Outcomes
3. Children use appropriate
behaviors to meet their needs
• Such as getting from place to place,
using tools like forks or crayon, and
feeding or dressing, etc.
12
The Child Outcome Areas…
• Represent critical functional outcomes
necessary in every day activities and routines
• Not domains based – not trying to separate
child development into discrete areas
(communication, gross motor, etc.)
• Emphasize how the child is able to
integrate across domains to carry out complex
meaningful behaviors
13
Outcomes are Functional
• Meaningful to the child in the context of
everyday living
• Integrated series of behaviors or skills
that allow the child to achieve the
outcomes.
• Not
– a single behavior
– the sum of a series of discrete behaviors
14
Functional Outcomes
• What does a child typically
do?
• Actual performance across
settings and situations
• How child uses his/her skills
to accomplish tasks
• Not the child’s capacity to
function under ideal
circumstances
15
So…how do we
demonstrate that our
programs produce good
outcomes for children?…
16
Demonstrate that…
• Children have positive outcomes
given who they are, their delays,
disabilities, functioning when they
entered, etc.
• Outcomes are better than they would
have been without the program
17
Complicating Issues
• Many children progress with no
intervention (maturing with age)
• Children with disabilities’ show
diverse progress and under the best
of programs, will experience different
outcomes
18
Solution
Document the number of
children for whom the
program has changed their
developmental
trajectories
19
Thinking about how children are doing
with regard to each outcome.
Movement away
from age-expected
Age-expected
skills & behavior
Movement toward
age-expected
20
OSEP Indicators –
categories of progress
• a. % of children who maintain
functioning at a level comparable to sameage peers
• b. % of children who reach functioning at
a level comparable to same-age peers
• c. % of children who improved
functioning but did not achieve functioning
comparable to same-age peers
• d. % of children who did not improve
functioning.
21
OSEP Indicators &
Measurement Categories
5 year old
level
Group a:
maintained or
reached typical
2 year
old level
Entry
Exit
22
Indicators & Measurement
Categories
5 year old
level
Group b: made
progress but…
2 year
old level
Entry
Exit
23
Indicators & Measurement
Categories
5 year old
level
Group c: did
not make
progress
2 year
old level
Entry
Exit
24
Indicators & Measurement
Categories
Group a:
maintained or
reached typical
5 year old
level
Group b: made
progress but..
Group c: did
not make
progress
2 year
old level
Enry
Exit
25
How do we document change in developmental
trajectories while in services ?
Obtain and compare a child’s
assessment data
at program
entry and exit.
26
Quality
Assessment

Quality
Service

Better
Outcomes
27
“Much of developmental psychology
(early childhood testing) as it now
exists is the science of the
strange behavior of children with
strange adults in strange settings
for the briefest possible periods
of time.”
(Bronfenbrenner, 1979)
28
What is assessment?
“Early childhood assessment is flexible,
collaborative decision-making process in
which teams of parents and
professionals repeatedly revise their
judgments and reach consensus about
the changing developmental,
educational, medical, and mental health
services needs of young children and
their families.”
Bagnato and Neisworth, 1991
Quoted in DEC Recommended Practices, 2005
29
DEC Recommended Practices
for Assessment
• Involves multiple sources (e.g., families,
professional team members, service
providers, caregivers)
• Involves multiple measures (e.g.,
observations, criterion-curriculum-based
instruments, interviews, curriculumcompatible norm-referenced scales, informed
clinical opinion, work samples)
30
What information is
gathered?
Norm-referenced, criterionreferenced or curriculum-based
assessments
Informed
Professional
Judgment &
Observation
Multiple
sources of data
are used to rate
a child’s functioning
Parent Input
31
Which anchor assessment
tools will we use in Idaho?
The following tools have been adopted for completing the
outcome rating scale.
•
•
•
•
•
•
•
•
•
•
BDI – II (Battelle)
HELP (Hawaii)
Creative Curriculum
AEPS
Carolina
OUNCE
Work Sampling
High Scope (COR)
Brigance
Bayley III (ITP program only)
32
Information to gather…
Informed professional judgment
–
–
Teachers, paraprofessionals, related
service providers
Anecdotal records, documented
observations and data, progress reports,
work samples, portfolios
Parental input
–
–
Best practice
Information from caregivers provide
critical information to determine how
child is doing across a variety of settings
33
So…our challenge
How do we take what
we know about
assessment and
apply it to
measuring the 3
outcomes…
…when there is no one
assessment tool that
assesses all 3
outcomes directly.
34
And through our assessment
lens…
• Each child is a collection of
numerous behaviors, skills,
traits, capabilities, interests,
strengths, and needs
• What an individual “tunes
into” depends on his/her
orientation
• Individual professional
training has provided an
“organizing framework” for
how one sees the child
35
How do assessment tools fit with
the three Child Outcome Areas?
• CROSSWALKS CAN HELP!
• A national center, the ECO Center is crosswalking the
most common assessments to the 3 child outcomes
• Crosswalks give a visual indication of how items on an
assessment tool covers the 3 outcomes
• Crosswalks show which areas/subareas map to which
outcome
36
Sample Crosswalk
The High/Scope Preschool Child Observation Record (2003):
Crosswalk to Child Outcomes
Outcome 1:
Positive social relationships
I. Initiative
C. Initiating play [social context of play]
II. Social Relations
E. Relating to adults
F. Relating to other children
G. Resolving interpersonal conflict
H. Understanding and expressing feelings
V. Language & Literacy
Q. Listening to and understanding speech
Outcome 2:
Knowledge and skills
III.
I.
J.
K.
Creative Representation
Making and building models
Drawing and painting pictures
Pretending
V.
R.
S.
T.
U.
V.
W.
X.
Language & Literacy
Using vocabulary
Using complex patterns of speech
Showing awareness of sounds in words
Demonstrating knowledge about books
Using letter names and sounds
Reading
Writing
Outcome 3:
Action to meet needs
I. Initiative
A. Making choices and plans
B. Solving problems with materials
D. Taking care of personal needs
VI. Mathematics & Science
Y. Sorting objects
Z. Identifying patterns
AA. Comparing properties
BB. Counting
CC. Identifying position and direction
DD. Identifying sequence, change, and
causality
EE. Identifying materials and properties
FF. Identifying natural and living things
Note: Areas that are not precursor to or components of any of the three outcomes, and therefore not included in the crosswalk, were:
IV. Movement & Music:
L. Moving in various ways
M. Moving with objects
N. Feeling and expressing steady beat
O. Moving to music
P. Singing
37
What will the process look like
in Idaho?....
• Parents will be well informed and contribute
information to the process
• An anchor assessment will be completed to assess
the child in the three outcome areas
• Information will be collected, compiled and
documented using anchor assessments, parent
information, and informed professional observation
and judgment.
• A rating decision regarding a child’s level of
functioning will be made by the team utilizing all the
information gathered
38
The Specifics:
Who, What,
and
When
39
Target Population for Entry
(Part C)
• Entry baseline data is required for:
– all children entering services on or after
July 1, 2006 will have COSF completed
within 45 days of IFSP development.
– except premature infants less than 6
months adjusted age
• Includes children receiving only a
single related-service (i.e. Speech
only or OT only)
40
Target Population for Exit
(Part C)
• Exit data is required for all children in services for
at least 6-months as of January 1, 2007.
• Unanticipated exits– use best information
available to complete form following exit.
• Assessment (with anchor tool) recommended for
all exiting children
• If necessary, those receiving only one relatedservice can use ASQ and ASQ-SE in lieu of full
developmental evaluation as one data source in
determining exit outcome rating
41
Time Lines at Exit (Part C)
• COSF completed near exit or transition
meeting with Part B
• Anchor assessment completed between
2.6 IFSP meeting and child’s 3rd birthday
• Outcome data is due to Data-Tot system
(and if appropriate, Part B system):
– no later than 30 days after child’s exit from
Part C, or
– no later than 30 days after child’s 3rd birthday
(whichever comes first).
42
Additional information about
child indicators (Part C)
•Child must be in program at least 6
months for EXIT data to be counted
•Data must be collected near entry and
near exit
•Collection of Entry data begins:
•for ITP July 1, 2006
•for SDE September 1, 2006
43
Roles and Responsibilities
(Part C)
• Child information compiled from multiple
sources – Service Coordinator
• Scoring/completion of COS Form –
Primary Therapist or Multi-disciplinary
Team
• Outcome Ratings transferred to Data-Tot
Entry form – Service Coordinator
• Data entered into Data-Tot – Data Entry
operator
44
Difference of Opinion?
(Part C)
• If therapists or other team members
can not agree on a rating…
– Review data at MDT, seek consensus
– If no resolution, provide Child Team
supervisor with all data
– Child Team Supervisor will assign final
rating
– Note different perspectives on form
45
Summary Steps in COS
Process (Part C)
• Gather information from multiple sources
(complete anchor assessment if
necessary)
• Synthesize data and complete COS Form
• Update Data-Tot enrollment form and
submit for entry
• Share data with Part B if appropriate
46
47
Data Collection Process for
Part B
Target populations and Entry data
• All children enrolled in early
childhood special education on
September 1, 2006 or later must
have entry data (baseline) collected
within 45 calendar days of initial
consent for placement.
• Children transitioning from Part C Exit from Part C may be used for
Part B entry if an approved anchor
48
tool was administered
Exit data will be collected
Part B
• EC Outcome Exit data will be collected
within 30 days of the end of the
school year (including ESY) for all
children who are 5 years old on or before
Sept 1st of the current school year (cutoff date for kindergarten entry) no matter
if they are staying in a preschool setting
or going into kindergarten.
• If moves to another preschool, transfer
outcome entry data and information with
eligibility report and student file.
49
Exit data will be collected
Part B (cont’d)
• If determined no longer eligible before 5,
exit outcome data is collected on all three
outcomes within 30 days of no longer
meeting eligibility criteria.
• Prior to a move out of state, collect data
30 days prior to leaving the program
• If child leaves before 5 without notice and
prior to administering the anchor
assessment, complete the COSF with the
information you have.
50
Roles (Part B)
• Preschool teachers have the lead role
to gather the necessary information
for the COSF, preferably during team
meetings.
• Related service providers also
provide assessment data,
observation, etc.
51
Determining the score
1-7 (Part B)
• This should be a team decision – by
consensus to insure validity and
reliability.
• Scores may be averaged
• If there is disagreement, an
administrator should make the final
determination after reviewing the
COSF.
52
Process (Part B)
• Gather all assessment information
• Complete the COSF – may be
incorporated at time of IEP meeting.
• Consent is not required for this
process.
Enter score 1-7 & (Y or N for exit) in
web-based data system
53
Exit Data Collection (Part B)
All children that have been in the
program for 6 months or more must
have exit data collected.
54
Data reporting to the state
(Part B)
• A real-time data collection system is
being constructed so data can be
entered at the convenience of the
preschool teacher to avoid lost data.
• Data should be entered as soon as
possible.
55
The Child Outcome
Summary Form
(COSF)
56
Child Outcome Summary Form
Key Features
• Is NOT an Assessment
• Is NOT about eligibility determination
• Does NOT rate or summarize
– Info about services provided to child
– Family’s satisfaction with services
– Information for planning services for child
• Blends diverse data from multiple
sources into a consistent, reliable unit
of measurement that can be
aggregated and tracked over time
57
Child Outcome Summary Form
Key Features
• Uses diverse information for global view of
a child
• Compares a child’s functioning to sameaged peers
• Documents progress and movement
toward typical development
• Considers functional abilities across
diverse settings (not under ideal
circumstances)
• Provides information about child’s overall
sense of functioning in the three outcome
58
areas
The Child Outcome Summary
Form (COSF)
– Cover sheet
• Summary of evidence used to complete
ratings
• People involved in assigning rating
– 7-point rating , with an overall rating for
each required outcome area
• Highest score (7) = outcome achieved at ageexpected level
• Lowest score (1) = farthest distance from
age-expectations
– Progress Question (to be completed only
at exit)
59
60
Thinking about how children are doing
with regard to each outcome.
Movement away
from age-expected
Age-expected
7
skills & behavior
Movement toward
age-expected
61
Outcome Ratings:
A 7-Point Scale
1
2
3
4
5
6
7
- Not Yet
- Between emerging and not yet
- Emerging
- Between somewhat and emerging
- Somewhat
- Between completely and somewhat
-Completely
62
63
Considerations when completing
Child Outcome Summary Form
– Team’s input is required
– Consider role of assistive
technology/accommodations (i.e. sign language,
wheel chair)
– Consider child’s culture and language
– Document special considerations that impacted
child’s development
–
Reflect the child’s actual functioning – not what
the child is capable of under the ideal or highly
unusual circumstances
64
Considerations when completing
Child Outcome Summary Form
• OSEP reporting requires two data
points for each outcome
– Near entry (sets baseline)
– Near exit (shows progress)
• At Exit only, answer question “Has
child shown new skills or made
progress in outcome area?”
• Child CAN make progress without
“changing numbers” on the scale
65
Identifying who made
progress
• Progress = moving up a point on the
scale in a subsequent rating, e.g., 3 to 4
• Progress= staying at the same rating
but having a “yes” for the progress
question
66
Parental Involvement
• Parental consent for evaluations is
required, as always.
• Parents give input to the COSF through
assessment process and IFSP/IEP
development.
• Parents need not be present when COSF is
completed.
• Completion of the COSF is NOT an
assessment and does not require parental
consent.
• Discuss/share Parent’s Handout to inform 67
parents about process and purpose.
What Did We Cover?
•
•
•
•
•
Background and History
OSEP’s Three Functional Outcomes
Assessment Practices
Specific Requirements for Part C & Part B
Tools (Anchor Assessment List,
Crosswalks, COSF, Instructions, Decision
Tree, Parent Talking points)
• Practice
• Local Planning
70
Good outcome
data is a tool to
produce good
outcomes for
children and
families.
72