Transcript Slide 1

Idaho’s
Early Childhood Outcomes System
(Idaho ECOS)
Updated
Measuring Early Childhood Outcomes
Idaho Early Childhood Special Education
Fall 2009
1
Agenda

Measuring Early Childhood Outcomes:
 Why, What, When, Who & How?

Review Child Outcome Summary Form,
(COSF-R) Tools & Resources

Reporting the data on the Idaho early
Childhood Outcome Collection System

Practice Exercises

2
Training Objectives
Participants will :
 Learn how to accurately identify,
record, and report a child’s outcome
data using the Idaho Child Outcome
Summary Form-Revised (Idaho
COSF-R)
 Increase valid and reliable data
reporting on the ECO’s – timely and
accurate
3
Goal of Early Childhood Special
Education
“…To enable young children to be active
and successful participants and learners
during the early childhood years in a
variety of settings – in their homes with
their families, in child care, in preschool
or school programs, and in the
community.”
Early Childhood Outcome Center
4
Why Collect Outcome Data?


Federal government is the driving force
behind the move to collect outcome
data, however it is not the only reason…
Data on outcomes are also valuable
To document program effectiveness
To improve district programs
To ultimately better serve children
5
WHY collecting outcome data
makes sense for Idaho





Improve current assessment practices and
program development
Promote progress toward age-expected skills
and behavior throughout the child’s day
Demonstrate value of Early Childhood Special
Ed (Part B) programs
Support DEC/NAEYC best practice guidelines
Meet Federal Reporting Requirements -Office
of Special Education Programs (OSEP)
6
OSEP’s Solution
Document the number of
children for whom the
program has changed their
developmental
trajectories
7
Thinking about how children are doing with
regard to each outcome.
Movement away
from age-expected
Age-expected
skills & behavior
Movement toward
age-expected
8
OSEP Indicators –
categories of progress

Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to
move nearer to functioning comparable to
same age peers
c. Improved functioning to a level nearer to
same-age peers but did not reach it
d. Improved functioning to reach a level
comparable to same aged-peers
e. Maintained functioning at a level
comparable to same-aged peers
9
OSEP Indicators & Measurement
Categories
maintained or
reached typical
5 year old
level
made progress
but..
did not make
progress
2 year
old level
Enry
Exit
10
How do we document change in
developmental trajectories while in services ?
Obtain and compare a child’s
assessment data
at program
entry and exit.
11
What happens with this data and why is
accuracy and timeliness is so important?



Individual student data is aggregated at state
level
States are required to report outcome data to
Feds according to % of children exiting ECSE
services –
State must set targets and report progress
toward meeting targets:


% who entered program below age expectations in
each outcome and substantially increased their rate
of growth
% who were functioning within age expectations in
each outcome by time they exited
So, who do we collect outcome
data on and when…


All children enrolled in ECSE must have EC
Outcome Entry data collected within 45
calendar days of initial consent for
placement/IEP start date (if they will be in
services for at least 6 months).
For children transitioning from Part C –
Part C outcome exit data may be used for
Part B entry data, if an approved anchor tool
was administered.
13
Who and when ….
•
EC Outcome Exit data will be collected within
30 days of the end of the school year
(including ESY) for all children who:
•
Have been in Idaho receiving services for at
least 6 months, and
•
Are 5 years old on or before Sept 1st of the
next school year (cut-off date for
kindergarten entry), no matter if they are
staying in a preschool setting or going into
kindergarten.
14
EC Outcome Exit data must also
be collected…


Within 30 days of determining a child no
longer meets eligibility criteria (prior to age 5
and been in program at least 6 months)
30 days prior to a child moving out of state
 If child leaves before 5 without notice and
prior to administering the anchor
assessment, complete the COSF-R with
the information available.
15
Who is responsible to collect
outcome data?


Preschool teachers/SLP’s should take a
lead role in gathering the necessary
data/information from multiple people and
measures through a team process
Related service providers also provide
assessment data, observation, etc.
16
What are the
OSEP’s Early Childhood
Outcomes?
17
OSEP’s 3 Child Outcomes
1.
Children have positive socialemotional skills (including social
relationships)
2.
Children acquire and use knowledge
and skills (including communication
and early literacy)
3.
Children use appropriate behaviors to
meet their needs
18
Outcome 1
1. Children have positive social-emotional
skills (including positive social
relationships)
•
Relating with adults, and other children
•
For older children: following rules related to
groups or interacting with others
•
Attachment/separation/autonomy
•
Responding to others, expressing emotions,
turn-taking, using appropriate social
greetings, play skills or having friendships
with same-age peers, etc.
19
Outcome 2
2. Children acquire and use knowledge and
skills (including communication and early
literacy)
•
Using symbols and language
•
Thinking, reasoning, remembering, problemsolving
•
Understanding physical and social worlds
•
Early concepts – symbols, pictures, numbers,
classification, spatial relationships
•
Imitation, object permanence
•
Expressing thoughts and ideas, listening to
and enjoying stories and books, or learning
new ways to do things, etc.
20
Outcome 3
3. Children use appropriate behaviors to
meet their needs
•
Taking care of basic needs
•
Getting from place to place
•
Using tools like fork, toothbrush, crayon
•
•
•
In older children, contributing to their own
health and safety
Self help skills (dressing, feeding, toileting,
grooming, household responsibility)
Acting on world to get what he needs
21
The Child Outcome Areas…
Represent
critical functional outcomes
necessary and meaningful in every day
activities and routines
Not domains based – not trying to
separate child development into discrete
areas (communication, gross motor, etc.)
and single behaviors

Emphasize how the child is able to
integrate series of behaviors or skills
across domains to achieve outcomes

22
Functional Outcomes





What does a child typically do
at this age level?
Actual performance across
settings and situations
How child uses his/her skills
to accomplish tasks
Not the child’s capacity to
function under ideal
circumstances
Not necessarily the child’s
performance in a structured
testing situation:" noncompliant”
23
So…how do we
demonstrate that our
programs produce good
outcomes for children?…
24
Data will demonstrate that…


Children have positive outcomes
regardless of, their delays,
disabilities, and functioning when
they entered ECSE services.
Individual child outcomes are better
than they would have been without
ECSE services.
25
Complicating Issues



Many children progress with no
intervention (maturing with age).
Children with disabilities’ show
diverse progress and under the
best of programs, will experience
different outcomes.
Children receive different amounts,
types and quality of services.
26
Quality
Assessment

Quality
Service

Better
Outcomes
27
“Much of developmental
psychology (early childhood
testing) as it now exists is the
science of the strange behavior
of children with strange adults
in strange settings for the
briefest possible periods of
time.”
(Bronfenbrenner, 1979)
28
What is assessment?
“Early childhood assessment is flexible,
collaborative decision-making
process in which teams of parents
and professionals repeatedly revise
their judgments and reach
consensus about the changing
developmental, educational, medical,
and mental health services needs of
young children and their families.”
Bagnato and Neisworth, 1991
Quoted in DEC Recommended Practices, 2005
29
DEC Recommended Practices for
Assessment


Involves multiple sources (e.g.,
families, professional team members,
service providers, caregivers)
Involves multiple measures (e.g.,
observations, criterion-curriculum-based
instruments, interviews, curriculumcompatible norm-referenced scales,
informed clinical opinion, work samples)
30
Uses and limitations of assessment
tools for measuring child outcomes…
•
•
•
•
•
Used to compare a child’s skills and behaviors
to those of his same-age peers.
Most are organized around domains.
Not designed for direct measurement of the
three outcomes.
Some call for standardized assessment items to
be administered in a setting other than the
child’s natural environment, making it
difficult to use the information to determine
whether a child uses this skill in everyday life.
Teams may have different information that
differs, thus “overriding” assessment tool
results.
31
Critical Assumptions Related to the
Three Child Outcomes



Measuring achievement of outcomes is
based on comparison with age
expectations
Children of different ages will
demonstrate achievement in
different ways, so expectations
change with age
There are many pathways to
functioning for children with atypical
development (using sign language,
wheelchair, etc.)
What information is gathered?
Anchor AssessmentNorm-referenced, criterion-referenced or
curriculum-based assessments
Informed
Professional
Judgment &
Observation
Multiple
sources of data
are used to rate
a child’s functioning
Parent Input
33
Which anchor assessment tools
will we use in Idaho?
The following tools have been adopted for completing the outcome rating scale.











BDI – II (Battelle)
HELP (Hawaii)
Creative Curriculum
AEPS
Carolina
OUNCE
Work Sampling
High Scope (COR)
Brigance
Developmental Assessment of Young Children
(DAYC)
Bayley III (ITP program only)
34
Information to gather…
Informed professional judgment


Teachers, paraprofessionals, related
service providers
Anecdotal records, documented
observations and data, progress
reports, work samples, portfolios
Parental/Caregiver input


Best practice
Provides critical information about
how a child functions across the day
and in a variety of settings
35
So, our challenge is…
How do we take
what we know
about
assessment
and apply it to
measuring the
3 outcomes…
…when there is no
single
assessment tool
that assesses all
3 outcomes
directly.
36
“Crosswalks” were developed by
National ECO Center to…


Show how the skills assessed by most
commonly used published instruments
map to the three outcomes.
Provide comparisons across instruments
to see their various strengths and
weaknesses with regard to the three
outcomes.
37
Early Childhood Outcomes Center
38
.
So, how do we put it all
together…
39
Child Outcomes Summary Form
(COSF-R)



Produces and compares data from
programs across the state that are using
different assessment tools
A 7-point scale for summarizing
information related to a child’s progress
on each of the three child outcome areas
Rating based on child’s functioning:
 Across settings and situations
 Compared with what skills are expected
given the child’s age
40
Child Outcome Summary Form
Key Features



Is NOT an Assessment
Is NOT about eligibility determination
Does NOT rate or summarize




Info about services provided to child
Family’s satisfaction with services
Information for planning services for child
Blends diverse data from multiple
sources into a consistent, reliable
unit of measurement that can be
aggregated and tracked over time
41
Child Outcome Summary Form
Key Features





Uses diverse information for global view
of a child
Compares a child’s functioning to sameaged peers
Documents progress and movement
toward typical development
Considers functional abilities across
diverse settings (not under ideal
circumstances)
Provides information about child’s overall
sense of functioning in the three
outcome areas
42
COSF Key Points





Assumption: Children can be described with regard
to how close they are to age-expected functioning for
each of the three outcomes
By definition, most children in the general
population demonstrate the outcome in an ageexpected way
Over time, some children will move farther away
from age-expected functioning (skills at older ages
are more demanding)
By providing services and supports, programs
are trying to move children closer to age-expected
functioning
Some children will never achieve this
Essential Knowledge for team
members completing the COSF
Know about the child’s functioning across
settings and situations
Understand age-expected child development
Understand the content of the three child
outcomes
Know how to use the rating scale
Understand age expectations for child
functioning within the child’s culture
Early Childhood Outcomes Center
44
Child Outcomes Summary Form
Early Childhood Outcomes
Center
45
The Form

Cover page + three outcome pages

On each outcome page:


Two questions per outcome – one at
entry and two at exit
Space to document the summary of
relevant results (and examples of ageappropriate, immediate foundational
and/or foundational skills), which will be
the basis for the rating
Early Childhood Outcomes
Center
46
The Two COSF Questions
for each outcome
a. To what extent does this child show
behaviors and skills related to this
outcome appropriate for his/her age
across a variety of settings and
situations? (Rating: 1-7)
b. Has the child shown any new skills or
behaviors related to [this outcome]
since the last outcomes summary?
(Yes-No)
Early Childhood Outcomes
Center
47
Summary Ratings Reflect
Global Functioning


Ratings on each outcome are a snapshot of:
 The whole child
 Status of the child’s current functioning
based on the various types of
information from a variety of people
who are familiar with the child across a
variety of typical settings and situations
 Team consensus to ensure validity and
reliability
Rather than reflective of :
 Skill by skill, in one standardized way, or
split by domains
Early Childhood Outcomes
Center
48
The Basis for the Ratings



Scale runs from 1-7 with a 6 or 7 indicating
age appropriate functioning for the outcome
Lower numbers indicate distance from age
appropriate functioning for the outcome
Resources – “Definitions for Outcome
Ratings” and “Decision Tree for Summary
Rating Discussions”
Early Childhood Outcomes
Center
49
Important Terms…


Foundational skills are….
Skills and behaviors that occur earlier in
development and serve as the foundation for
later skill development
Teachers and interventionists often use
foundational skills to help children move to
the next level developmentally
Immediate foundational skills are…
Skills that are conceptually linked to later skills
and immediately precede the later skills
developmentally
Example: Children play alongside one another
before they interact in play
7 – Completely



The child shows behaviors and skills expected
in all or almost all everyday situations that
are part of the child’s life
 Home, store, park, child care, with
strangers, etc.
The child’s functioning is considered
appropriate for his/her age
No one has significant concerns about the
child’s functioning in this outcome area
Early Childhood Outcomes
Center
51
6 – Between Completely
and Somewhat
•
The child’s functioning generally is
considered appropriate for his or her
age, but there are some significant
concerns about the child’s functioning in
this outcome area
•
May border on not keeping pace with age
expectations so monitoring and extra
support may be needed
Early Childhood Outcomes
Center
52
5 – Somewhat



The child shows functioning expected for
his/her age some of the time and/or in
some situations
The child’s functioning is a mix of ageappropriate and not appropriate functioning
The child’s functioning might be described as
like that of a slightly younger child
Early Childhood Outcomes
Center
53
4 – Between a 5 and a 3


Child shows some age appropriate
functioning some of the time or in some
situations or settings but most of the child’s
functioning would be described as not yet
age appropriate
The child’s functioning might be described as
like that of a younger child
Early Childhood Outcomes
Center
54
3 – Emerging



The child does not yet show functioning
expected of a child his/her age in any
situation
The child’s behaviors and skills include
immediate foundational skills most or
all the time, on which to build ageappropriate functioning
The child’s functioning might be described
as like that of a younger child
Early Childhood Outcomes
Center
55
2 – Between 3 and 1



The child does not yet show functioning
expected of a child his/her age in any
situation
The child’s behaviors and skills do have
some immediate foundational skills on
which to build age-appropriate functioning
but these are not displayed very often
More functioning reflects skills that are not
immediate foundational
Early Childhood Outcomes
Center
56
1 – Not Yet




The child does not yet show functioning expected of a
child his/her age in any situation
The child’s skills and behaviors also do not yet include
any immediate foundational skills on which to build
age-appropriate functioning
The child’s functioning might be described as like that of
a much younger child
Children with 1 ratings still have skills, just not yet at an
immediate foundational level
Early Childhood Outcomes
Center
57
“Concerns” That Distinguish
Ratings of 7 vs. 6


All children have strengths and
weaknesses. Families and providers identify
areas to work on to support ongoing
growth (some call these “concerns”)
What types of concerns would result in a
rating of 6?
Early Childhood Outcomes
Center
58
“Concerns” That Distinguish
Ratings of 7 vs. 6


Concerns raised for which families and providers may
want to offer extra support and strategies to promote
development, but the area of concern is not a
possible indicator or precursor of a significant
developmental problem (7)
vs.
Developmental concerns—weaknesses significant
enough to watch closely and definitely support.
Although age expected now, the child’s development
borders on not keeping pace with age-expected
levels or shows early signs of possible developmental
problems (6)
Early Childhood Outcomes
Center
59
Where to Focus in Deciding
the Rating


Focus on the child’s overall functioning
across settings and situations
Functioning that is displayed rarely and/or
when the child is provided with a lot of
unusual support or prompts is of little
significance for the rating
Early Childhood Outcomes
Center
60
The Process for Answering
Questions 1a, 2a, 3a
For each outcome…
1. Discuss the child’s current functioning in
this outcome area across settings and
situations
2. Identify areas where the child’s functioning is
age appropriate
3. If not all functioning is age appropriate,
identify areas where the child’s functioning
reflects immediate foundational skills
4.Decide which rating best describes the child’s
current functioning
Early Childhood Outcomes Center
61
Answering the Questions at Exit:
1b, 2b, & 3b




At Exit only, answer question “Has child
shown new skills or made progress in
outcome area?” (Yes or No)
Small steps of progress count!
Child CAN make progress without
“changing numbers” on the scale!
Most will select “Yes”
Identifying who made progress


Progress = moving up a point on the
scale in a subsequent rating, e.g., 3 to 4
Progress= staying at the same rating
but having a “yes” for the progress
question
63
Important considerations…
•
•
•
•
•
How to get team input
Role of assistive technology and
accommodations (i.e. sign language,
wheel chair)
Child’s culture and language
Special considerations that impacted
child’s development
Reflect the child’s actual functioning –
not what the child is capable of under
the ideal or highly unusual
circumstances
64
Parental Involvement





Parental consent for evaluations is required,
as always.
Completion of the COSF is NOT an
assessment and does not require parental
consent.
Parents give input to the COSF through
assessment process and IFSP/IEP
development.
Parents need not be present when COSF is
completed.
Inform parents about process and purpose.
65
What will the process look like in
Idaho?....






Team members, including the parents will be
well informed and contribute information to the
process
An anchor assessment will be completed to
assess the child in the three outcome areas
Information (anchor assessments, parent input,
and informed professional observation and
judgment) be collected, compiled and
documented using the COSF-R
Team will make a rating decision (1-7 and Y or
N for exit) regarding a child’s level of functioning
Data will be entered on the web-based data
system
Completed COSF-R from entry or exit will be
placed in child’s file
66
Early Childhood Outcome Data
Collection System

https://www.sde.idaho.gov/ECOutcome/default.aspx

http://itcnew.idahotc.com/dnn/EarlyChildhood/tabid/10
92/Default.aspx

Web-based Data Collection System
Instructions and Technical Information
https://www.sde.idaho.gov/ECOutcome/docs/
ECOutcomeInstructions.pdf


Districts are required to enter student data on the
website in a timely manner!
67
Practice activities…
Good outcome
data is a tool to
produce good
outcomes for
children and
families.
70