Transcript ECO Overview New Mexico - osep

Issues in Using Assessments
in Accountability Systems for
Young Children with
Disabilities
Kathleen Hebbeler
SRI International
Project Director’s Meeting
Washington, DC July-August 2006
Early Childhood Outcomes Center
Objective

Provide background information
relevant to the panel discussion,
including what the ECO Center
has learned about the current
state of assessment practices
through our work with states
Early Childhood Outcomes Center
Issue: Limitations of existing
assessment tools
“Assessment of young children poses
greater challenges than people generally
realize….assessment results—in particular,
standardized tests, that reflect a given
point in time—can easily misrepresent
children’s learning…There is widespread
dissatisfaction with traditional normreferenced standardized tests which are
based on early 20th century psychological
theory.”
National Research Council, 2001
Early Childhood Outcomes Center
Problem: Nature of the young
child




Not well suited to a standardized
testing situation.
Performance varies from day to
day, place to place, person to
person.
Don’t perform well for strangers
or on demand.
Growth is sporadic and uneven.
Early Childhood Outcomes Center
Problem: Response capabilities
of children with disabilities



Same issue as with school-age children:
assessment assumes child who can see,
hear and understand spoken language,
point, etc.
Few assessments include
accommodations
Almost no data on validity of
accommodations
Early Childhood Outcomes Center
Problem: Psychometric properties of
existing instruments


Some of the most common
instruments in use in the field
have limited or no reliability and
validity data.
None have validity or reliability
data when used for outcomes
reporting.
Early Childhood Outcomes Center
Response: New forms of
assessment



Growing recognition that the only way
to get a valid picture of what a child can
do/does to is look at performance over
a variety of settings and people
including what the child does
spontaneously with familiar adults.
Can’t base conclusions about child’s
capabilities on elicited responses alone.
“Authentic assessment”
Early Childhood Outcomes Center
Response: Use multiple
sources of information (best
practice)
“A single test, person, or occasion is
not a sufficient source of information.
This means that we must gather
information from several sources,
instruments, settings and occasions
to produce the most valid description
of the child’s status of progress”
 ---DEC Recommended Practices
Early Childhood Outcomes Center
Issue: Validity vs. credibility
dilemma
Strangers can’t elicit valid data on
young children’s performance
capabilities in a testing situation
BUT
can data produced by those who know
the child and whose programs are
being evaluated, be credible in an
accountability system?
Early Childhood Outcomes Center
Issue: Existing assessments are
domains-based



OSEP outcomes are functional and
cut across domains
Existing assessments provide
scores for domains, not 3
outcomes
Existing assessments vary in the
extent to which they assess
functioning vs. isolated skills
Early Childhood Outcomes Center
Issue: Variation in provider
knowledge of assessment
(Based on ECO work with states)
 Some providers are skilled in
administering and interpreting multiple
assessments, some in one, some rarely
use any.
 Many children served in programs for
typically developing children where
knowledge and use of assessment is
limited.
 How will providers be trained and
supervised?
Early Childhood Outcomes Center
Issue: Variation in how assessments
are incorporated into service
provision



Possible scenario: Eligibility assessment,
assessment for intervention planning, ongoing
assessment to monitor progress.
Are programs where the only formal
assessment is for eligibility.
[Note: Distinction between assessment for
eligibility (norm-referenced, gives percent
delay) and curriculum-based assessment or
progress monitoring tools (gives information
that is used day to day work with child)]
Early Childhood Outcomes Center
Issue: Many children do not have
problems in all 3 outcome areas



States must report data on all children with
IFSP/IEPs in the 3 outcomes areas, even if the
child has no problems in an area and is not
receiving services.
E.g., many preschoolers are receiving services
only for a communication problem, possibly
only an articulation problem so they are not
routinely assessed in other areas.
Can a screening tool be used in an
accountability system to verify that children
are maintaining functioning comparable to
same aged peers in those areas?
Early Childhood Outcomes Center
Issue: Including families in the
assessment process




Families provide a unique perspective on
the child’s functioning
Few assessment tools have procedures for
incorporating that perspective
Programs vary in how much and how they
share assessment data with families,
especially with regard to communicating
developmental ages or extent of a child’s
delay.
OSEP reporting anchored to performance
of same aged peers
Early Childhood Outcomes Center
Bottom Line



States will be reporting data on
outcomes
These data could be critical to policy
decisions related to Part C and Part B
Preschool in the future.
How to we make sure assessments
make a meaningful contribution?
Early Childhood Outcomes Center