Transcript Building Quality Assessment Systems for IDEA Infant

Building Quality Child Assessment
Systems for IDEA Infant/Toddler and
Preschool Programs
Where We’ve Been Lately and
Where We Might Be Going Next
Mary McLean
8/27/2008
Where We’ve Been Lately
Working on Outcomes
Assessment!
IDEA 2004 and Accountability
Is money spent on programs for
young children with disabilities
producing good outcomes?
Decisions, decisions….
 What to measure ?
 Status vs progress ?
 One instrument or many ?
 Direct assessment or observation-based?
 Sample or report on all children?
Part C and Preschool Child
Outcomes
% of children who demonstrate improved:
 Positive social emotional skills (including
positive social relationships)
 Acquisition and use of knowledge and
skills (including early language/
communication [and early literacy])
 Use of appropriate behaviors to meet their
needs
Reporting Categories
A. % of children who did not improve functioning
B. % of children who improved functioning but not sufficient to
move nearer to functioning comparable to same-aged
peers
C. % of children who improved functioning to a level nearer to
same-aged peers but did not reach it
D. % of children who improved functioning to reach a level
comparable to same-aged peers
E. % of children who maintained functioning at a level
comparable to same-aged peers
Timelines
__X__December 2005 – States submit
State Performance Plans
__X__Feb 2007 – APR: Child status at
entry data
__X__Feb 2008 – APR: First progress data
_____2009 - First report to the public
_____2010 – Targets to be set
Time to Celebrate!!
Where We Are Now
Working on Quality
Assessment Practices
As states work through the many challenges
involved in developing accountability
systems, we may find that the push for
accountability has the unintended positive
consequence of building better assessment
practices.
Hebbeler, Barton & Mallik, 2007
Building a better assessment
system
for children, families and
programs
Recommended Practices for
Assessment
 Neisworth, J.& Bagnato, S. (2005). DEC recommended
practices: Assessment. In Sandall, Hemmeter, Smith &
McLean (Eds) DEC recommended practice: A comprehensive
guide for application. Longmont, CO: Sopris West Publishing
Co.
 NAEYC and NAECS/SDE (2003). Early childhood curriculum,
assessment, and program evaluation: Building an effective
accountable system in programs for children birth through age 8.
http://www.naeyc.org/about/positions/cape.asp
 DEC (2007). Promoting positive outcomes for children with
disabilities: Recommendations for curriculum, assessment and
program evaluation. www.dec-sped.org
 Snow, C. & VanHemel, C. (2008). Early childhood assessment:
Why, what and how? Washington, DC: National Academies
Press.
Characteristics of a Quality
Assessment System
 Useful (utility)
 Authentic
 Collaborative
 Universal
Useful
assessment utility
The Birth to 6 Child Outcome System
1.
Assess at
entry and
exit.
4.
2.
Determine
status
ratings at
entry and
exit
3.
Provide this
information
to the state.
The state collects
this data and
reports to OSEP:
percentages of
children meeting
criteria for each
reporting category
5.
The state
determines
goals and
improvement
activities.
COSF
Social Emotional
Acquiring knowledge and skills
Actions to meet needs
Formative and Summative
Assessment
Formative Assessment
 Assessment for
learning
 Ongoing during
intervention
 Informs instruction
 Benefits the children
who are being
assessed
Summative Assessment
 Assessment of learning
 Completed as a child is
exiting a program
 Informs accountability or
program evaluation
 Benefits programs
Wisconsin Model Early Learning Standards
Teaching Cycle
Ongoing Assessment
Gathering information to determine what the child can
do and what the child is ready to learn
Implementation
Providing meaningful, experiential
activities that support individual
and group goals guided by
supportive interaction and
relationships
Planning
Deciding what should be done to promote
development and what we want children
to learn.
Ongoing Assessment
 Helps decide what to teach
 Helps decide how to teach
 Helps to know when to make changes in
teaching
 Is “assessment in the service of
instruction”
(McAfee & Leong, 2002)
An Assessment System with High
Utility
 Informs program planning and progress
monitoring and accountability
 Informs general early childhood
planning/monitoring and IFSP/IEP
planning/monitoring
The Birth to 6 Child Outcome System
1.
Utilize
authentic,
on-going
assessment
practices.
4.
2.
Determine
status
ratings at
entry and
exit
3.
Provide this
information
to the state.
The state collects
this data and
reports to OSEP:
percentages of
children meeting
criteria for each
reporting category
5.
The state
determines
goals and
improvement
activities.
COSF
Social Emotional
Acquiring knowledge and skills
Actions to meet needs
Authentic
Authentic assessment
Observation of child behavior
over time in typical routines
and activities
Standardized_________________________________Authentic
Direct Assessment__________________Observation-based assessment
“…the science of the strange behavior of children
in strange situations with strange adults for the
briefest possible period of time”
Bronfenbrenner (1977)
(standardized assessment)
“The best way to understand the development of
children is to observe their behavior in natural settings
while they are interacting with familiar adults over
prolonged periods of time.” Bronfenbrenner, 1977
(authentic assessment)
Observation_______Direct Assessment
 Consistency depends on 
teacher training and
monitoring of
implementation
 Behavior measured is
child’s typical behavior
 Increased utility for
instruction
(Mathematica, 2007)
Procedures for consistency in
administration and scoring are
built in
 Behavior sampled may not be
representative of child’s typical
behavior
Strategies for authentic assessment
Observation and documentation
anecdotal notes
event sampling
activity protocols
portfolio assessment
Embed assessment into
ongoing routines and activities
(Raver, S.,2003; Sandall & Schwartz, 2008)
Activity Matrix
(Sandall & Schwartz, 2008)
Schedule
Shelby
Arrival
Respond to peers
Free Play
Engage in activity
Circle
Imitate adult words
Matthew
Remove and hang up coat
Join ongoing play
Imitate adult actions
Outside
Throw ball with 2 hands
Join ongoing play
Snack
Use pincer grasp
Free Play
2-word utterance
Imitate adult actions
Circle/Departure
2-word utterance
Identify positional concepts
Davion
Follow 2-step directions
Request preferred item
Tell full name
Request preferred item
Pour juice
Transitions
Sort shapes
Follow 2-step direction
Anecdotal notes
Event Sampling
Schedule
Shelby
Matthew
Davion
Remove and hang up coat
Arrival
Respond to peers
2/2
Follow 2-step directions 2/2
with verbal prompt
Free Play
Engage in activity
Join ongoing play
8 minutes
2/2
Request preferred item
3/4
Tell full name
Circle
Imitate adult words
5/7
Imitate adult actions 4
with model
Outside
Throw ball with two hands
Join ongoing play
4/4
1/1
Request preferred item 1/2
Pour juice
Snack
Use pincer grasp
4/9
with physical guidance
Free Play
Circle/Departure
Transitions
2-word utterance
4
2-word utterance 3
Imitate adult actions 3
Copy square
2/2
Identify positional concepts
2/2 under, on top
Follow 2-step direction 2/2
Activity Protocols
An activity protocol provides a list of skills
from a curriculum-referenced assessment
that are likely to be observed in a
particular activity or routine
Grisham-Brown, Hemmeter & Pretti-Frontczak, 2005
Activity Protocol: Playdough
AREA
ITEM
Motor
M-1
CHILD BEHAVIORS
Holds object with one hand and
manipulates with other
Cognitive C-3
Demonstrates understanding of size concepts
Adaptive
Fastens button on art smock
A-4
Portfolio Assessment
 Paper
 Electronic
Use of Authentic Assessment to
Inform Multiple Measures
 To inform curriculum-referenced
assessments which measure progress
toward EC learning targets or curricular
goals
 To measure progress toward IEP goals or
IFSP outcomes
 To inform measures of accountability
Collaborative
…with families and teams
Gathering information from families
Family involvement
expands the
validity of
assessment
information to
home and
community
environments
Strategies for Gathering Information
 Utilize home visits as a strategy to connect with
families
 Make periodic requests for information from
families (describe specific skills to watch for at
home.)
 Use assessment tools that have family report
forms: AEPS, HELP.
 Use existing informal communication mechanisms
(traveling notebook, daily conversations, e-mail,
telephone.)
Gathering information from other
service providers
How to Include Information from Other
Providers Systematically
 Plan collaborative activities for observations
 Schedule periodic team meetings or
staffings
 Share information and request information
Universal
…and individually
appropriate
Universality
Design and/or accommodations which
enable all children to demonstrate their
underlying functional capabilities
Bagnato, Neisworth,& Pretti-Frontczak, (in preparation).
Assessment must be valid for all children
including children who are English
Language Learners and children with
disabilities
Assessment and Learning Targets
Individualized
IFSP/IEP
Targeted Curricula
(e.g., early literacy, social emotional)
General
Curriculum
Commercial
General
Curriculum
Locally
Developed
Universal
Early Learning Standards or
Foundations
Universal Design
The design of products to be useable by all
people to the greatest extent possible
(Thompson, Johnstone & Thurlow, 2002)
Center for Applied Special Technology (CAST)
http://www.cast.org/research/udl/index.html
Desired Results access Project
California Department of Education
www.draccess.org
Accommodations/Adaptations
Adaptations are changes in environmental arrangements or differences
in observed behavior that allow children with disabilities to be
accurately assessed in the natural environment:
Augmentative or alternative communication system
Alternative mode for written language
Visual support
Assistive equipment or device
Functional positioning
Sensory support
Alternative response mode
www.draccess.org
Where Are We Going Next?
Innovative Practices in
Measurement
Innovative Models and Frameworks
Assessment Framework - RTI
Measurement Model
- IRT / Rasch
Assessment Framework: RTI
Center for Response to Intervention in Early
Childhood
http://www.crtiec.org/
Recognition and Response: An Early
Intervening System for Young Children At
Risk for Learning Disabilities
http://www.fpg.unc.edu/~randr/
Where did RTI come from?
 Initiatives in the field of learning disabilities
 Written into IDEA 2004 – “early
intervening”
 Part B money can be used to fund
programs for children who have not yet
been found eligible for special education
RTI Tiered Model
 Tier 1: Universal periodic screening and
resulting interventions
 Tier 2: Research-based small group
interventions
 Tier 3: Individualized interventions and
possible referral for special education
What Might RTI Mean for Early
Childhood?
 Universal and Periodic Screening
(Prevention)
 Progress Monitoring
 Outcomes Measurement
Screening and Progress Monitoring:
Individual Growth and Development
Indicators
(IGDIs)
General Outcomes Measurement
(GOM)
IGDI’s Communication Trajectory
Child’s
Observed
Trajectory
Normative
Trajectory
Below
Average
(-1.5 SD)
Trajectory
www.igdi.ku.edu
IGDIs Available
Infant and Toddler
Preschool
Early Communication
Picture Naming
Early Problem Solving
Alliteration
Early Movement
Rhyming
Early Social
www.igdi.ku.edu
www.ggg.umn.edu
Accountability: Crosswalk to OSEP Outcomes
Measurement Model:
Rasch Scaling
Item Response Theory
Rasch/IRT Measurement Model
 “Allows the creation of interval-scaled calibrations when
raw scores are ordinal”
Translation: a criterion or curriculum-referenced
instrument can be calibrated so that it yields scaled
scores (equal interval scores)
 The child’s ability is described relative to position on a
specific developmental path not relative to scores from
the norming population.
Classic Psychometric Theory:
Normal Curve
2%
14%
-2
34%
-1
34%
-1
Mean
Median Mode
Standard Deviation
14%
2%
-2
Rasch Scaling: Demonstrating higher
level skills results in a higher score
More of the construct or skill
(later developing skill)----
Less of the construct or skill
(earlier developing skill)----
So…..
 Allows measurement of progress as
acquisition of skills – not as position relative
to scores from the norming population
 Allows creation of an item bank which
prevents “narrowing of the curriculum” or
teaching to the test
(Meisels, 2002)
Rasch/IRT Instruments
 Desired Results Developmental Profile access
(DRDP access)
www.draccess.org
 Ages and Stages Questionnaire (ASQ)
www.brookespublishing.com/store/books/bricker-asq/
 Assessment, Evaluation and Programming System for
Infants and Children (AEPS)
www.brookespublishing.com/store/books/bricker-aeps/
What Do We Need?
What Do We Need?
Research on:
 direct assessment and observation-based
assessment with young children
 new assessment instruments
 assessment systems that weave general early
childhood and EI/ECSE
 procedures for assessing children who are
English Language Learners
 accommodations for children with disabilities
What Do We Need?
Personnel Development:
 pre-service preparation that provides a solid
foundation in assessment skills
 system for continuous in-service training to
address local needs
 emphasis on measurement skills in doctoral
programs
What Do We Need?
Policy
 alignment of early learning guidelines,
assessment practices, curricular practices and
accountability requirements across agencies
and programs serving young children
Thank You!
Mary McLean, Ph.D.
Kellner Professor of Early Childhood Education
Department of Exceptional Education
School of Education
University of Wisconsin-Milwaukee
414-229-2213
[email protected]