Transcript How are you currently using AEPS (CBA)

Assessment, Evaluation, and
Programming System for Infants
and Children (AEPS™)-AEPSi
KDEC: 2013
Part 1
Nancy Miller, M.Ed., OTR/L
Stephanie Parks, Ph.D., OTR/L
• Authentic assessment
• AEPS Overview (brief)
• Fidelity in Assessment
• Scoring, Team Consensus,
and Efficiency
• Pitfalls to Fidelity
• Resources
The root of the word assessment is
assidere, which means “to sit
beside and get to know.”
Authentic Assessment
Authentic Assessment
o
o
o
o
Familiar people….
In familiar settings…
With familiar objects/toys…
Doing familiar things.
Adapted from: Sophie Hubbell, M.A.T
Kent State University
Assessment Evaluation and Programming System
(AEPS) for Infants and Children (Second Edition)
AEPS Interactive (AEPSi)
• Secure, web-based tool
• Allows for easy means to record, score, track,
aggregate, archive, and report on the results of
the AEPS Test
• http://aepsinteractive.com
• http://www.aepsi.com
How are you currently using
AEPS (CBA)
•Screening
•Eligibility
•Program Planning
•Progress Monitoring
•Program Evaluation
What is the AEPS
• It is a comprehensive system that ties
together assessment, goal
development, intervention, and
ongoing monitoring and evaluation.
It is
•
•
•
•
Criterion-referenced
Curriculum-based
Domain Specific
Developmental
• Can be used to corroborate eligibility
decisions
• Programmatic: can help you determine
priority goal areas and focus your
interventions
linked system framework
• Assessment
• Goal
Development
Collecting
Information
Observations
Direct Tests
Report
Monitoring
Weekly
Quarterly
Annual
• Evaluation
Summarizing
Information
Student goals
Family outcomes
Curricular
Approach
Intervention based
on Extended
Standards
Instruction
• Intervention
This
Not This
Fine
motor
Gross
motor
Social
AEPS
DOMAINS
Social
Communication
Adaptive
Cognitive
Organizational Structure of AEPS test items
AREA
STRAND A
Goal 1
Goal 2
Obj. 1.1
Obj. 2.1
Obj. 1.2
Obj. 2.2
Obj. 1.3
Obj. 2.3
STRAND B
Goal 3
Goal 1
Goal 2
Obj. 3.1
Obj. 1.1
Obj. 2.1
Obj. 3.2
Obj. 1.2
Obj. 2.2
Obj 2.3
Goals: Easy to More Difficult
Strands: Easy to More Difficult
Objectives become more difficult as the goal is approached.
© Jennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007
“One mark of excellent
teachers is the frequency
with which they evaluate
and change children’s
instructional programs
continually adapting
them to meet the needs
of each child.”
Bailey and Wolery, 1992
FIDELITY
Evidence is needed that
an assessment such as a
CBA [AEPS] is
administered,
summarized, interpreted,
and used in the way that
it was designed, intended
and validated.
Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The
Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.
Collecting Assessment and Evaluation
Information with Fidelity-Scoring
• ONGOING Observation (PREFERRED)
• Within routines/activities
• Direct Test*
• Report
*Note: Scoring guidelines when using the Direct Test
method are not the same as the Observation guidelines.
Refer to page 47 in Volume 1.
Scoring Guidelines:
Observation or Direct Test
• 2 = Consistently meets criterion
• 1 = Inconsistently meets criterion;
emerging
• 0 = Does not meet criterion; currently does not
exhibit the skill; (in preschool aged child-may not yet be
expected)
2
Consistently
meets
criterion
Observation
1
Inconsistently
meets
criterion
• Child consistently performs the item as specified in the
criterion
• Child performs the item independently
• Behavior is a functional part of the child’s repertoire
• Child uses the skill across time, materials, settings, and
people
• Child does not consistently perform the item as specified in the criterion
• Child performs the item with assistance
• Child does not perform all components of the item or does not meet all
aspects of the specified criterion (i.e., the behavior is emerging
• Child performs the item only under specific situations or conditions (i.e.,
with certain people or in certain settings)
Observation
0
Does not
meet criterion
• Child does not perform the item as specified in the criterion when given
repeated opportunities or assistance or when modifications and adaptations are
made.
• Child was not observed performing the item because it is not expected based on
knowledge of development (e.g., the child’s chronological age is 6 months and
he or she would not be expected to perform such items as categorizing similar
objects, copying simple shapes, or walking up and down stairs)
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
2
Consistently
meets
criterion
• Child performs the item as specified in the criterion on at
least two out of three trials
• Child performs the item independently on two out of three
trials
• Child uses the skill on two out of three trials across time,
materials, settings, and people
DIRECT TESTING
1
Inconsistently
meets
criterion
• Child performs the item as specified in the criterion on one out of
three trials
• Child performs the item with assistance on one out of three trials
• Child performs the item under one situation or one set of
conditions
DIRECT TESTING
0
Does not
meet criterion
• Child does not yet perform the item as specified in the criterion on zero out of
three trials when assistance is provided or when modifications and adaptations
are made
• Child was not observed performing the item because it is not expected based on
knowledge of development (e.g., the child’s chronological age is 6 months and
he or she would not be expected to perform such items as categorizing similar
objects, copying simple shapes, or walking up and down stairs)
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
Scoring with Fidelity ….
It’s not just a 0, 1, 2
Use of Notes and Comments to
enhance your assessment data
• A = assistance provided (1 or 0)
• B = behavior interfered (1 or 0)
• D = direct test (2, 1, 0)
• M = modification/adaptations (2, 1, 0)
• Q = quality of performance (2, 1)
• R = report (2, 1, 0)
A
B
• Assistance Provided:
• When a child is provided some form of assistance, an “A”
should be noted in the space next to the performance score
box. If assistance is provided, then the only scores allowed
are 1 and 0 because a score of 2 indicates full independent
performance.
• Assistance includes any direct verbal or physical prompt, cue
or model that assists the child in initiating or performing the
desired behavior.
• Behavior Interfered
• At times a child’s behavior may interfere with the
demonstration of the desired skill. In such cases, the item
may be scored 1 or 0 with a “B” noted next to the
performance score.
• This note indicates that the child may have the skill, but
disruptive or noncompliant behavior interfered with its
demonstration.
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
D
M
• Direct Test:
• When the examiner directly elicits a behavior, a “D” is noted
next to the performance score and the guidelines for
determining the score for “Direct Test” are followed.
• Modification/Adaptation:
• At times, an examiner may need to modify the stated criteria
(e.g., rate or mode of response) or adapt the
environment/materials (e.g., adaptive equipment is necessary)
to asses children with sensory or motor disabilities.
• When modifications are made in gathering child performance
information, an “M” is noted next to the performance score
and a 2, 1, or 0 is used.
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
Q
• Quality of Performance:
• At times, a child is able to perform a skill independently,
but the team feels the quality of the performance
hinders the ability to meet criteria. At other times, a
child is able to meet or partially meet the criteria, but
the team wishes to continue strengthening the quality
of performance.
• When the quality of the performance is in question,
teams are encouraged to use a “Q” in the notes
accompanied by a score of 2 or 1.
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
R
• Report:
• When an item is assessed by report, an “R” is noted next to the
performance score. Report is used under one of three conditions:
• When assessment information is collected by another person or
documented source (e.g. written evaluation), the item is scored
2, 1, or 0, and an “R” is noted.
• When the item is judged inappropriate because it assesses a
primitive or developmentally easier response (e.g., sucking on a
nipple when the child is able to drink from a cup), the item is
scored 2 and an R is noted.
• When the item is judged inappropriate because it is too
advanced or beyond the child’s developmental level (e.g.,
walking when the child is unable to stand), the item is scored 0
and an “R” is noted.
From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.
Last but not least…
Your written
comments to
quantify, describe,
enhance the
observations you
have made.
Team Consensus Builds
Scoring Fidelity
Team Discussions
• What do these scores mean to you?
• Consistently/Inconsistently
• The power of the Notes section
• With assistance
• Modifications/Adaptations
• Team use of the Comment Section
• Under what conditions/strategies etc.
• Team responsibilities discussion
Don’t forget the Family Report
• 2 Levels (birth to three and three to six)
• 2 sections
• Family Routines
• Often done through
conversation/interview
• Family Observations
• Scored: Yes, Sometimes, Not Yet
Consider using the
second section of the
Family Report with
Community and Daycare
Providers since the skills
parallel the AEPS CODRF
across the
developmental domains.
Score as “Report”
Ways to gather and document
• Individually
• CODRF
• Group
• Group routine/activity
matrix
• Family Report
• I and II (Spanish version
available)
Assessment Activity Plans
• AEPS comes with 12 prewritten activities to assess a
variety of children across
developmental areas (see
Volume 2)
• OR you can create your own
that parallel existing planned
activities or those provided in
the AEPS.
Adapted from: Sophie Hubbell, M.A.T
Kent State University
Administering the AEPS with Groups of Children
AEPSi Assessment Activities: Center-based
(Level 1 and Level 2)
• Book About Me
• Classroom Transitions
and Routines
• Dramatic Play
• Meals and Snack
• Story Time
• Playdough and
Manipulatives
• Outdoor Play
• Conversation with
Familiar Adults
Jennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007
AEPSi Assessment Activities: Routine-Based
(Level 1)
• Rough & Tumble
• Quiet Time
• Mystery Box
• Feeding & Meals
• Daily Transitions &
Routines
• Conversations with
Caregivers
• Busy Hands
© Jennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007
Reasons why CBAs are not
implemented with fidelity
• Teachers may find the actual
implementation of a CBA to be
overwhelming (particularly in
classrooms where large amounts of
data have to be collected on many
children).
• Sometimes teachers lack the
training or support to administer
the CBA. (teachers may become
frustrated and implement the
assessment with low fidelity)
Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach.
Baltimore: Paul H. Brookes Publishing Co.
Pitfall # 1: Most ECI assessments, do not include
administration checklists that can be used for
integrity/fidelity checks.
• REMEDIES:
• use the authentic assessment
fidelity measure by Grisham-Brown
and colleagues (2008)
• develop a fidelity measure that
relates to procedures of the
assessment being used by your
program
Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach.
Baltimore: Paul H. Brookes Publishing Co.
http://www.ehhs.kent.edu/ceecrt/index.php/research/current
Pitfall #2: Teams administer assessments without
sufficient training or ongoing support
• REMEDIES:
• utilize a coaching system in which
teachers check on another and
provide support for those who are
new to using the assessment.
• engage in ongoing professional
development to ensure accuracy in
scoring and use and avoid drift over
time.
Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach.
Baltimore: Paul H. Brookes Publishing Co.
“Regardless of how information is
gathered, what information is
gathered, or even how the
information is summarized, if it isn’t
used to plan and guide instruction,
then the process is a waste of the
teacher’s time and provides no
advantage for young children.” (p,
170)
Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach.
Baltimore: Paul H. Brookes Publishing Co.
KITS/TASN TA Packet:
Embedding
Assessment into Daily
Activities and
Routines
http://www.kskits.org
/ta/Packets/embedAss
essment.shtml
AEPS BLOG: Screencast Series
http://aepsblog.blogspot.com/
Colorado Department of Education: Results Matter
http://www.cde.state.co.us/resultsmatter/R
MVideoSeries_PracticingObservation.htm#t
op
National Center on Quality Teaching and Learning
NCQTL
http://eclkc.ohs.acf.hhs.gov/hslc/ttasystem/teaching
NCQTL
http://eclkc.ohs.acf.hhs.gov/hslc/ttasystem/teaching/center/practice/ISS
http://www.aepsinteractive.com/
How can we collaborate and
share across the state?
Nancy Miller, M.Ed., OTR/L
[email protected]
Stephanie Parks, Ph.D., OTR/L
[email protected]