Transcript Raggedy Ann: Prompts, Supports, Scaffolds

Raggedy Ann or
Student Performance:
Construct Relevance in
Alternate Achievement
Standards
Assessments
How can we support students but not
interfere with the construct ?
• Does the assessment truly measure that which
it intended to measure?
– Content Standards in
• Reading
• Math
• Science
• How can we support students maximizing the
assessment population but not interfere with
the construct?
Where it gets messy…..
• AA-AAAS necessarily require the teacher to
– Directly observe student performance or recall
observation using ongoing (formative assessment)
techniques
• Performance tasks
– Level of prompt required to make a response
– Select the correct response from an array of responses
• Portfolios
– Compile on-going data collection sheets based either based on
non-instructional probes or on level of prompt required to make
a response or a combination of both
– EVEN Construct items that measure the intended
construct…
Historically, in Effective Practices in
Special Education
• The purpose of assessment was
– Formative (instructional improvement)
• Instruction and assessment in severe disabilities
which follows
– Techniques from Applied Behavior Analysis
– Relies on heavily on
•
•
•
•
Observational techniques
Task analysis and chains of behaviors
Collection of performance data during instruction
Identifying “controlling prompts” which ensure the intended
behavior will occur under the specified stimulus conditions
Ongoing Instruction or formative
assessments
• Must correspond to the skills being taught and
the unique characteristics of the learner
• (Deno & Mirikin, 1977)
• Directly observe student performance using
ongoing continuous direct assessment techniques
– Non instructional “probes/ tests”
– Systematic instructional procedures that record
the level of assistance required for the student
to make a response
– Or a combination of these approaches
(Wrestling & Fox, 2004; Alberto & Troutman, 2002; Browder, 1991;
Brown & Snell, 1993; Wolery et. al, Sailor & Guess, 1983; White &
Haring, 1980)
Prompt Definition
• Prompt – teacher behaviors that increase the
probability of a correct response – used in the
acquisition of behaviors/targets and faded as
learning improves.
– ( Wolery et al. 1992; Wrestling & Fox, 2004)
Support Definition
• Support – modifications in the
presentation of an instructional
target, materials, response modes
• referring to instruction and levels of participation in
general curriculum, inclusive education.
( Weymer, Lattin, & Agran 1992)
Probe Definition
• Probe – an opportunity for the student to
perform the skill given only the target
stimulus with NO OTHER input from the
teacher.
– (Wolery, et al. 1992; Wrestling & Fox, 2004)
Scaffold Definition
• Instructional scaffolding is the provision of sufficient support to
promote learning when concepts and skills are being first
introduced to students. These supports may include the following:
– Resources
– A compelling task
– Templates and guides
• These supports are gradually removed as students develop
autonomous learning strategies, thus promoting their own
cognitive, affective and psychomotor learning skills and knowledge
.
• The teacher’s level and type of support change over time from
directive, to suggestion, to encouragement, to observation.
– (Bruner; Vytgosky; Cazden & others)
The Question of the Day!
How do we “control” for irrelevant variance in
the design of assessment of students with
significant cognitive disabilities who need
supports in order to provide an independent,
discriminatory response?
Prompt Confusion
• The task direction is not a prompt
–”Find one more” is NOT a verbal prompt; it’s the
cue to respond. Another example, “Read the
word”
• Support to get student ready to respond is not a
prompt
– Allowing student to select one answer gets Kyle
ready to respond. Another example- placing
students hands above all answers so can slide
hand to correct card is NOT a physical prompt.
Wakeman adapted from Browder (2008) Prompting: Impact on Inferences about Student Learning
Student Achievement vs. Something
else
• Student achievement
• – Select picture for
main idea
• Full credit‐ eye gaze,
point
• Find main idea across
stories
• Solve problems- eye
gaze to correct answers
for TA
• “Something Else”
• Select picture with
model prompt- point
where I point
• Student works with a
peer to select picture
• Student did not select
picture but could select
not my best work
Wakeman adapted from Browder (2008)
Worksheet
• What is the targeted skill?
– Is it to trace numbers or add or subtract decimals?
• Could make very little inference about student
learning for tracing numbers given.
• Better way to assess AND support learning using
a scribe?
– One idea: Steps in a TA to solve problem and have
student eye gaze at or select from written choices
correct answer for each step. Teacher or student could
place correct answers in the equation.
Wakeman
Teaching vs. Testing
• This teacher may have previously assessed the
student without prompts on this task and is
now teaching the student.
• Assessment= unprompted independent
responses
• Teaching= prompts (changes in definitions of
terms to support student understanding;
review of individual items; provides correct
answers when student is wrong)
Wakeman
Scaffolding
• Interesting/confusing when considering scaffolds in
assessment- does the scaffolded support change the
construct?
• Behaviorist would consider a verbal prompt hierarchy
• Where do we get apples?
• – Let’s read the paragraph again (rereads paragraph)
• – Let me read some key sentences (rereads sentences
• with correct answer)
• – Listen to this sentence‐ “The apples came from an
orchard of well established trees.”
Wakeman adapted from Browder (2008)
Consideration for Partial Credit?
• When reducing the number of choices, it
can become an issue of independent
response vs. an accurate response.
– What is the priority within the state scoring
system?
– Inferences about student learning in
assessment- with only two options, guessing
plays a significant role. How does that play out
in scoring of the assessment?
– Good teaching practice regarding distractors
(relevancy – “What did they ride in the story- a
bus or a goldfish?” vs. “a bus or a car?”) may be
based upon student need.
Wakeman
Response Prompts
• Excellent example of Least to Most prompts
• Valuable in teaching to support students
• Could demonstrate learning for some students
(growth) in an assessment if they demonstrate a
change over time in the level of prompt needed
across different content
– (not master Romeo and Juliet and then right back to
needing physical prompts for Macbeth but show
change over time and content needing fewer physical
prompts for each story)
Wakeman
Lack of Appropriate Response Options
or Support = Bias in the Assessment
• When an item does not include the
appropriate support (picture symbols) as
this one does not, the assessment is then
biased against the student (no way to
participate in the item).
• By defining an active response the
student CAN make; minimize the barrier
of response mode
Wakeman adapted from Browder (2008)
Construct Relevance
Using the AA-AAAS Test Administration
Manual to Communicate the
Appropriate Use of Prompts
Test Administration Handbook
• Definitions of Prompts and Prompt
Hierarchies
• Use of prompts in task/item/artifact
development
• Recording use of prompts
• Scoring rubrics and prompts
• Instructional use of prompts
Different prompt hierarchies
• Verbal-Gesture-model-partial physicalfull physical
• Gesture-verbal-visual-partial physical-full
physical
• Auditory-visual-physical
• Verbal-gesture-physical
Different definitions for prompts
• Verbal prompt – This level of prompt requires
the teacher to give a specific verbal direction
in addition to the task direction.
(indirect/direct)
• Verbal: Auditory Statement (e.g., more than
repeat prompt)
• Auditory - Any sound or verbal direction given
to help the student initiate or participate in
the completion of the skill in an activity.
Different Definitions of Prompts
• Visual - Any picture, gesture, or modeling given to help the
student initiate or participate in the completion of the skill
in an activity.
• Gesture prompt – This level of prompt requires the teacher
to move his/her finger, hand, arm, or make a facial
expression that communicates to the student specific
information (e.g., teacher taps scanner switch button).
• Model prompt – This level of prompt requires the teacher
to demonstrate the correct response for the student, and
the student imitates the teacher’s model (e.g., the teacher
demonstrates how to push the switch and then asks the
student to repeat).
Code of Ethics
IT IS A BREACH OF PROFESSIONAL ETHICS FOR
SCHOOL PERSONNEL TO PROVIDE VERBAL AND
NON-VERBAL CLUES OF ANSWERS, TEACH ITEMS
ON THE TEST, SHARE WRITING PROMPTS, COACH,
HINT, OR IN ANY WAY INFLUENCE A STUDENT’S
PERFORMANCE DURING THE TESTING SITUATION. A
BREACH OF ETHICS MAY RESULT IN INVALIDATION
OF TEST RESULTS AND LEA OR MSDE DISCIPLINARY
ACTION.
Prompts in Task/Item/Artifact
Development
Various levels of direction
• Data Charts. The template for the Full Physical
Prompt data chart includes an area to list the
various assistive technology devices a student
may use to explore the full range of assistive
technologies to support demonstration of skills.
• Prompts may not be used in the baseline
• Use of Prompts No prompts are allowed. Be
careful not to use physical or verbal prompts to
encourage student to make a choice (for
example, gesturing, looking, pointing)
Recording of Prompt Use During
Assessment
• Record on data charts
• Report on student work
• State after videotaping or audio taping of
student performance
Prompts: Scoring Rubric
Prompts That Prevent Grading for Accuracy
The teacher must grade the item as incorrect;
the prompt has made it impossible to
determine student performance.
Full Physical Prompt
Model of Exactly the Same Task
Telling Student the Answer
Scoring
Level of Independence
• The student seldom requires cues or prompts
when demonstrating skills based on the
documented AGLIs. (80-100%Independence)
• The student requires limited cues or prompts to
demonstrate skills based on the documented
AGLIs.(60-79%Independence)
• The student requires extensive cues or prompts
to demonstrate skills based on the documented
AGLIs. (30-59% Independence)
Scoring
• The number of steps not requiring prompts
per number of steps or items in the
assessment task determines the percentage of
independence.
• 80% or greater is attained, and the prompt
level and number do not exceed prompt level
and number of prompts indicated in the
mastery objective.
I have noticed some significant changes with our students (who represent
those with the greatest challenges of all who are tested). One struggle in
working with students with SCD is that when they develop skills, these are so
often “prompt dependent” skills. Students display these behaviors only once
others in their environment elicit them. Despite professional inservice training
to minimize over prompting, staff often seems to persist with giving many
prompts. This is one of the areas I have worked on in training but the testing
process seems to have made the most significant difference in helping staff be
more mindful of prompts they are giving as well as providing more
wait time for students to respond without prompts. This certainly moves them
toward greater independence and that is a wonderful and positive thing!
— Submitted by Roslyn Canosa, Principal; Baltimore City Public Schools
Advice for Minimizing Irrelevant
Variance in AA-AAAS
• Use procedures that support independent responses.
– If prompted responses are accepted, score appropriately.
• Provide structure (e.g. scripts, protocols) for the
administration of “items”.
• Specify appropriate and inappropriate “supports”,
“scaffolds”, “prompts” in the administration manual.
• Provide items or provide structure/ protocol for item
development specifying the construct.
• Caution is warranted when using systematic least to most
prompting systems as they may
– Increase the number of item presentations for students who
may require higher levels of prompts
– Prompts are aimed at producing a correct response