Transcript RTI_assess_CBM_intro..

Response to Intervention
Reliable Methods to Measure
Student Progress in Basic
Literacy Skills
Jim Wright
www.interventioncentral.org
www.interventioncentral.org
Response to Intervention
Monitoring Student Academics: Curriculum-Based
Measurement
www.interventioncentral.org
Response to Intervention
Models in Reading & Math
5 Strands of Mathematical
Proficiency
5 Big Ideas in Beginning
Reading
1. Understanding
1. Phonemic Awareness
2. Computing
2. Alphabetic Principle
3. Applying
3. Fluency with Text
4. Reasoning
4. Vocabulary
5. Engagement
5. Comprehension
Source: Big ideas in beginning reading.
Source: National Research Council. (2002). Helping
University of Oregon. Retrieved September 23,
children learn mathematics. Mathematics Learning Study
2007, from http://reading.uoregon.edu/index.php
Committee, J. Kilpatrick & J. Swafford, Editors, Center for
Education, Division of Behavioral and Social Sciences
and Education. Washington, DC: National Academy
Press.
www.interventioncentral.org
3
Response to Intervention
Curriculum-Based Evaluation: Definition
•
•
•
•
“Whereas standardized commercial achievement tests measure
broad curriculum areas and/or skills, CBE measures specific skills
that are presently being taught in the classroom, usually in basic
skills. Several approaches to CBE have been developed. Four
common characteristics exist across these models:
The measurement procedures assess students directly using the
materials in which they are being instructed. This involves sampling
items from the curriculum.
Administration of each measure is generally brief in duration
(typically 1-5 mins.)
The design is structured such that frequent and repeated
measurement is possible and measures are sensitive to change.
Data are usually displayed graphically to allow monitoring of student
performance.”
SOURCE: CAST Website: http://www.cast.org/publications/ncac/ncac_curriculumbe.html
www.interventioncentral.org
4
Response to Intervention
SOURCE: CAST Website: http://www.cast.org/publications/ncac/ncac_curriculumbe.html
www.interventioncentral.org
Response to Intervention
Curriculum-Based Measurement: Advantages as a Set of Tools to
Monitor RTI/Academic Cases
• Aligns with curriculum-goals and materials
• Is reliable and valid (has ‘technical adequacy’)
• Is criterion-referenced: sets specific performance levels for specific
tasks
• Uses standard procedures to prepare materials, administer, and
score
• Samples student performance to give objective, observable ‘lowinference’ information about student performance
• Has decision rules to help educators to interpret student data and
make appropriate instructional decisions
• Is efficient to implement in schools (e.g., training can be done quickly;
the measures are brief and feasible for classrooms, etc.)
• Provides data that can be converted into visual displays for ease of
communication
Source: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.
www.interventioncentral.org
6
Response to Intervention
Standards for Judging Academic Measures for RTI
(National Center for Student Progress Monitoring)
www.interventioncentral.org
7
Response to Intervention
RTI : Assessment & Progress-Monitoring
•
•
•
To measure student ‘response to instruction/intervention’ effectively,
the RTI Literacy model measures students’ reading performance and
progress on schedules matched to each student’s risk profile and
intervention Tier membership.
Benchmarking/Universal Screening. All children in a grade level are
assessed at least 3 times per year on a common collection of literacy
assessments.
Strategic Monitoring. Students placed in Tier 2 (supplemental)
reading groups are assessed 1-2 times per month to gauge their
progress with this intervention.
Intensive Monitoring. Students who participate in an intensive,
individualized Tier 3 reading intervention are assessed at least once
per week.
Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools:
Procedures to assure scientific-based practices. New York: Routledge.
www.interventioncentral.org
8
Response to Intervention
Example of Curriculum-Based Assessment
Reading Probe
www.interventioncentral.org
Response to Intervention
DIBELS Reading Probe: Benchmark 2.1
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
11
Response to Intervention
57
WPM
www.interventioncentral.org
12
Response to Intervention
CBM Student Reading Samples:
What Difference Does Fluency Make?
• 3rd Grade: 19 Words Per Minute
• 3rd Grade: 70 Words Per Minute
• 3rd Grade: 98 Words Per Minute
www.interventioncentral.org
13
Response to Intervention
Evaluating the ‘RTI
Readiness’ of School
Assessments
Jim Wright
www.interventioncentral.org
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
15
Response to Intervention
Interpreting the Results of This Survey…
• YES to Items 1-3. Background. The measure gives valid general
information about the student’s academic skills and performance. While
not sufficient, the data can be interpreted as part of a larger collection of
student data.
• YES to Items 4-5. Baseline. The measure gives reliable results when
given by different people and at different times of the day or week.
Therefore, the measure can be used to collect a current ‘snapshot’ of the
student’s academic skills prior to starting an intervention.
• YES to Items 6-7. Goal-Setting. The measure includes standards (e.g.,
benchmarks or performance criteria) for ‘typical’ student performance
(e.g., at a given grade level) and guidelines for estimating rates of
student progress. Schools can use the measure to assess the gap in
performance between a student and grade level peers—and also to
estimate expected rates of student progress during an intervention.
• YES to Items 8-11. Progress Monitoring. The measure has the
appropriate qualities to be used to track student progress in response to
an intervention.
www.interventioncentral.org
16
Response to Intervention
Background: Validity
• Content Validity. Does the measure provide meaningful
information about the academic skill of interest?
• Convergent Validity. Does the measure yield results that
are generally consistent with other well-regarded tests
designed to measure the same academic skill?
• Predictive Validity. Does the measure predict student
success on an important future test, task, or other
outcome?
www.interventioncentral.org
17
Response to Intervention
Baseline: Reliability
• Test-Retest/Alternate-Form Reliability. Does the measure
have more than one version or form? If two alternate,
functionally equivalent versions of the measure are
administered to the student, does the student perform
about the same on both?
• Interrater Reliability. When two different evaluators
observe the same student’s performance and
independently use the measure to rate that performance,
do they come up with similar ratings?
www.interventioncentral.org
18
Response to Intervention
Benchmarks & Goal-Setting
• Performance Benchmarks. Does the measure include
benchmarks or other performance criteria that indicate
typical or expected student performance in the academic
skill?
• Goal-Setting. Does the measure include guidelines for
setting specific goals for improvement?
www.interventioncentral.org
19
Response to Intervention
Progress-Monitoring and Instructional Impact
• Repeated Assessments. Does the measure have sufficient
alternative forms to assess the student weekly for at least
20 weeks?
• Equivalent Alternate Forms. Are the measure’s repeated
assessments (alternative forms) equivalent in content and
level of difficulty?
• Sensitive to Short-Term Student Gains. Is the measure
sensitive to short-term improvements in student academic
performance?
• Positive Impact on Learning. Does research show that the
measure gives teachers information that helps them to
make instructional decisions that positively impact student
learning?
www.interventioncentral.org
20
Response to Intervention
Team Activity: Evaluating the ‘RTI
Readiness’ of School Assessments
At your table:
•
Review the handout
Evaluate the RTI Readiness
of Your School’s Academic
Measures.
•
Discuss how your school or
district might use such a form
to evaluate common classroom
academic measures.
www.interventioncentral.org
21
Response to Intervention
Curriculum-Based Measurement Tools for
Reading Skills
www.interventioncentral.org
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Initial Sound Fluency
: 3 minutes
Administration: 1:1
Description: The student is shown 4 pictures, each depicting an object that begins with a different
letter sound. The examiner gives the student a letter sound and asks the student to select the
picture of the object that begins with that letter sound. The process is repeated with new sets of
pictures until the time
Where to get materials: DIBELS https://dibels.uoregon.edu/
www.interventioncentral.org
23
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Phoneme Segmentation Fluency
 1 minute
Administration: 1:1
Description: The student is read a list of words that contain from 2 to five phonemes. For each
word, the student is asked to recite all of the phonemes that make up the word.
Where to get materials: DIBELS https://dibels.uoregon.edu/
www.interventioncentral.org
24
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Nonsense Word Fluency
 1 minute
Administration: 1:1
Description: The student is shown a list of nonsense words of 2 to 3 letters in length. For each
word, the student is to read the word or give the sounds that make up the word.
Where to get materials: DIBELS https://dibels.uoregon.edu/
www.interventioncentral.org
25
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Letter Naming Fluency
 1 minute
Administration: 1:1
Description: The student is presented with a list of randomly arranged letters. The student names
as many letters as possible.
Where to get materials: DIBELS https://dibels.uoregon.edu/
www.interventioncentral.org
26
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Letter Sound Fluency
 1 minute
Administration: 1:1
Description: The student is presented with a list of randomly arranged letters. The student gives
the sounds of as many letters as possible.
Where to get materials: www.interventioncentral.org
www.interventioncentral.org
27
Response to Intervention
Reading: Phonemic Awareness / Alphabetics
 Word Identification Fluency
 1 minute
Administration: 1:1
Description: The student is presented with a list of words randomly selected from a larger word list
(e.g., Dolch Wordlist). The student reads as many words as possible.
Where to get materials:
Easy CBM http://www.easycbm.com
Intervention Central http://www.interventioncentral.org (Dolch wordlists)
www.interventioncentral.org
28
Response to Intervention
Reading: Fluency
 Oral Reading Fluency
: 1 minute
Administration: 1:1
Description: The student reads aloud from a passage and is scored for fluency and accuracy.
Passages are controlled for level of reading difficulty.
Where to get materials:
DIBELS https://dibels.uoregon.edu/
AimsWeb http://www.aimsweb.com/
Easy CBM http://www.easycbm.com
Intervention Central http://www.interventioncentral.org (Use the OKAPI page to create
customized ORF passages)
www.interventioncentral.org
29
Response to Intervention
Reading: Basic Comprehension
 Maze Passages
: 1-3 minutes
Administration: Group
Description: The student is given a passage in which every 7th word has been removed. The
student reads the passage silently. Each time the student comes to a removed word, the student
chooses from among 3 replacement words: the correct word and two distractors. The student
circles the replacement word that he or she believes best restores the meaning of the text.
Where to get materials:
AimsWeb http://www.aimsweb.com/
Intervention Central http://www.interventioncentral.org (Use the Maze Passage Generator page
to create customized Maze passages)
www.interventioncentral.org
30
Response to Intervention
Reading: Basic Comprehension
 Multiple-Choice Reading
Comprehension
: Unknown
Administration: Group
Description: The student is given a passage to read. The student then answers a series of
standardized comprehension questions based on the text.
Where to get materials:
Easy CBM http://www.easycbm.com
www.interventioncentral.org
31
Response to Intervention
Example: Using Local Reading Norms in
Coordination with Research Norms
www.interventioncentral.org
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
LOCAL NORMS EXAMPLE: Twenty-three 4th-grade students
were administered oral reading fluency Curriculum-Based
Measurement passages at the 4th-grade level in their school.
In their current number form, these data are not easy to
interpret.
So the school converts them into a visual display—a boxplot —to show the distribution of scores and to convert the
scores to percentile form.
When Billy, a struggling reader, is screened in CBM reading
fluency, he shows a SIGNIFICANT skill gap when compared to
his grade peers.
www.interventioncentral.org
33
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
January Benchmarking
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
National Reading
Norms: 112 CRW
Per Min
Group Norms: Converted
to Box-Plot
Median (2nd Quartile)=71
1st Quartile=43
3rd Quartile=108
Billy=19
Hi Value=131
Low Value=31
0
20
40
Source: Tindal, G.,
Hansbrouck, J., &
Jones, C. (2005).Oral
reading fluency: 90
years of
measurement
[Technical report
#33]. Eugene, OR:
University of Oregon.
60
80
100
120
140
160
Correctly Read Words-Book 4-1
www.interventioncentral.org
34