Transcript Document

Response to Intervention
RTI, Student Literacy
Assessment, & CurriculumBased Measurement: A SkillBuilding Lab
Jim Wright
www.interventioncentral.org
www.interventioncentral.org
Response to Intervention
PowerPoints & Handouts from Workshop Available at:
http://www.interventioncentral.org/
GSTBOCES.php
www.interventioncentral.org
2
Response to Intervention
Workshop Topics
1. RTI Assessment Requirements
& Curriculum-Based Measurement
2. Evaluating the ‘RTI Readiness’ of Literacy Measures
3. Review of Common CBM Literacy Assessments
4. CBM Oral Reading Fluency: Case Study
5. Collecting School-Wide CBM Norms: Procedures
6. Application of Local Norming Data to Evaluation
of Core Literacy Programs, Development of CSE
Eligibility Rules
www.interventioncentral.org
Response to Intervention
Five Core Components of RTI Service Delivery
1.
2.
3.
4.
5.
Student services are arranged in a multi-tier model
Data are collected to assess student baseline levels
and to make decisions about student progress
Interventions are ‘evidence-based’
The ‘procedural integrity’ of interventions is measured
RTI is implemented and developed at the school- and
district-level to be scalable and sustainable over time
Source: Glover, T. A., & DiPerna, J. C. (2007). Service delivery for response to intervention: Core components and directions
for future research. School Psychology Review, 36, 526-540.
www.interventioncentral.org
4
Response to Intervention
RTI ‘Pyramid of
Interventions’
Tier 3
Tier 2
Tier 1
Tier 3: Intensive interventions.
Students who are ‘nonresponders’ to Tiers I & II may be
eligible for special education
services, intensive interventions.
Tier 2 Individualized
interventions. Subset of
students receive interventions
targeting specific needs.
Tier 1: Universal interventions.
Available to all students in a
classroom or school. Can consist
of whole-group or individual
strategies or supports.
www.interventioncentral.org
5
Response to Intervention
Avg Classroom Academic
Performance Level
Discrepancy 1: Skill Gap
Discrepancy 2:
(Current
Gap in Rate of
Performance Level)
Learning (‘Slope
Target
of Improvement’)
Student
‘Dual-Discrepancy’: RTI Model
of Learning Disability (Fuchs 2003)
www.interventioncentral.org
6
Response to Intervention
NYSED RTI Guidance Memo: April 2008
www.interventioncentral.org
Response to Intervention
www.interventioncentral.org
8
Response to Intervention
“The Regents policy framework for RtI:
Defines RtI to minimally include:
Appropriate instruction delivered to all students in the
general education class by qualified personnel. Appropriate
instruction in reading means scientific research-based
reading programs that include explicit and systematic
instruction in phonemic awareness, phonics, vocabulary
development, reading fluency (including oral reading skills)
and reading comprehension strategies.
Screenings applied to all students in the class to identify
those students who are not making academic progress at
expected rates.”
www.interventioncentral.org
9
Response to Intervention
“Instruction matched to student need with increasingly intensive
levels of targeted intervention and instruction for students who do
not make satisfactory progress in their levels of performance
and/or in their rate of learning to meet age or grade level
standards.
Repeated assessments of student achievement which should
include curriculum based measures to determine if interventions
are resulting in student progress toward age or grade level
standards.
The application of information about the student’s response to
intervention to make educational decisions about changes in
goals, instruction and/or services and the decision to make a
referral for special education programs and/or services.”
www.interventioncentral.org
10
Response to Intervention
“Written notification to the parents when the student
requires an intervention beyond that provided to all students
in the general education classroom that provides information
about the:
-amount and nature of student performance data that will be
collected and the general education services that will be
provided;
-strategies for increasing the student’s rate of learning; and
-parents’ right to request an evaluation for special education
programs and/or services.”
www.interventioncentral.org
11
Response to Intervention
“The Regents policy framework for RtI:
Defines RtI to minimally include:
Requires each school district to establish a plan and policies
for implementing school-wide approaches and prereferral
interventions in order to remediate a student’s performance
prior to referral for special education, which may include the
RtI process as part of a district’s school-wide approach. The
school district must select and define the specific structure
and components of its RtI program, including, but not limited
to the:
-criteria for determining the levels of intervention to be provided to
students,
-types of interventions,
amount and nature of student performance data to be collected, and
-manner and frequency for progress monitoring.”
www.interventioncentral.org
12
Response to Intervention
Effective Formative Evaluation: The Underlying Logic…
1. What is the relevant academic or behavioral
outcome measure to be tracked?
2. Is the focus the core curriculum or system,
subgroups of underperforming learners, or
individual struggling students?
3. What method(s) should be used to
measure the target academic skill or behavior?
4. What goal(s) are set for improvement?
5. How does the school check up on progress
toward the goal(s)?
www.interventioncentral.org
Response to Intervention
Summative data is static information that provides a fixed ‘snapshot’
of the student’s academic performance or behaviors at a particular
point in time. School records are one source of data that is often
summative in nature—frequently referred to as archival data.
Attendance data and office disciplinary referrals are two examples of
archival records, data that is routinely collected on all students.
In contrast to archival data, background information is collected
specifically on the target student. Examples of background information
are teacher interviews and student interest surveys, each of which can
shed light on a student’s academic or behavioral strengths and
weaknesses. Like archival data, background information is usually
summative, providing a measurement of the student at a single point in
time.
www.interventioncentral.org
14
Response to Intervention
Formative assessment measures are those that can be administered
or collected frequently—for example, on a weekly or even daily basis.
These measures provide a flow of regularly updated information
(progress monitoring) about the student’s progress in the identified
area(s) of academic or behavioral concern.
Formative data provide a ‘moving picture’ of the student; the data
unfold through time to tell the story of that student’s response to
various classroom instructional and behavior management strategies.
Examples of measures that provide formative data are CurriculumBased Measurement probes in oral reading fluency and Daily Behavior
Report Cards.
www.interventioncentral.org
15
Response to Intervention
Formal Assessment Defined
“Formative assessment [in academics] refers to the
gathering and use of information about students’
ongoing learning by both teachers and students to
modify teaching and learning activities. ….
Today…there are compelling research results indicating
that the practice of formative assessment may be the
most significant single factor in raising the academic
achievement of all students—and especially that of
lower-achieving students.” p. 7
Source: Harlen, W. (2003). Enhancing inquiry through formative assessment. San Francisco, CA: Exploratorium. Retrieved on
September 17, 2008, from http://www.exploratorium.edu/ifi/resources/harlen_monograph.pdf
www.interventioncentral.org
16
Response to Intervention
Academic or Behavioral Targets Are Stated as
‘Replacement Behaviors’
“A problem solution is defined as one or more
changes to the instruction, curriculum, or
environment that function(s) to reduce or
eliminate a problem.” p. 159
Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school
psychology V (pp. 159-176).
www.interventioncentral.org
17
Response to Intervention
School Instructional Time: The Irreplaceable Resource
“In the average school system, there are 330
minutes in the instructional day, 1,650 minutes in
the instructional week, and 56,700 minutes in the
instructional year. Except in unusual circumstances,
these are the only minutes we have to provide
effective services for students. The number of years
we have to apply these minutes is fixed. Therefore,
each minute counts and schools cannot afford to
support inefficient models of service delivery.”
p. 177
Source: Batsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best practices in problem analysis. In A. Thomas & J.
Grimes (Eds.), Best practices in school psychology V (pp. 177-193).
www.interventioncentral.org
18
Response to Intervention
Formative Assessment: Essential Questions…
1. What is the relevant academic or behavioral
outcome measure to be tracked?
Problems identified for formative assessment should be:
1. Important to school stakeholders.
2. Measureable & observable.
3. Stated positively as ‘replacement behaviors’ or goal statements
rather than as general negative concerns (Bastche et al., 2008).
4. Based on a minimum of inference (T. Christ, 2008).
Source: Batsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best practices in problem analysis. In A. Thomas & J.
Grimes (Eds.), Best practices in school psychology V (pp. 177-193).
Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V
(pp. 159-176).
www.interventioncentral.org
19
Response to Intervention
Academic or Behavioral Targets Are Stated as
‘Replacement Behaviors’
“The implementation of successful interventions begins
with accurate problem identification. Traditionally, the
student problem was stated as a broad, general
concern (e.g., impulsive, aggressive, reading below
grade level) that a teacher identified. In a competencybased approach, however, the problem identification is
stated in terms of the desired replacement behaviors
that will increase the student’s probability of successful
adaptation to the task demands of the academic
setting.” p. 178
Source: Batsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best practices in problem analysis. In A. Thomas & J.
Grimes (Eds.), Best practices in school psychology V (pp. 177-193).
www.interventioncentral.org
20
Response to Intervention
Inference: Moving Beyond the Margins of the ‘Known’
“An inference is a tentative conclusion without direct
or conclusive support from available data. All
hypotheses are, by definition, inferences. It is critical
that problem analysts make distinctions between
what is known and what is inferred or
hypothesized….Low-level inferences should be
exhausted prior to the use of high-level inferences.”
p. 161
Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school
psychology V (pp. 159-176).
www.interventioncentral.org
21
Response to Intervention
Examples of High vs. Low Inference Hypotheses
The results of grade-wide benchmarking in reading show that a
target 2nd-grade student can read aloud at approximately half the
rate of the median child in the grade.
High-Inference Hypothesis. The student has an
auditory processing issue that prevents success in
reading. The student requires a multisensory
approach to reading instruction to address reading
deficits.
Unknown
Known
Unknown
Low-Inference Hypothesis. The student needs
to build reading fluency skills to become more
proficient in decoding.
www.interventioncentral.org
Known
22
Response to Intervention
Adopting a Low-Inference Model of Reading Skills
5 Big Ideas in Beginning Reading
1. Phonemic Awareness
2. Alphabetic Principle
3. Fluency with Text
4. Vocabulary
5. Comprehension
Source: Source: Big ideas in beginning reading. University of Oregon. Retrieved September 23, 2007, from
http://reading.uoregon.edu/index.php
www.interventioncentral.org
23
Response to Intervention
Formative Assessment: Essential Questions…
2. Is the focus the core curriculum or system,
subgroups of underperforming learners, or
individual struggling students?
Apply the ‘80-15-5 ‘Rule (T. Christ, 2008) :
–
–
–
If less than 80% of students are successfully meeting academic or behavioral
goals, the formative assessment focus is on the core curriculum and general
student population.
If no more than 15% of students are not successful in meeting academic or
behavioral goals, the formative assessment focus is on small-group ‘treatments’
or interventions.
If no more than 5% of students are not successful in meeting academic or
behavioral goals, the formative assessment focus is on the individual student.
Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school
psychology V (pp. 159-176).
www.interventioncentral.org
24
Response to Intervention
Using Local Norms in Coordination with
Benchmark Data
www.interventioncentral.org
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
LOCAL NORMS EXAMPLE: Twenty-three 4th-grade students
were administered oral reading fluency Curriculum-Based
Measurement passages at the 4th-grade level in their school.
In their current number form, these data are not easy to
interpret.
So the school converts them into a visual display—a boxplot —to show the distribution of scores and to convert the
scores to percentile form.
When Billy, a struggling reader, is screened in CBM reading
fluency, he shows a SIGNIFICANT skill gap when compared to
his grade peers.
www.interventioncentral.org
26
Response to Intervention
Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students
January Benchmarking
Group Norms: Correctly Read Words Per Min: Book 4-1: Raw Data
31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112
115 118 118 131
National Reading
Norms: 112 CRW
Per Min
Group Norms: Converted
to Box-Plot
Median (2nd Quartile)=71
1st Quartile=43
3rd Quartile=108
Billy=19
Hi Value=131
Low Value=31
0
20
40
Source: Tindal, G.,
Hansbrouck, J., &
Jones, C. (2005).Oral
reading fluency: 90
years of
measurement
[Technical report
#33]. Eugene, OR:
University of Oregon.
60
80
100
120
140
160
Correctly Read Words-Book 4-1
www.interventioncentral.org
27
Response to Intervention
Formative Assessment: Essential Questions…
3. What method(s) should be used to
measure the target academic skill or behavior?
Formative assessment methods should be as direct a measure as
possible of the problem or issue being evaluated. These assessment
methods can:
– Consist of General Outcome Measures or Specific Sub-Skill
Mastery Measures
– Include existing (‘extant’) data from the school system
Curriculum-Based Measurement (CBM) is widely used to track basic
student academic skills. Daily Behavior Report Cards (DBRCs) are
increasingly used as one source of formative behavioral data.
Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools:
Procedures to assure scientific-based practices. New York: Routledge.
www.interventioncentral.org
28
Response to Intervention
Curriculum-Based Measurement: Assessing Basic
Academic Skills
www.interventioncentral.org
Response to Intervention
Curriculum-Based Evaluation: Definition
•
•
•
•
“Whereas standardized commercial achievement tests measure
broad curriculum areas and/or skills, CBE measures specific skills
that are presently being taught in the classroom, usually in basic
skills. Several approaches to CBE have been developed. Four
common characteristics exist across these models:
The measurement procedures assess students directly using the
materials in which they are being instructed. This involves sampling
items from the curriculum.
Administration of each measure is generally brief in duration
(typically 1-5 mins.)
The design is structured such that frequent and repeated
measurement is possible and measures are sensitive to change.
Data are usually displayed graphically to allow monitoring of student
performance.”
SOURCE: CAST Website: http://www.cast.org/publications/ncac/ncac_curriculumbe.html
www.interventioncentral.org
30
Response to Intervention
RTI Literacy: Assessment & Progress-Monitoring (Cont.)
•
•
•
To measure student ‘response to instruction/intervention’ effectively,
the RTI Literacy model measures students’ reading performance and
progress on schedules matched to each student’s risk profile and
intervention Tier membership.
Benchmarking/Universal Screening. All children in a grade level are
assessed at least 3 times per year on a common collection of literacy
assessments.
Strategic Monitoring. Students placed in Tier 2 (supplemental)
reading groups are assessed 1-2 times per month to gauge their
progress with this intervention.
Intensive Monitoring. Students who participate in an intensive,
individualized Tier 3 reading intervention are assessed at least once
per week.
Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools:
Procedures to assure scientific-based practices. New York: Routledge.
www.interventioncentral.org
31
Response to Intervention
Curriculum-Based Measurement: Advantages as a Set of Tools to
Monitor RTI/Academic Cases
• Aligns with curriculum-goals and materials
• Is reliable and valid (has ‘technical adequacy’)
• Is criterion-referenced: sets specific performance levels for specific
tasks
• Uses standard procedures to prepare materials, administer, and
score
• Samples student performance to give objective, observable ‘lowinference’ information about student performance
• Has decision rules to help educators to interpret student data and
make appropriate instructional decisions
• Is efficient to implement in schools (e.g., training can be done quickly;
the measures are brief and feasible for classrooms, etc.)
• Provides data that can be converted into visual displays for ease of
communication
Source: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.
www.interventioncentral.org
32
Response to Intervention
Formative Assessment: Essential Questions…
4. What goal(s) are set for improvement?
Goals are defined at the system, group, or individual student level.
Goal statements:
–
–
–
Are worded in measureable, observable terms,
Include a timeline for achieving those goals.
Are tied to the formative assessment methods used to monitor progress toward
the goal(s).
www.interventioncentral.org
33
Response to Intervention
Interpreting Data: The Power of Visual Display
www.interventioncentral.org
Response to Intervention
Sample Peer Tutoring Chart
www.interventioncentral.org
35
Response to Intervention
Sample Peer Tutoring Chart
www.interventioncentral.org
36
Response to Intervention
Single-Subject (Applied) Research Designs
“Single-case designs evolved because of the need to
understand patterns of individual behavior in response
to independent variables, and more practically, to
examine intervention effectiveness. Design use can be
flexible, described as a process of response-guided
experimentation…, providing a mechanism for
documenting attempts to live up to legal mandates for
students who are not responding to routine instructional
methods.” p. 71
Source: Barnett, D. W., Daly, E. J., Jones, K. M., & Lentz, F.E. (2004). Response to intervention: Empirically based special
service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38, 66-79.
www.interventioncentral.org
37
Response to Intervention
Single-Subject (Applied) Research Designs: Steps
“The basic methods [of single-case designs] are
•
•
•
•
•
•
selecting socially important variables as dependent measures or target
behaviors
taking repeated measures until stable patterns emerge so that participants
may serve as their own controls (i.e., baseline)
implementing a well-described intervention or discrete intervention trials
continuing measurement of both the dependent and independent
variables within an acceptable pattern of intervention application and/or
withdrawal to detect changes in behavior and make efficacy attributions
graphically analyzing the results to enable ongoing comparisons of the
student’s performance under baseline and intervention conditions, and
replicating the results to reach the ultimate goal of the dissemination of
effective practices.”
Source: Barnett, D. W., Daly, E. J., Jones, K. M., & Lentz, F.E. (2004). Response to intervention: Empirically based special
service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38, 66-79.
www.interventioncentral.org
38
Response to Intervention
Formative Assessment: Donald: Grade 3
www.interventioncentral.org
39
Response to Intervention
Formative Assessment: Donald: Grade 3
www.interventioncentral.org
40
Response to Intervention
Formative Assessment: Essential Questions…
5. How does the school check up on progress
toward the goal(s)?
The school periodically checks the formative assessment data to
determine whether the goal is being attained. Examples of this
progress evaluation process include the following:
–
–
–
System-Wide: A school-wide team meets on a monthly basis to review the
frequency and type of office disciplinary referrals to judge whether those referrals
have dropped below the acceptable threshold for student behavior.
Group Level: Teachers at a grade level assembles every six weeks to review
CBM data on students receiving small-group supplemental instruction to
determine whether students are ready to exit (Burns & Gibbons, 2008).
Individual Level: A building problem-solving team gathers every eight weeks to
review CBM data to a student’s response to an intensive reading fluency plan.
Sources: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools:
Procedures to assure scientific-based practices. New York: Routledge.
Shinn, M. R. (1989). Curriculum-based measurement: Assessing special children. New York: Guilford.
www.interventioncentral.org
41
Response to Intervention
Team Activity: Formative Assessment and Your
Schools
At your tables, discuss:
•
What kinds of formative measures
in literacy that your schools tend to
collect most often.
•
How ‘ready’ your schools are to
collect, interpret, and act on
formative assessment data..
www.interventioncentral.org
42
Response to Intervention
Team Activity: CBM Literacy Assessments: First
Steps?
At your tables:
•
Discuss what process your school
might follow in selecting or
updating it’s academic screening
tools for RTI.
•
Think about the ‘first steps’ that
your school should adopt in the fall
to move forward with selecting or
updating RTI academic screening
tools.
www.interventioncentral.org
43
Response to Intervention
END
www.interventioncentral.org
44