Transcript Curriculum-Based Measurement and Its Use in a Problem

The Basics of Curriculum-Based
Measurement (CBM): Tools for
Measuring Skills and RTI
Presented by:
Mark R. Shinn, Ph.D.
Chief Scientist
[email protected]
http://markshinn.org
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.

Show why and how CBM was developed to
meet the needs of special education students
more than 30 years ago.

Present information on CBM and how it is used
in (universal) screening and progress
monitoring.

Address controversies in CBM and facilitating
implementation
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
2
Disclosure



Mark R. Shinn, Ph.D. Serves as a Consultant for
AIMSweb, which provides CBM assessment materials
and organizes and report the information from 3 tiers,
including RTI in the Role of Chief Scientist
Mark R. Shinn, Ph.D. Serves as a Consultant for
Glencoe Publishing for their Jamestown Reading
Navigator (JRN) product
Mark R. Shinn, Ph.D. Serves as a Consultant for
Vmath, a remedial mathematics intervention from
Voyager
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
3
For More Information, Explore
http://markshinn.org
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
4
Some Modest Credibility
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
5
Why and How
CBM Researchers Around 1981
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
6
Developed to Provide
Tools to Write IEP Goals
and Monitor Progress



Original Legislation (EACHA or PL 94-142) Required
“New Concept” of IEPs, Annual Goals, Progress
Monitoring
Best Available Technology (...with 80% Accuracy) Was
Not Defensible
Emerging Behavior Practices with the Importance of
Single Subject Methods and Graphing
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
7
1978
Minnesota Institute for Research on Learning Disabilities
(IRLD), Funded by Congress, Formalizes Stanley Deno’s
Research that Began in 1971 and Conceptualized in..
Deno, S. L., & Mirkin, P. (1977).
Data-based program
modification: A manual. Reston,
VA: Council for Exceptional
Children.
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
8
1979
Set of Criteria for Scientifically Based Progress
Monitoring Tools Established as Described in...
Jenkins, J. R., Deno, S. L., & Mirkin, P. K. (1979).
Measuring pupil progress toward the least restrictive
environment. Learning Disability Quarterly, 2, 81-92.
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
9
1979 Criteria



Psychometrically Sound
 Reliable
 Valid
Sensitive to Short Term Improvement
Easy to Do
 Easy to Learn
 Easy to Score
 Time Efficient
 Inexpensive
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
10
1979 Criteria


Note the Similarity of These Criteria to those
Adopted in 2003 by the US Department of
Education, Office of Special Education Programs
(OSEP) National Center for Student Progress
Monitoring
These 2003 Criteria Were Derived from APA,
AERA, and NCM Assessment Standards
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
11
Initial Outcomes: Progress
Monitoring Tools in




Reading
Written Expression
Spelling
Very Preliminary Work in Mathematics
Computation
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
12
Validated Progress Monitoring Measure:
A Standardized Measure of Oral Reading
(R-CBM)
It was a pretty good composition. I felt proud knowing it
was the best one at my school. After I’d read it five
times, I was impatient to start reading it out loud. I
followed the book’s directions again. First I read the
composition out loud without trying to sound impressive,
just to hear what the words sounded like. I did that a
couple of times. Then I moved over to my full-length
mirror and read the composition out loud in front of it a
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
few times. At first I just read it. Then I practiced looking
up and making eye contact. Of course I was making
eye contact with myself, and that felt pretty silly, but that
was what the book said to do. Then I went on to reading
the composition to an audience. This consisted of my
favorite teddy bear and Amanda, my best doll, the only
one I couldn’t bear to give up when I outgrew dolls last
Amy, A Typical 4th
Grade Student
year.
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
13
1979
Field Testing Begins, Primarily in Pine County
Special Education Cooperative, Minnesota
Initiated by Gary Germann, Special Education Director,
Who..
Became First Setting for Implementing What Became an RTI
Model
Retired 20 Years Later and Founded Edformation and
AIMSweb
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
14
1980
Practical Use of CBM Begins, Primarily in Pine
County Special Education Cooperative and St.
Paul MN and Minneapolis MN Schools
As Progress Monitoring Tool in Special Education
Applications
Screening and Entitlement Decisions
General Education “Benchmarking”
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
15
1981
First Refereed Journal Article Describing
Scientific Integrity
Deno, S. L., Mirkin, P., & Chiang, B. (1982). Identifying
valid measures of reading. Exceptional Children,
49(1), 36-45.
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
16
Fast Forward
Through the Years
Explosion of Research, Spearheaded by L.S. Fuchs
and Doug Fuchs, Among Numerous Others
Expansion of Research to Other Areas (e.g., Early
Literacy, Mathematics) and Other Measures (e.g.,
Maze)
Expansion of Practice to Use of CBM as Part of a
Comprehensive Assessment Model
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
17
See...
Fuchs, L. S., & Fuchs, D. (2008). Best practices in progress monitoring
reading and mathematics at the elementary level. In A. Thomas & J.
Grimes (Eds.), Best practices in school psychology V (pp. 2147-2164).
Bethesda, MD: National Association of School Psychologists.
Hosp, M. K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM: A practical
guide to Curriculum-Based Measurement. New York, NY: Guilford.
Miura Wayman, M., Wallace, T., Ives Wiley, H., Ticha, R., & Espin, C. (2007).
LIterature synthesis on curriculum-based measurement in reading.
The Journal of Special Education, 41(2), 85-120.
Shinn, M. R. (2008). Best practices in Curriculum-Based Measurement
and its use in a Problem-Solving model. In A. Thomas & J. Grimes
(Eds.), Best practices in school psychology V (pp. 243-262). Bethesda,
MD: National Association of School Psychologists.
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
18
CBM in 2008 Practice
The Big Ideas for Preventing Reading Failure in
Grades K-3:
1. Increase the quality, consistency, and reach of instruction
in every K-3 classroom
2. Universal Screening and Timely and Valid
Assessments of Reading Growth for Progress
Monitoring
3. Provide more intensive interventions to “catch up” the
struggling readers
Modified from J. Torgesen, www.fcrr.org
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
19
Challenges in Fulfilling
This Vision

Educators Have LOTS Of Ways of Testing Students in Reading

Not All These Ways of Testing Students Are Scientifically Based,
Particularly For Progress Monitoring

Until We Have a Scientifically Based, Efficient, and Uniform
Method for Assessing Response for ALL Students, Selecting and
Evaluating Interventions Will Be “Coke versus Pepsi”

It Would Be Desirable that the Tool(s) We Use to Monitor
Progress Also Could Be Used for Universal Screening
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
20
R-CBM Can Be Used for Screening and
Progress Monitoring
It was a pretty good composition. I felt
proud knowing it was the best one at my
school. After I’d read it five times, I was
impatient to start reading it out loud. I
followed the book’s directions again. First I
read the composition out loud without
trying to sound impressive, just to hear
what the words sounded like. I did that a
couple of times. Then I moved over to my
full-length mirror and read the composition
out loud in front of it a few times. At first I
just read it. Then I practiced looking up
and making eye contact. Of course I was
making eye contact with myself, and that
felt pretty silly, but that was what the book
said to do. Then I went on to reading the
composition to an audience. This consisted
of my favorite teddy bear and Amanda, my
best doll, the only one I couldn’t bear to
give up when I outgrew dolls last year.
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Amy, A Typical 4th
Grade Student
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
21
We Can Measure the
“Discrepancy” or Quantify
“Educational Need”
No Significant
Discrepancy
Educational
Need
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
22
Schools Use CBM in Universal
Screening Instead of Referral Driven
Practices
< 25th
Tier 2 Candidates
<10th
Individual Problem
Solving and/or
Tier 3 Candidates
Progress Monitoring Over a
Short Term for All Students
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
24
Or Even More Frequently
for Some Students
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
25
Provides Qualitative Information as
Well As Quantitative
Reads Efficiently
√
Reads Accurately--Greater than 95%
√
Reads with Expression and Prosody
√
Has Effective Strategy for Word They Don’t Know
√
Reading Errors Preserve Meaning Rather than
√
DISTORT Meaning
Engages in Comprehension Self-Monitoring (Self
Corrects)
√
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
26
Universal Screening and
More Severe Need
More Severe Educational Need
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
27
US Department of Education Review
Gives Us Confidence About Scientifically
Based
www.studentprogress.org
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
28
Most Tools that Meet
Standards are
Members of the
Curriculum-Based
Measurement (CBM)
“Family”
Progress Monitoring Across 3
Tiers
Tier 3 and Special Education:
Assess Progress Most Frequently
(1x-2x Per Week)
Tier 2 Assess Progress MORE Frequently (12x Per Month)
Tier 1: Assess Frequently Enough
for Universal Screening and Some
Progress (3x Per Year)
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
30
Begin By Building Quality Tier
3 Progress Monitoring
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
31
Tier 2: Strategic Monitoring
of At Risk
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
32
Monitoring Progress at Tier 1:
Benchmark Assessment to Measure
Educational Need and Benefit for All
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
33
Schools Use the Highly Similar Process
to Monitor a Specific Student’s “RTI”
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
34
Build a Data Base Across Years
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
35
Controversies and
Clarifications
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
36
What Does Oral Reading
(and Maze) Measure?
Phonemic Awareness?
ALL
These Skills
Alphabetic
Understanding?
Fluency?
General
Reading Skill
Vocabulary?
Comprehension?
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
37
The National
Reading Panel
Graphic
Phonemic Awareness
Phonics
Fluency
Vocabulary
Comprehension
http://www.nationalreadingpanel.org/
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
38
General Reading and
Comprehension




General Reading or “Word Reading” Skills are
Necessary, But Not Sufficient for Understanding
Comprehension is Dependent on What You’re Being
Asked to Read
Comprehension is Dependent on How It is Assessed
Comprehension Presumes Motivation and Interest
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
39
• Life Experience
• Content Knowledge
• Activation of Prior
Knowledge
And Here
• Knowledge about
Texts
Language
• Oral Language Skills
• Knowledge of Language
The Bigger
Deficits Here
Structures
• Vocabulary
• Cultural Influences
Fluency*
Reading
Oral
Reading
is
the
EASIEST
to
We Refer to It as
Knowledge
General Reading Skills
Measure--Let’s
Get
This
Down
and
Comprehension
And the MOST Unmotivated Here
For
Some,
the Hardest
Thing They’ll
Ever Do
Add MORE Tools
The
Easiest
Thing
To Teach
The Longer It Takes...
• Motivation &
Engagement
• Active Reading
Strategies
• Monitoring Strategies
• Fix-Up Strategies
Metacognition
• Prosody
• Automaticity/Rate
• Accuracy
• Decoding
• Phonemic Awareness
*modified slightly from presentations by Joe Torgesen,
Ph.D. Co-Director, Florida Center for Reading Research;
www.fcrr.org
CBM and DIBELS
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
41
DIBELS is Derived
from CBM



Most Research on DIBELS is based on the
Body of Knowledge regarding R-CBM
DIBELS was designed to be a downward
extension of CBM before reading
With the Exception of DORF (R-CBM put into
the Fluency “box”), All DIBELS Measures
are Short-Term or Mastery Measures
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
42
Because They Are
Short Term or
Mastery Measures...



Can Over-Test Children
May Test on a “Pre-Skill” When You Can
Test on a Higher-Level Skills
Can Be Logistically Overwhelming
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
43
Mark’s CBM
Suggestions




Use LNF as a Fall K Screener Only
Adopt LSF (Letter Sounds) as the Primary Tool
for K beginning in November
Target Reading Wholly or Highly Decodable
Passages as the Kindergarten Outcome
Use the DIBELS (or other measures)
Diagnostically if a Student Is Not Progressing
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
44
Norm vs Standards
Based Decision
Making
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
45
Standards-Based
Approaches
Images and Analyses Courtesy of Ben Ditkowsky, Ph.D.
[email protected]
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
46
Use Standards-Based
Approaches


For Program Evaluation
As One Goal Setting Method to Help Set
Goals to “Reduce the Gap”
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
47
Using StandardsBased for Program
Evaluation
Images and Analyses Courtesy of Ben Ditkowsky, Ph.D.
[email protected]
Use Norm-Based
Approaches


To Assess the Performance Discrepancy
(Educational Need)
Align Individual Students to Tiered Instruction
Based on the Performance Discrepancy
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
49
Consider Tier 2
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
50
Progress Monitoring
Material
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
51
Mark’s CBM
Suggestions
Read the Manual!
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
52
What Material is PM Material?
Potential Goal
(and PM) Material
Current Level of
Performance
Expected Level
of Performance
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
53

Show why and how CBM was developed to
meet the needs of special education students
more than 30 years ago.

Present information on CBM and how it is used
in (universal) screening and progress
monitoring.

Address controversies in CBM and facilitating
implementation
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
54
Let’s Go!
Copyright © 2008 Pearson Education, Inc. or its affiliates. All rights reserved.
55