Transcript Progress Monitoring & Curriculum Based Assessment

Progress Monitoring
Developed By:
Lehigh University
University of Pittsburgh
Parent Education Network
PA Training and Technical Assistance Network
Objectives: Participants will
Discuss the essential elements of progress
monitoring
Compare and contrast two approaches to
progress monitoring


General Outcomes
Specific Skills
Apply progress monitoring approach to:


Review progress toward objectives/outcomes
Inform instruction
7/20/2015
2
Rationale for Focus on Progress
Monitoring
Involvement of parents
Chapter 14


Facilitating annual IEP process and
Reevaluation process
Data-based decision making to drive instruction
 Student progress increases student achievement
 Specific feedback increases student achievement
7/20/2015
3
Progress Monitoring Cycle
Initial Assessment:
•Direct Intervention
•Historical Data
•Standardized Assessments
•Diagnostic Assessment
•Curriculum-Based Assessment
•Parent Input
Design Instruction:
• Develop Goals & Objectives
• Identify SDI
• Alignment to General Curriculum
• Instructional Grouping & Scheduling
• Identify Progress Monitoring
Ongoing Evaluation:
•Evaluate Effectiveness of Instruction by
Monitoring Progress on an Ongoing Basis
•Record and Use Data to Make Decisions &
Assess Progress
•Adjust Goals & Objectives
•Adjust SDI & Instruction As Needed
•Report to Parents
7/20/2015
Deliver Instruction:
•Deliver Instruction According to Goals &
Objectives Using SDI
•Collect Data on Progress
•Monitor Student Response & Feedback
4
Now Here’s a Thought….
We should measure what we
treasure.
7/20/2015
5
What is Progress Monitoring?
Progress Monitoring is the ongoing process
which involves:
Collecting and analyzing data to determine student
progress toward specific skills or general outcomes.
Making instructional decisions based on the review
and analysis of student data.
7/20/2015
6
The Goals of
Progress Monitoring
Provide data to assist in making decisions about
students.

To guide instructional decisions
Provide data on student performance.


To determine current level of learning/behavior/performance
To measure and report progress towards goals and
objectives/outcomes
Provide data for the reevaluation process.

To determine if the student still meets eligibility for special
education AND still needs specially-designed instruction
7/20/2015
7
Benefits of
Progress Monitoring
Parents and students know what is expected
Teachers have organized record of students’ performance
Teachers know what is working or not working with their
instruction based on data
“Easy to Understand” way to show parents the progress
IEP teams have comprehensive data on student
performance for decision making
And …
7/20/2015
8
Progress Monitoring:
A Team Responsibility
Parent
General Ed.
Teacher
Guidance
Counselor
Student
Related Service
Provider
School
Psychologist
Administrator
7/20/2015
Special Ed.
Supervisor
Paraeducator
Special Ed.
Teacher
9
Essential Elements of Progress
Monitoring
Measurable
goals/outcomes
Efficient and effective for
use in the classroom
Sensitive to
increments of
student growth
Usable format
Data driven
7/20/2015
Connected with general
ed. curriculum/appropriate
activities
10
Progress Monitoring:
Two Approaches
General Outcomes
Appropriate for all students,
including learning support
and at-risk students,
working in the general
education curriculum
(reading and math).
The goals and objectives
are based in the general
education curriculum.
7/20/2015
Specific Skills
Appropriate for all students
with IEPs, particularly those
with significant disabilities.
The goals and objectives are
referenced to the general
education curriculum.
11
Progress Monitoring: Two Approaches
General Outcomes - Examples
• The medical profession
measures height, weight,
temperature, and/or blood
pressure.
• Companies report earnings per
share.
• Wall Street measures the DowJones Industrial Average.
• McDonald’s measures how
many hamburgers they sell.
• General Outcomes approach
measures Oral Reading
Fluency.
7/20/2015
Specific Skills - Examples
• Bone density to measure
prediction of growth patterns.
• Cost of oranges, apples, and
bananas during the first week
of September at Giant.
• Price of Microsoft and Dell
stock over a two week period.
• Merck reports they sold X
number of Vioxx pills this
week.
• Specific Skills approach
measures segmenting and
blending sounds.
12
Progress Monitoring:
A General Outcomes
Approach
Dr. Naomi Zigmond, University of Pittsburgh
Dr. Edward Shapiro & Dr. Lana Edwards, Lehigh University
7/20/2015
13
Overview
Curriculum-Based Assessment and
General Outcomes Measures
Reading Assessment: Oral Reading
Fluency Tied to Comprehension

Administration and Scoring
Math Assessment: Computation Fluency

7/20/2015
Administration and Scoring
14
Curriculum-Based Assessment
A procedure for determining the
instructional needs of a student based
upon the student’s ongoing
performance within existing course
content.
7/20/2015
15
They are simple, accurate, and reasonably inexpensive
in terms of time and materials.
They are considered so important to doing business well
that they are routine.
They are collected on an ongoing and frequent basis.
They shape/inform a variety of important decisions.
7/20/2015
16
Models of CBA:
General Outcome Measurement
Standardized, Reliable, & Valid
Index growth in general curriculum over time
and across a wide range of skills
May or may not be measuring directly the
curriculum of instruction
Do suggest when instructional modifications are
needed
Do not specifically suggest instructional
modification
7/20/2015
17
Progress Monitoring
Reading
Assessment
7/20/2015
18
Training Overview
Why oral reading fluency?
How do you assess oral reading fluency?

Administration procedures and scoring
How do you document student progress?


LoTech (Pencil-and-Paper Graphing)
HiTech (AimsWeb System)
7/20/2015
19
Oral Reading Fluency
Provides formative assessment of student
performance –an important outcome and
indicator!
Provides formative miscue analysis
Allows teachers to assess qualitative features of
good reading
Allows for assessment of accuracy and fluency
Includes predictive validity (e.g., first grade
reading outcomes are strongly related to third
grade outcomes)
7/20/2015
20
Materials Needed
Unnumbered copy of passage (student copy)
Numbered copy of passage (examiner copy)
Stopwatch
Pencil
Colored Pencil (for indicating aimline on graphs)
Tape Recorder (optional)
7/20/2015
21
Oral Reading Fluency Directions
1. Place the unnumbered passage in front of the
student.
2. Place the numbered passage in front of you
but shielded so the student cannot see what
you record.
7/20/2015
22
Oral Reading Fluency Directions
(continued)
3. Say these specific directions to the student for the
first passage:
When I say “begin,” start reading aloud at the top of
the page.
Read across the page (DEMONSTRATE).
Try to read each word. If you come to a word you
don’t know, I’ll tell it to you. Be sure to do your best
reading. (pause)
7/20/2015
23
Oral Reading Fluency Directions
(continued)
4. Say “Begin” and start your stopwatch when the student
says the first word. If the student fails to say the first
word of the passage after 3 seconds, tell them the word
and mark it as incorrect, then start your stopwatch.
5. Follow along on your copy. Put a slash (/) through words
read incorrectly.
6. If a student stops or struggles with a word for 3 seconds,
tell the student the word and mark it as incorrect.
7. After 1-minute, place a bracket ( ] ) after the last word
and say, “Stop.”
7/20/2015
24
Helpful Scoring Hints
1. If students appear to understand the instructions
following the administration of the first passage, the
examiner need only point to the first word at the top
of the subsequent passage saying “Begin.”
2. If you completely lose track of where the student is
reading, discontinue the reading and begin another
passage.
3. Score reading passages immediately after
administration.
7/20/2015
25
Sample Passage - Examiner Copy
It was a pretty good composition. I felt proud knowing
it was the best one at my school. After I’d read it five times,
I was impatient to start reading it out loud.
I followed the book’s directions again. First I read the
composition out loud without trying to sound impressive, just
times. Then I moved over to my full-length mirror and read the
composition out loud in front of it a few times. At first I just
read it. Then I practiced looking up and making eye contact.
10
24
33
43
52
65
79
90
Total Words Read:
_______
Errors:
_______
Words Read Correctly: _______
7/20/2015
26
Sample Passage - Student Copy
It was a pretty good composition. I felt proud knowing
it was the best one at my school. After I’d read it five times,
I was impatient to start reading it out loud.
I followed the book’s directions again. First I read the
composition out loud without trying to sound impressive, just
times. Then I moved over to my full-length mirror and read the
composition out loud in front of it a few times. At first I just
read it. Then I practiced looking up and making eye contact.
7/20/2015
27
Scoring Reading Passages
What is a “word”?
example
cat
TW = 1
read as:
“cat”
WRC = 1
example
7/20/2015
I sat
TW = 2
read as:
“I sat.”
WRC = 2
28
What is a “correctly read word?”
Rule 1. Correctly Read Words are pronounced
correctly. A word must be pronounced
correctly given the context of the sentence.
example: The word “r-e-a-d” must be
pronounced “reed” when presented in the
context of:
He will read the book
WRC = 5
not as:
“He will red the book.”
7/20/2015
WRC = 4
29
What is a “correctly read word?”
Rule 2. Self-corrected Words are counted as
correct. Words misread initially but corrected
within 3 seconds, are counted as read correctly.
example:
The river was cold.
WRC = 4
read as:
The river was could...(2 seconds).. cold
WRC = 4
7/20/2015
30
What is a “correctly read word?”
Rule 3. Repeated Words are counted as
correct. Words said over again correctly
are ignored.
example:
Ted ran swiftly.
read as:
“Ted ran...Ted ran swiftly.”
7/20/2015
WRC = 3
WRC = 3
31
What is a “correctly read word?”
Rule 4. Dialect Variations in pronunciation
that are explainable by local language
norms are not errors.
example:
They washed the car.
read as:
“They warshed the car.”
7/20/2015
WRC = 4
WRC = 4
32
What is a “correctly read word?”
Rule 5. Inserted Words are ignored. When
a student adds extra words, they are not
counted as correct words nor as reading
errors.
example:
Sue was happy.
read as:
“Sue was very happy.”
7/20/2015
WRC = 3
WRC = 3
33
What is an “incorrectly read word?”
Rule 6. Mispronounced or Substituted Words
are counted as incorrect.
example:
The dog ate the bone.
read as:
“The dig ate the bone.”
7/20/2015
WRC = 5
WRC = 4
34
What is an “incorrectly read word?”
Rule 7. Omitted Words are counted as
errors
example:
Mario climbed the oak tree..
read as:
“Mario climbed the tree.”
7/20/2015
WRC = 5
WRC = 4
35
What is an “incorrectly read word?”
Rule 8. Hesitations When a student hesitates
or fails to correctly pronounce a word within 3
seconds, the student is told the word and an
error is scored.
example:
Mark saw an elephant.
read as:
“Mark saw an...(sec 3).”
or read as:
Mark saw an ell-ee...(3 sec)”
7/20/2015
WRC = 4
WRC = 3
WRC = 3
36
What is an “incorrectly read word?”
Rule 9. Reversals. When a student transposes
two or more words, those words not read in the
correct order are errors.
example:
Charlie ran quickly.
read as:
“Charlie quickly ran.”
7/20/2015
WRC = 3
WRC = 1
37
What is an “incorrectly read word?”
Rule 10. Numbers Written as numerals
are counted as words and must be read
correctly within the context of the
passage.
example:
May 5, 1999.
WRC = 3
read as:
“May five, one nine eight nine.” WRC = 1
7/20/2015
38
What is an “incorrectly read word?”
Rule 11a. Hyphenated Words. Each morpheme
separated by a hyphen(s) is counted as an
individual word if it can stand alone.
example:
Fifty-seven.
or:
“Daughter-in-law.”
7/20/2015
WRC = 2
WRC = 3
39
What is an “incorrectly read word?”
Rule 11b. Hyphenated Words. If one or
more of the morphemes separated by a
hyphen(s) cannot stand alone, the
entire sequence is counted as one
word.
example:
co-opt
or:
“re-evaluate.”
7/20/2015
WRC = 1
WRC = 1
40
What is an “incorrectly read word?”
Rule 12. Abbreviations are counted as words, and
must be read correctly within the context of the
sentence.
example:
Dr. Adams received a promotion.
should be read as:
“Doctor Adams received a promotion.”
not as:
“D-R Adams received a promotion.”
7/20/2015
WRC = 5
WRC = 5
WRC = 4
41
Qualitative Features of Good Reading
1. Is highly fluent (speed and accuracy)
2. Uses effective strategies to decode words.


effective word attack
context
3. Adjust pacing (i.e., slows down and speeds
up according to level of text difficulty)



7/20/2015
of word(s)
syntax (word order)
semantics (word meaning)
42
Qualitative Features of Good Reading
4. Attends to prosodic features.




inflection (pause, voice goes up and down)
reads with expression
punctuation (commas, exclamation points, etc.
predicts level of expression according to syntax
5. Possesses prediction-orientation


7/20/2015
seems to look ahead when reading
reads at a sentence or paragraph level
43
Qualitative Features of Good Reading
6. Self-monitors what she/he is reading.
 Self-corrects if makes meaning distortion errors
7. Makes only meaning preservation errors.
 more errors that preserve meaning (e.g., “house” for
“home”)
 fewer meaning distortion errors (e.g., “mouse” for
“house.”)
8. Automaticity on reread words.
 words that appear throughout text are read
automatically (e.g., become “sight words”)
7/20/2015
44
Practice Example:
Administration and Scoring
It was a pretty good composition.
I felt proud knowing it was the best
one at my school. After I’d read it
five times, I was impatient to start
reading it out loud. I followed the
book’s directions again. First I read
the composition out loud without
trying to sound impressive, just
times. Then I moved over to my
full-length mirror and read the
composition out loud in front of it a
few times. At first I just read it.
Then I practiced looking up and
making eye contact.
* See Training Practice Packet for examples.
7/20/2015
45
Graphing Data
1. Establish baseline: Median of three passages
administered on the same day
Passage 1:
32 wcpm
Passage 2:
40 wcpm
Passage 3:
27 wcpm
2. Set up graph:
•
Write student name on the graph;
•
Mark vertical axis with NUMBER OF WORDS READ CORRECTLY;
•
Mark horizontal axis with TESTING SESSIONS/DATE;
•
Mark baseline score on vertical axis in pencil
7/20/2015
46
Graphing Data
3. Set Goal: For 2 word per week improvement over
baseline (one word per test session; mark goal at last
testing session for the project)
4. Draw Aimline: In colored pencil, draw line from
baseline to goal
5. Measure Student Performance: Twice per week
6. Plot Student Performance: On graph in pencil
7/20/2015
47
Graphing Data
7. Connect Indicators of Student Performance: to
show trend line
8. Analyze Student Performance: Show progress, by
checking last six data points: if four of the last six points
are below aimline, student is not making adequate
progress
9. Make Instructional Changes: If indicated by student
performance
10. Continue to Measure and Monitor Student
Performance
7/20/2015
48
Sam
40
40
39
38
37
36
wcpm
35
34
33
32
31
30
29
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
49
Graphing Practice # 1
Kyle
•Baseline:
Passage 1:
Passage 2:
Passage 3:
53 wcpm
60 wcpm
55 wcpm
•Determine Goal and Draw Aimline
•Graph ORF Scores:
Session 1:
Session 2:
Session 3:
Session 4:
Session 5:
Session 6:
Session 7:
Session 8:
7/20/2015
53
58
60
61
55
61
63
64
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
50
64
Kyle
63
63
62
61
60
wcpm
59
58
57
56
55
54
53
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
51
Graphing Practice # 2
Carrie
•Baseline:
Passage 1:
Passage 2:
Passage 3:
15 wcpm
17 wcpm
12 wcpm
•Determine Goal and Draw Aimline
•Graph ORF Scores:
Session 1:
Session 2:
Session 3:
Session 4:
Session 5:
Session 6:
Session 7:
Session 8:
7/20/2015
13
14
14
12
17
19
15
19
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
wcpm
52
23
Carrie
23
22
21
20
19
wcpm
18
17
16
15
14
13
12
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
53
Consider a “HiTech”
Approach. . .
7/20/2015
54
Example Graph from AimsWEB
(www.edformation.com)
Slope/Trend
Aimline
7/20/2015
55
Progress Monitoring
Math
Assessment
7/20/2015
56
Training Overview
Why math computation and a digits correct per
minute scoring system?
How do you assess basic math computation
skills?

Administration procedures and scoring
How do you document student progress?

7/20/2015
Pencil-and-Paper Graphing
57
Monitoring Basic Skills in Math Computation
Provides formative assessment of student performance
Allows for assessment of accuracy and fluency
Provides formative error analysis
Focuses on computational objectives (the foundation for
success in other aspects of mathematics)
Includes probes designed to reflect the scope and
sequence of the mathematics curriculum
7/20/2015
58
Scoring Digits Correct: A Rationale
Sensitive metric for measuring progress
Can identify types of errors with this scoring
system



Substitution
Omission of numbers or steps in a problem
Insertion of digits in the steps or in the answer
. . .Scoring a problem as correct or incorrect
does not address a child’s gradual acquisition
of skills across time.
7/20/2015
59
Materials Needed
Student Probe
Answer Key (Digits Correct Scoring Guide)
Stopwatch
Pencil
Colored Pencil (for indicating aim line on graphs)
7/20/2015
60
Math Probe Directions
1. Place the math probe in front of the student.
2a. Say these specific directions to the student for the
first probe:
Today we are going to learn about a math test that you will take
every week. This test has all kinds of math problems you are
learning to do this year. This is what the test looks like. (Hold up a
test).
This test has 25 problems. You may not know how to do some of
the problems. That’s OK. . .Just start with problems that you know
and do them first. Then work on the more difficult problems. Try
to do your best work. . . etc.
7/20/2015
61
Math Probe Directions (continued)
2b. Say these specific directions to the student for the
remaining probes:
It is time to take your weekly math test. As soon as I give your
test, write your first name, your last name, and the date. After
you’ve written your name and your date on the test, turn your
paper over and put your pencil down so I’ll know you are ready.
Modify the directionsI want you to do as many problems as you can. Remember, start
with problems that are easier for you. Then do the more difficult
ones. Try to do your best work. Ready. . .Turn your papers over and
begin [start timer].
7/20/2015
62
Math Probe Directions (continued)
3. Say “Begin” and start your stopwatch when the student
starts the first problem.
4. Time Limit:
Grade 1:
Grade 2:
Grade 3:
Grade 4:
Grade 5:
Grade 6:
2
2
3
3
5
6
minutes
minutes
minutes
minutes
minutes
minutes
5. After the designated time is completed, say “Stop” and
place a bracket ( ] ) after the last problem solved.
7/20/2015
63
Scoring
Count the number of digits correct and incorrect per
minute (For all skills except long division, only digits
below the line are counted).
If the child does not complete the full probe before time
is up. . .
Digits Correct
= Digits Correct Per Minute
# of MinutesTimed For Probe
Example:
30 Digits Correct
= 10 Digits Correct Per Minute
3 Minutes (e.g., 3rd grade)
7/20/2015
64
Scoring
• If the child finishes the probe before the time is up Digits Correct
# of Min. Timed For Probe
X
Total Sec. Timed for the Probe = Digits per Min.
Total Sec. Used to Complete Probe
Example:
3rd Grade, 3 Minute Probe (180 Seconds)
Digits Correct = 30
Completed Probe in 153 Seconds
30
3
7/20/2015
X
180
153
= 11.7 digits correct per minute
65
Scoring Digits Correct:
Addition and Subtraction Problems
Each CORRECT digit in the answer is
counted.
Use a “Number to Number Matching”
approach to score digits correct.
7/20/2015
66
Scoring Digits Correct for
Addition and Subtraction Problems
EXAMPLES:
12
+4
16 (2 digits correct)
54
-27
27 (2 digits correct)
7/20/2015
145
+672
817 (3 digits correct)
2675
-1089
1586 (4 digits correct)
67
Scoring Digits Correct:
Addition and Subtraction Problems
STUDENT WORK:
ANSWER KEY:
15
-9
15
-9
6
6
76
+17
76
+17
81
93
TOTAL:
7/20/2015
SCORING:
Digits Correct
Incorrect
1
0
0
2
1
2
68
Scoring Digits Correct:
Addition and Subtraction Problems
STUDENT WORK:
ANSWER KEY:
80
-4
80
-4
84
76
15
+66
15
+66
81
81
TOTAL:
7/20/2015
SCORING:
Digits Correct
Incorrect
0
2
2
0
2
2
69
Scoring Digits Correct:
Multiplication Problems
STUDENT WORK:
ANSWER KEY:
75
x 26
75
x 26
450
150_
450
150_
2850
1950
SCORING:
Digits Correct
9
2
OR
2
7/20/2015
Incorrect
2
70
Scoring Digits Correct:
Multiplication Problems
STUDENT WORK:
ANSWER KEY:
75
x 26
75
x 26
435
285_
450
150_
3415
1950
7/20/2015
SCORING:
Digits Correct
4
Incorrect
7
OR
0
4
71
Scoring Digits Correct:
Multiplication Problems
STUDENT WORK:
ANSWER KEY:
550
x 66
550
x 66
3300
1216_
3300
3300_
15460
36300
SCORING:
Digits Correct
6
8
OR
1
7/20/2015
Incorrect
4
72
Scoring Digits Correct:
Multiplication Problems
STUDENT WORK:
ANSWER KEY:
550
x 66
550
x 66
3300
1216_
3300
3300_
15460
36300
SCORING
Digits Correct
Incorrect
10
4
OR
1
7/20/2015
4
73
Scoring Digits Correct: Division Problems
• Count the digit incorrect only if the incorrect operation
is performed or if incorrect place values are used.
• When a remainder is present, the student is given an
additional digit correct for indicating a correct remainder
next to the answer.
• The student is not given credit for remainders of 0, nor
is she/he given credit for the addition of unnecessary
digits.
7/20/2015
74
Scoring Digits Correct:
Division Problems
STUDENT WORK:
283
25 7375
50
237
200
75
50
25
7/20/2015
ANSWER KEY:
295
25 7375
50
237
225
125
125
0
SCORING
Digits Correct
7
OR
1
75
Scoring Digits Correct:
Division Problems
STUDENT WORK:
1410 R2
3 452
3
15
12
32
30
2
7/20/2015
ANSWER KEY:
3
150 R2
452
3
15
15
02
0
2
SCORING
Digits Correct
4
OR
2
76
Scoring Digits Correct: Fractions
• The choices available to solve fraction
problems can create difficulty when
scoring by digits correct
6 + 12 = 12 + 12 = 24 = 1
12 24
24 24 24
OR
6 + 12 = 1 + 1 = 1
24
2 2
7/20/2015 12
77
Scoring Digits Correct: Fractions
• Score the answer according to the most specific
and extensive procedure with all possible
intervening steps. Score the student’s work as
the percentage of digits computed correctly
• Example:
6 + 12 = 12 + 12 = 24 = 1
12 24 24 24 24
• Example:
200 - 25 = 165 =
3
9
18 18 18
18
7/20/2015
(13/13 digits correct)
(8/10 digits correct)
78
Scoring Digits Correct: Fractions
• Score students’ work by giving credit for their
solution to the furthest point that is consistent
with the answer key or expressed as equivalent
form.
• To resolve discrepancies in working fraction
problems, include directions for administering
the assessment that informs students of the
scoring procedures.
7/20/2015
79
Scoring Digits Correct: Decimals
• Scoring strategies are the same as those
described for addition, subtraction,
multiplication, and division.
7/20/2015
80
Practice Example:
Administration and Scoring
* See Training Practice Packet for examples.
7/20/2015
81
Graphing Data
1. Establish baseline: Median of three probes
administered on the same day
Probe 1:
14 digits correct per minute
Probe 2:
15 digits correct per minute
Probe 3:
10 digits correct per minute
2. Set up graph:
•
Write student name on the graph;
•
Mark vertical axis with “NUMBER OF DIGITS CORRECT PER
MINUTE”
•
Mark horizontal axis with TESTING SESSIONS/DATE;
•
Mark baseline score on vertical axis in pencil
7/20/2015
82
Graphing Data
3. Set Goal: For ____ digit per week improvement over
baseline (____ digit per test session; mark goal at last
testing session for the project)
4. Draw Aimline: In colored pencil, draw line from
baseline to goal
5. Measure Student Performance: Twice per week
6. Plot Student Performance: On graph in pencil
7/20/2015
83
Graphing Data
7. Connect Indicators of Student Performance: to
show trend line
8. Analyze Student Performance: Show progress, by
checking last six data points: if four of the last six points
are below aimline, student is not making adequate
progress
9. Make Instructional Changes: If indicated by student
performance
10.Continue to Measure and Monitor Student
Performance
7/20/2015
84
Sam
22
22
21
20
19
18
Digits correct 17
per minute 16
15
14
13
12
11
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
85
Graphing Practice # 1
Kyle
•Baseline:
Probe 1:
Probe 2:
Probe 3:
12 digits correct per minute
9 digits correct per minute
11 digits correct per minute
•Determine Goal and Draw Aimline
•Graph Digits Correct Per Minute:
Session 1:
11 digits correct
Session 2:
12 digits correct
Session 3:
15 digits correct
Session 4:
13 digits correct
Session 5:
16 digits correct
Session 6:
18 digits correct
Session 7:
17 digits correct
7/20/2015
Session 8:
18 digits correct
per
per
per
per
per
per
per
per
minute
minute
minute
minute
minute
minute
minute
minute
86
20
Kyle
19
19
18
17
16
15
Digits correct
14
per minute
13
12
11
10
9
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
87
Graphing Practice # 2
Carrie
• Baseline:
Probe 1:
Probe 2:
Probe 3:
22 digits correct per minute
20 digits correct per minute
24 digits correct per minute
• Determine Goal and Draw Aimline
• Graph ORF Scores:
Session 1:
Session 2:
Session 3:
Session 4:
Session 5:
Session 6:
Session 7:
7/20/2015
Session 8:
23
25
24
27
30
29
27
32
digits
digits
digits
digits
digits
digits
digits
digits
correct
correct
correct
correct
correct
correct
correct
correct
per
per
per
per
per
per
per
per
minute
minute
minute
minute
minute
minute
minute
minute
88
32
Carrie
30
31
30
29
28
27
Digits correct
26
per minute
25
24
23
22
21
7/20/2015
1
2
3
4
5
Testing Session
6
7
8
89
For Reading or Math Assessment. . .
Students may graph
their own data.
7/20/2015
90
Progress Monitoring
Making Instructional
Decisions: Briefly
7/20/2015
91
Interpreting Data: Making
Instructional Decisions
Summarizing Performance
Decision-Making


7/20/2015
Goal-oriented decisions
Intervention-oriented decisions
92
Guidelines For Summarizing
Student Performance
At least three characteristics of
graphed data can be used to describe
and summarize student performance:
(a) Level of performance
(b) Slope of performance
(c) Variability of performance
7/20/2015
93
Example: Levels of Performance
100
90
80
70
Test Score
60
50
40
The difference between
these two lines is the
change in level.
30
20
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16 94 18
Example: Slope/Trend
Aimline
Slope/Trend
7/20/2015
95
Example: Variability
100
90
80
70
Test Score
60
50
40
30
20
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16
18
96
Guidelines for Instructional
Decision Making
1. Goal-Oriented Approach

Emphasis on attaining a goal and program
modifications occur only when attainment of
the goal is in doubt.
2. Intervention-Oriented Approach

“Experimental” analysis of behavior.
7/20/2015
97
Goal-Oriented Decision
Making
A consistent rule is to make program changes
when performance falls below the aimline (for
accelerating behaviors) or above the aimline
(for decelerating behaviors) for 3 consecutive
days.
Using aimlines and decision rules takes much
of the guess work out of data analysis.
7/20/2015
Aimline
98
Example Decision Rules Are. . .
If a student’s performance is below the
aimline on 3 consecutive days, but is parallel
to the aimline, one may decide to “wait” to
see if student performance accelerates in
level to reach the original aimline.
If the student performance continues below
the original aimline, implement a different
teaching strategy. Draw a vertical line of the
graph that symbolizes a change in the
program.
7/20/2015
99
Example Decision Rules Are. . .
If the student’s performance is above the
aimline after 3-days (or wait another 3-days),
 it may be appropriate to raise the aimline.
Draw a line parallel to and above the
aimline.
(A variation of the 3-day rule is to compare the
slope of actual improvement for the most
recent 7-10 day period to the slope of
expected improvement, which is reflected by
the aimline).
7/20/2015
100
Example: Decision Rules
100
90
Baseline
Intervention 1
Intervention 2
80
60
Test Score
X
50
40
Aim line
30
20
X
10
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16
18
101
100
Intervention 1
Baseline
90
80
X
70
Test Score
60
Time
to to
make
- Time
make
aachange.
change.
50
40
X
30
20
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16
18
102
Intervention-Oriented Decision Making
Intervention-oriented decision making
has no formal rules for determining when
to change programs, but has some
guidelines.
Generally, changes between adjustment
phases are analyzed.
7/20/2015
103
Example Guidelines Are. . .
Some time after 5 days (which is the
minimum number for evaluating an
intervention) but no longer than 15 days (the
maximum number of days for allowing any
program to run), analyze the data and
develop a program change.
Purpose is to compare data before the
intervention to that following the intervention
using analyses of (a) change in level, (b)
slope, and (c) variability.
7/20/2015
104
Example: Intervention-Oriented
55
50
Intervention 1
Intervention 2
45
40
35
Test Score
30
25
20
15
10
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16105 18
Example: Intervention-Oriented
65
60
Intervention 1
Intervention 2
55
50
45
Test Score
40
35
30
25
20
0
2
7/20/2015
4
6
8 10
12
Successive Days
14
16106 18
Some Valuable Resources
•Great site for intervention suggestions
http://www.interventioncentral.org/
•Site for developing probes for assessing math computation
http: //www.lefthandlogic.com/htmdocs/tools/mathprobe/addsing.shtml
•Site for assessing beginning reading skills
http://dibels.uoregon.edu/
•Site for conducting progress monitoring for reading
http://www.edformation.com/
7/20/2015
107
Things to Take Away…
Indicators of academic performance reflect
academic skill development
Data needs to be collected on a frequent
(2x/week) basis for good decision-making
Look at level, slope, variability of data
Use decision rules when interpreting data
Use knowledge of best teaching practices to
develop good interventions
7/20/2015
108
Let the Data Drive
the Train!
7/20/2015
109
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
110
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
111
Step 1. Measurable Goals:
Just Checking
Condition
Student’s name
Clearly defined behavior
Performance criteria
Example
RWLS Standard 1.5.3.F. Edit writing using the conventions of language,.
Spell common, frequently used words correctly.
Given a random selection of 20 words from a pool of
100 words with a CVC, CVCC, CVCe, CCVCC, or VCe
pattern, Eva will spell 95% of the words correctly on
three consecutive weekly spelling tests.
7/20/2015
112
Performance Criteria Examples
% of time
# times/#times
with # or % of accuracy
with no more than # words
words/digits/ per minute
with “x” movement on a
prompting hierarchy
independently
7/20/2015
with a grade of “x” or
better
“x” or better on a rubric
with no more than “x”
occurrences of…
with an “x” or better on
“x” rating scale
with “x/x” points on an
assessment checklist
113
Step 1. Measurable Goals:
I do!
Non-Measurable Example
John will decrease calling out in class.
Does this goal include:
 Condition?
 Student’s name?
 Clearly defined behavior?
 Performance criteria?
Measurable Example
During reading class, John will increase hand
raising and waiting to be called on by the teacher
to 10 times per day for five consecutive days using
event recording.
7/20/2015
114
Step 1. Measurable Goals: We Do!
RWLS Standard 1.1.5. C. : Use knowledge of phonics, syllabication,
prefixes, suffixes, the dictionary or context clues to decode and
understand new words during reading.
Non-Measurable Example:
Vicki will use decoding skills while reading.
Does this goal include:
 Condition?
 Student’s name?
 Clearly defined behavior?
 Performance criteria?
Measurable Example:
7/20/2015
115
Short-Term Objectives
Should include:
Condition
Student’s name
Clearly defined behavior
Performance criteria
7/20/2015
116
Short Term Objectives - Construction
Sequential
Target Important Skills
Construct Short Term Objectives so they are
based on a sequence or set of skills that build
toward the annual goal.
7/20/2015
117
Step 1. Measurable Goals:
You Do!
Refer to the Seven-Step Worksheet…

Create a measurable goal and objective.

Make sure it meets all criteria.
7/20/2015
118
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
119
Step 2. Data Collection Decisions
What is the purpose for
collecting the data?
Where will the data be
collected?

What type of data will
be collected?







7/20/2015
Frequency or Rate
Percentage or Accuracy
Duration
Latency
Quality
Level of Assistance
Number
Consider in what
settings/situations the skill
will be practiced.
How often will the data
be collected?
Who will collect the data?
120
Step 2: Type of Data
•Frequency: used to measure how many times a
student is engaged in a behavior.
•Sue will __________ at least 3 times daily…
•Sue will __________ at least once per 5 min. interval
observed…
•Sue will ___________ at least 3 times per social peer
interaction…
7/20/2015
121
Step 2: Type of Data
•
Percentage Correct: Used to measure how
often/how many times a student is engaged in a
behavior.
•Jim will ___________ 70 % of opportunities…
•Jim will ___________ with 90% accuracy…
•Jim will ___________ with an average of 80% across 5
consecutive opportunities …
7/20/2015
122
Step 2: Type of Data
•Duration: used to measure the length of time a
student engages in a particular behavior.
•Barb will ___________ for 5 minutes…
•Barb will ___________ throughout circle time…
•Barb will ___________ during __________ …
7/20/2015
123
Step 2. Data Collection Decisions:
I do!
Example
RWLS Standard 1.1.3. C. Use knowledge of phonics, word
analysis, syllabication, picture and context clues to decode
and understand new words during reading.
Given multi-syllabic (2-4 syllables) words, Eddie
will identify, by clapping, the number of
syllables in words presented orally ten out of
ten times on three consecutive probes.




Type of data – Percentage Correct/Accuracy
Where – Reading group
How often – 3 times per week
Who – Classroom teacher
7/20/2015
124
Step 2. Data Collection Decisions:
We do!
Example
ECLCI: 1.6.A Listens in order to understand and follow
directions.
With the consistent use of visual schedules and
cues, Mike will independently transition from
one routine activity to the next at school and
home (dinner to bedtime) 4 of 5 transitions on
five consecutive trials.
Type of data –
 Where  How often –
7/20/2015
 Who 
125
Step 2: Data Collection Decisions:
You do!
Refer to the Seven-Step Worksheet

Using the objectives you developed, make
your data decisions!
 Type of data –
 Where –
 How often –
 Who –
7/20/2015
126
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
127
Step 3: Data Collection Tools & Review
Schedule
Data Decisions
 Type of data needed
 Where, by whom, and how often will data
be collected
Guide your selection of a meaningful data
collection tool
7/20/2015
128
A Sampling of Tools
Examples of tools include…




Structured Interviews
or Surveys
Observations
Teacher/Standardized
Test
Rating Scales/
Assessment Checklists






7/20/2015
Task Analytic
Recording
Portfolio Assessment
Curriculum Based
Assessment
Anecdotal Records
Incident Record
Rubrics
129
Review Schedule
Daily, weekly, monthly
or quarterly monitoring
activities are driven by



7/20/2015
Goals and objectives
(what you want students
to know and be able to
do)
The data collection
method and tools
Knowledge of student’s
learning profile
130
Step 3. Data Collection Tools &
Review Schedule: I Do!
Example
Given 2-4 syllable words,
Eddie will identify, by
clapping, the number of
syllables in words presented
orally with 90% accuracy on 3
consecutive probes.





Type of data – Percentage
Correct/Accuracy
Where – Reading group
How often – 3 times per
week
Who – Classroom teacher
Tool – Percentage/Accuracy
Chart for Syllable
Segmentation Performance
7/20/2015
Percentage/Accuracy Chart for
Syllable Segmentation
performance
Child
Monday
Wednesday
Friday
Joy
2 - syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
3- syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
4- syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
Eddie
2 - syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
3 - syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
4 - syllable
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
+/- +/- +/+/- +/-
131
Step 3. Data Collection Tools &
Review Schedule: We Do!
Example
With the consistent use of visual
schedules and cues, Mike will
independently transition from one routine
activity to the next at school and home
(dinner to bedtime) 4 of 5 transitions on
five consecutive trials.
 Type of data –
 Where –
 How often –
 Who –
 Tool –
7/20/2015
Describe your
data collection
tool.
132
Step 3. Data Collection Tools &
Review Schedule: You Do!
Refer to the Seven-Step Worksheet


Select a data collection tool based on the goal
and objective written.
Describe how often the data will be reviewed.
7/20/2015
133
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
134
Step 4. Representing the Data
Why Represent Data Visually?

Communicate program effectiveness to the
teacher, parents, student, etc. about
 Instruction
 IEP
 Reevaluation


7/20/2015
Provide reinforcement and feedback
Make decisions about continuing or improving
instructional practices
135
Step 4. Representing the Data
Appropriate representation requires a
graph to be:

Simple

Stand alone

Understandable
7/20/2015
136
Step 4. Representing the Data
Most common types of graphs used
include:

Line Graph – used to reveal trends over time

Bar Graph – used to compare sets of values
7/20/2015
137
Step 4. Representing the Data:
I Do!
Can you determine any
patterns or trends by the way
the data is represented?
Can you determine any
patterns or trends by the way
the data is represented?
100
W
8/10
10/10
8/10
F
10/10
8/10
9/10
80
Percent of Time on Task
M
Wk.1 6/10
Wk.2 6/10
Wk.3 6/10
90
70
60
50
40
30
20
10
0
Monday
Wednesday
Week 1
7/20/2015
Friday
Week 2
Week 3
138
Step 4. Representing the Data:
We Do!
Example
Given a random selection of 20 (CVC or CVCC)
words presented orally from a pool of 100 words,
Eddie will produce the initial sound with 100%
accuracy on three consecutive weekly probes.
How will you
represent
Eddie’s data?
Data collected on correct sounds:
11/14 – 14 (baseline median)
11/20 – 12
11/28 – 16
12/4 – 14
12/11 – 12
12/19 – 18
1/16 - 20
1/22 - 18
1/28 – 20
7/20/2015
139
N
ov
.1
N 4
ov
.
N 20
ov
.2
D 8
ec
D .4
ec
.1
D
ec 1
.1
9
Ja
n.
Ja 6
n.
1
Ja 3
n.
20
Number
Eddie’s Data
20
18
16
14
12
10
8
6
4
2
0
7/20/2015
Number correct
140
Step 4. Representing the Data:
You Do!
Refer to the Seven-Step Worksheet
Determine what type of graph will best
represent the data you are collecting.
7/20/2015
141
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
142
Step 5. Evaluate the Data
Assumptions to consider before evaluating
the data

Teacher is implementing instructional strategies

Student is actively engaged in instruction


Aim line has been correctly identified based on
present levels of educational performance
Appropriate identification of annual goals and
objectives
7/20/2015
143
Step 5. Evaluate the Data
Why evaluate the data?

Effectiveness and efficiency of instruction
Who should evaluate the data?

7/20/2015
Teacher, Parents, Student, Others
144
Step 5. Evaluate the Data
What to look for when evaluating the data…

Is the student making progress toward the goals and
objectives?
 “Rule of Thumb” – If 4 of the last 6 data points fall below
aim line, student is not making progress




Baseline data
Established timeline
Established aim line
How is the student responding to the intervention?
 Specially designed instruction
 Instructional materials and methods
7/20/2015
145
Step 5. Evaluate the Data:
Additional Decision Rules
Data Patterns
Making progress.
Errors flat or decreasing
Progress stalled at
20%-50% correct.
Progress at or near
zero. High error
rate.
Interpretation
Program is working
Student can
perform some but
not all parts of the
task.
Task is too difficult.
Suggestion
• Continue present
instructional program.
• Provide more direct or
intensive instruction in
difficult steps.
• Teach prerequisite
skills.
Adapted from Wolery, 1988
7/20/2015
146
Step 5. Evaluate the Data:
Additional Decision Rules
Data Patterns
Interpretation
Suggestion
Progress stalled close
to goal, no increase in
rate
Student is ready
for fluency building
Provide frequent
Meets aim line
Successful
instructional
program
Implement
opportunities for practice
to increase accuracy and
rate.
maintenance and
generalization programs.
Move on to new task.
Adapted from Wolery, 1988
7/20/2015
147
Step 5. Evaluate the Data:
I Do!
Example: Katie will remain on task during independent work
periods 80% of the time on three consecutive probes as
measured by time sampling three times a week.
100
90
Aim Line
Percent of Time on Task
80
70
60
50
40
30
20
10
0
M onday
Week 1
7/20/2015
Wednesday
Week 2
Friday
Week 3
Week 4
148
Step 5. Evaluate the Data: We Do!
Example
Given double-digit addition and
subtraction problems that use regrouping,
Jeff will complete the problems with 90%
accuracy on three consecutive weekly
worksheets.
Let’s Look at the Data
7/20/2015
149
Is Jeff Making Progress?
100
80
60
40
20
W
ee
k
10
9
W
ee
k
8
W
ee
k
7
W
ee
k
6
W
ee
k
5
W
ee
k
4
W
ee
k
3
W
ee
k
2
ee
k
W
W
ee
k
1
0
Percent correct
7/20/2015
150
Step 5. Evaluate the Data: You Do!
Refer to the Seven-Step Worksheet
Brainstorm two questions the team
should anticipate when evaluating the
data.
7/20/2015
151
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
152
Step 6. Instructional Adjustments
If the student IS making progress…



Celebrate!
Keep doing what you are doing
Consider increasing expectations as
appropriate
 Increasing accuracy/Decreasing errors
 Increasing automaticity
 Applying knowledge independently
7/20/2015
153
Step 6. Instructional Adjustments
If the student IS NOT making progress

FIRST consider changes in intervention
strategies
 Specially Designed Instruction (intensity, duration,
frequency)
 Materials
 Methods
 Reinforcement schedule
 Use of peers
7/20/2015
154
Step 6. Instructional Adjustments
If changes do not positively effect
progress


Consider changing goals and objectives
Don’t make major changes without input or
a meeting with the entire IEP team
7/20/2015
155
Step 6. Instructional Adjustments:
I Do!
Example
Using graphic organizers, Stephanie will take
notes during all class lectures in Science,
Algebra, and World Cultures with a score of
9 or higher on three consecutive graphic
organizer rubrics as measured every other
week.
Progress data on next slide.
7/20/2015
156
9/
15
/2
00
9/
2
30
/2
10
00
/1
2
5/
20
10
02
/3
0/
20
11
02
/1
4/
20
11
02
/2
9/
20
12
02
/1
4/
20
12
02
/2
9/
20
1/
02
13
/2
00
3
Stephanie’s Data
100
90
80
70
60
50
40
30
20
10
0
7/20/2015
Science
Algebra
World Cultures
157
Step 6. Instructional Adjustments:
I Do!
Example
What instructional adjustments are necessary?


7/20/2015
Provide direct instruction on how to use
guided notes to write down key points of
class lectures in algebra and world cultures.
Examine strategies Stephanie uses in
science class and help Stephanie apply
these to algebra and world cultures classes.
158
Step 6. Instructional Adjustments:
We Do!
Example
Annie will write complete sentences that include a noun, verb,
object, and correct punctuation 4 out of 5 times on five
consecutive weekly writing probes.
Instructional Strategies Implemented
1) Annie’s classroom teacher provided her with written examples of
complete sentences and required Annie to identify the noun, verb,
object, and punctuation marks. The teacher also had Annie
identify sentences as complete or incomplete within.
2) Data was collected during lessons provided weekly by the reading
specialist.
7/20/2015
159
Step 6. Instructional Adjustments:
We Do!
Is Annie making progress?
5
4
3
Number
Correct
2
1
0
W
7/20/2015
k
ee
1
W
k
ee
2
W
k
ee
3
W
k
ee
4
W
k
ee
5
W
k
ee
6
W
k
ee
7
W
k
ee
8
W
k
ee
9
160
Step 6. Instructional Adjustments:
We Do!
Example
What instructional adjustments are needed for Annie?
7/20/2015
161
Step 6. Instructional Adjustments:
You Do!
Refer to the Seven-Step Worksheet
Brainstorm effective instructional
strategies for …


Progress, as appropriate
No progress
7/20/2015
162
Progress
Monitoring:
A Specific Skills
Approach
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
6. Instructional Adjustments
7. Communicating Progress
7/20/2015
163
Step 7. Communicating Progress
Communication is a two-way street


Home issues that effect student progress
Class/instructional issues that effect progress
Ways to keep the lines of communication open





7/20/2015
Weekly phone calls
Communication books and data logs
Progress reports (including graphs!)
Parent – teacher conference
Others…
164
Step 7. Communicating Progress:
You Do!
Refer to the Seven-Step Worksheet
Brainstorm…


7/20/2015
Ways that you would communicate to a
parent about the child’s progress.
Ways to involve the parent in the
evaluation of the child’s progress.
165
1. Measurable Annual Goals and Objectives
2. Data Collection Decisions
3. Data Collection Tools & Schedule
4. Representing the Data
5. Evaluation of Data
Reevaluation
6.
Determine if
additional Data/Info
is needed
7. Determine Eligibility
7/20/2015
Instruction/IEP
6.
Instructional
Adjustments
7. Communicating
Progress
166