Transcript Document

Progress Monitoring
Goal Setting
Overview of Measures
Keith Drieberg, Director of Psychological Services
John Oliveri, School Psychologist
Cathleen Geraghty, School Psychologist
Overview
Comparison of Terms Between Systems
Term
AIMSWEB
District
Levels
Tiers
III (Red Zone)
Progress
Monitor
II (Yellow Zone)
I (Green Zone)
I/II/III
Grade
Level
Frequency
Intensive
Instructional
Weekly
Strategic
Targeted
Assigned
1 x Monthly
Benchmark
Schoolwide
Screenings
Assigned
3 Times a
Year
General Education
Screening and Progress
Monitoring in General Education
Screening
• Screening allows for quick identification of
students that are at-risk for academic
difficulties
• Different screening measurements
• When to do screenings
• Who is screened?
Screening When and Why?
• 1st Screening - identification
• 2nd Screening - prediction and
identification
• 3rd Screening - goal attainment
Which Measures
• Kindergarten
– Fall - Letter Naming Fluency (LNF)
– Winter - LNF & Phoneme Segmentation Fluency (PSF)
– Spring - LNF & PSF
• 1st Grade
– Fall - Nonsense Word Fluency (NWF) & Oral Reading
Fluency (ORF)
– Winter - NWF & ORF
– Spring - NWF & ORF
• 2nd Grade
– Fall - ORF
– Winter - ORF
Spring - ORF
Administration for Early Literacy
Letter Naming Fluency
Example: Letter Naming Fluency
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Phoneme Segmentation Fluency
Example: Phoneme Segmentation Fluency
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Nonsense Word Fluency
Example: Nonsense Word Fluency
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Oral Reading Fluency
ORF (R-CBM) Administration
• Student’s read the passage aloud for 1
minute
• Number of words read correct and
number of errors are counted
– WRC/errors
Area Assessed
Timing
Test Arrangement
What is Scored
CBM Oral
Reading Fluency
(R-CBM)
1 minute
Individual
# of Words Read
Correct (WRC)
and the # of
Errors
Example: Oral Reading Fluency
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
Curriculum Based Measurement Reading Maze
CBM Maze is designed to provide educators a more complete
picture of students’ reading skills, especially when comprehension
problems are suspected.
Area
CBM Maze
Reading
Timing
3 minutes
Test Arrangements
What is Scored?
Individual, Small
Group, or Classroom
Group
# of Correct Answers
An example of CBM Maze
Who is Screened?
• Ideally, all students are screened in the
Fall, Winter, and Spring
• Students scoring below the 25th
percentile are considered at-risk and
should be progress monitored
What Is & Why Progress Monitor
• What is Progress Monitoring
– Formative assessment tool
• Why Progress Monitor
– Tell us whether students are profiting from
the curriculum, and whether or not an
intervention is effective for that particular
student
• Some Everyday Examples of How We
Use Informal Progress Monitoring
What is Progress Monitoring
•
Technically adequate
– reliability and validity
•
Capacity to model growth
– able to represent student achievement growth within and across academic
years
•
Treatment sensitivity
– scores should change when students are learning
•
Independence from specific instructional techniques
– instructionally eclectic so the system can be used with any type of
instruction or curriculum
•
Capacity to inform teaching
– should provide information to help teachers improve instruction
•
Feasibility
– must be doable
Why Progress Monitor
• Screening is not enough for some students because
they may be in ineffective programs for too long.
• Progress monitoring allows for individualized goals to
be written and determination of a feasible amount of
time for the goal to be reached.
• Allows for an analysis of student need and resources
for determining progress monitoring frequency.
– Programs that are more intensive (e.g., special education),
should monitor student outcomes more frequently that 3x per
year.
More Frequent Evaluation
Progress Monitoring Plan
• Progress Monitoring Plan
– No Learning Center
• Monitor at-risk students, once a month
at grade level
– Learning Center
• Monitor at-risk students
– once a month at grade level
– every week at instructional level
• Collect data on Wednesday or
Thursday
Goal Setting - General
• Each student should have a year-long
goal (you want the student to be
performing at the 50th percentile on
grade level material)
• Every goal should have:
– Time frame (when the goal should be reached)
– Behavior (what the desired level of performance is)
– Condition (which measure and where you obtained
it from)
– Criterion (which grade level passage you are using
- if applicable)
Sample ORF Goal
• Oral Reading Fluency (Fluency)
– In (#) of weeks (student name) will read
(#) word correct in 1 minute as
measured by a (grade ___ ) (AIMSWeb
Oral Reading Fluency Measure).
Norms and Growth Rates
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Year-Long Goal
• Expected weekly growth
– If the student is not at-risk use the average
rate of growth
– If the student is at-risk use the ambitious
rates of growth (e.g., below the 25th
percentile)
• Multiply the growth rate by 36 (weeks of
school) and add it to the baseline level
of performance
Year-Long Goal Example
• In the fall, a 4th grade student’s
instructional level is 2nd grade and the
student reads 22 words correct per
minute. Ambitious growth for 2nd grade
is 2 words per week
– 22 + (2 x 36) = 94
– The year-long goal would be 94 words
correct per minute
Short-Term Goal
• All students being progress monitored
need shorter-term goals
• These goals should use ambitious rates
of growth to help the students ‘catch-up’
Short-Term Goals
• Determine what level of performance is
needed to move up a percentile rank (e.g.,
10th to 25th; 25th to 50th)
• Take the difference in performance (e.g., how
many words correct the student will need to
move from the 10th to 25th percentile), and
divide by the ambitious rate of growth
– This product will be the approximate number of
weeks needed for the student to reach the next
percentile rank
Short-Term Goals
• If using early literacy measures (LNF,
PSF, NWF) move up instructional level
until the student has mastered that skill,
then move up to the next skill
Literacy & Writing Measures - Suggested Timeline for Administration
M AZE (Comprehension)
Oral Reading Fluency
Spelling
Written Expression
Nonsense Word Fluency
Phoneme Segmentation Fluency
Letter Sound Fluency
Letter Naming Fluency
Beg
Initial Sound Fluency
M id
End
Beg
Preschool
M id
Kindergarten
End
Beg
M id
First Grade
End
Beg
M id
Second Grade
End
Beg
M id
End
Third Grade and Abov e
Short-Term Goals
• In the Winter, a 3rd grade student’s
instructional level is 2nd grade and the
student reads 29 words correct per minute.
– The student is currently performing at the 10th
percentile. The short term goal should be at the
25th percentile.
• Difference between the 25th and 10th percentile at the
2nd grade level is 24.
• Divide 24 by the ambitious growth rate (2.0) and you get
12. That is the number of weeks it should take the
student to move from the 10th to the 25th percentile.
Charting Progress
• Once you set the goal, you need to chart progress to
see if the student is responding to the change in
instruction
Graphing Progress Monitoring Charts:
What Goes on a Graph:
General Conventions
Corresponding Chart Information
Label for the Horizontal Axis (X-axis)
Lavel for the Vertical Axis (Y-axis)
Progress Monitoring Dates
Number Frequncy Value
Line separating base line data from intervention data; also
used when intervention is modified
The first 'x' is placed at median base line data point. Second
'x' is placed at the point where the student is expected to
be performing at after a certain period of time or at the
end of an intervention.
A line connecting the 'x''s stated above represents studetn's
expected or desired rate of progress. In general the goal
line slants up concerning academics and slants down when
behavior is an issue.
Represents student's estimated rate of actual progress
based on charter performance data
Phase Change Line
Goal
Goal Line or Aimline
Trend Line
Some Basic Procedures:
For Base Line / Current Level of Performance, Use Median (Middle) Score of the Three Data Points
Minimum of Six to Eight Progress Monitoring Response to Intervention (RtI) Data Points Needed
Evaluating Goal Attainment
• Process of assessing student achievement during
instruction to determine whether an instructional
program is effective for individual students.
– When students are progressing, keep using your
instructional programs.
– When tests show that students are not
progressing, you can change your instructional
programs in meaningful ways.
– Has been linked to important gains in student
achievement (Fuchs, 1986) with effect sizes of .7
and greater.
Trendline
Goal Line
Trendline
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Goal Line
Goal line
Trend Line
Special Education
How can Progress Monitoring be
used in Special Education?
Educational Benefit
• Progress monitor at both instructional
and grade level
– Grade Level Goals
• How is the student performing relative to
his/her peers?
– Instructional Level Goals
• How is the student’s performance changing as
a function of the current instruction /
intervention?
– Instructional level progress monitoring is sensitive to
growth.
General Goal Setting
• Long-Term Goal
– The long-term goal should be either to
move the student:
• Up a full grade level (Ex. 50th percentile on a 1st
grade probe to 50th percentile on a 2nd grade
probe)
• Up to a higher skill (Ex. Letter Naming Fluency
to Letter Sound Fluency)
General Goal Setting (continued)
• Short-Term Goals (Benchmark 1 & 2)
– The benchmark goals should be to move
the student up a percentile level within their
instructional level (could be the same as
grade level). (Ex. Move from the 25th to the
50th percentile).
Review/Overview
Comparison of Terms Between Systems
Term
AIMSWEB District
Grade
Level
Frequency
I/II/III
III (Red
Zone)
II (Yellow
Zone)
I (Green
Zone)
Levels
Tiers
Progress
Monitor
Intensive
Instructional
Weekly
Strategic
Targeted
Assigned
Benchmark Schoolwide
Screenings
1x
Monthly
Assigned 3 Times
a Year
First Example
(Topic Two )
The Movie “City Slickers”
The Cattle Drive----
That Great Line:
• “Don’t Know Where We Are
At;
• Don’t Know Where We Are
Going;
• But We Are Sure Are
Making A Lot Of Progress”
That Great Line:
A Case For Level I
Bench Mark Screenings At Assigned Grade Levels For All
Students
Don’t Know Where
We Are At
Don’t Know Where
We Are Going
Not Sure Of
Progress
No Base Line
Point
No Goal
Point
No Goal Line
or Aimline
What We Already Know:
A Case For Level II Strategic/ Targeted
Assigned Grade Level Progress Monitoring On a Regular Basis
The Earlier We Start
Interventions; The
Less Behind the
Student is in
Comparison to Their
Grade Level Peers;
and The Sooner A
Student Starts to
Makes Progress
Their Peer Growth Rate Is
High And Student
“Catches Up Quickly”
(within about a year)
Because They are Not
Too Far Behind
(Assigned Grade Level
Progress Monitoring /
Strategic/Targeted – ideal
1 x Monthly)
What We Need to Think More About:
A Case For Level III Progress Monitor/Intensive
Instructional Level Progress Monitoring
•Higher Grade Levels
(Especially 4th Grade +)
•Student Significantly
Farther Behind Assigned
Grade Level Peers
•Need Long Term Goals
(Takes Longer Than One
Year to “Catch Up”)
•Kids Give Up On Themselves
(Highest Drop Out Rate 9th
Grade)
•We Expect Less of Students
at Higher Grade Levels and
Blame Others
•(Instructional Grade Level
Progress Monitoring To Know
If Intervention is Workng;
Appreciate Progress Sooner;
and Receive Frequent
Feedback )
What We Need to Think More About:
Instructional Grade Level Progress Monitoring
•Higher Grade Levels
(Especially 4th Grade +)
•Student Significantly
Farther Behind Assigned
Grade Level Peers
•Need Long Term Goals
(Takes Longer Than One
Year to “Catch Up”)
In Addition,
•Peer Growth Rate Is Slower at
Higher Grade Levels
•If We and the Student
Continue Ambitious Goals Each
Year
•The Student Will Make
Significant Gains and
Eventually “Catch Up”•(Instructional Grade Level
Progress Monitoring)
Why We Need Both Assigned Grade Level
and Instructional Grade Level Goals and
Progress Monitoring
ASSIGNED GRADE LEVEL
TESTING SUMMARY SHEET
Last Name4
First Name4
Teacher2
Primary
Intervention2 Tutor2
Advancing
Alpha
Anaximander
Andrew
Alice
Albert
Smith
Smith
Applegate
Learning Cent
Learning Cent
Bef School
Helpful
Helpful
Goodguy
Last Year
Quintile
3
2
1
Beginning/Fall
Middle/Spring
End/Spring
ORF 1 Screening
ORF 2 Screening2
ORF Screening3
79
65
60
95
90
75
110
105
90
6th Grade 90 WC
7th Grade 120 WC
8th Grade 150 WC
End of Year
Goal
109
95
90