Transcript Document

The ABC’s of Progress
Monitoring
Instructors:
Keith Drieberg, Ph.D.,Coordinator of Psychological Services
Brad McDuffy, Principal of Highland Pacific
John Oliveri, Ph.D., Psychologist
Vicky Rogers, RSP Teacher, Head of Learning Center-Highland-Pacific School
1
7/21/2015
The ABC’s of Progress
Monitoring
Agenda:
Overview of Progress Monitoring – Keith
Why do we need progress monitoring – Brad
Initial steps – Brad
Data collection – Vicky
How do we use this information to inform instruction – Vicky
Aimsweb Practice session – John
Scoring & interpreting data – John
Decision making with data – John
Conclusions:
Lessons learned from Highland Pacific – Brad
School site teams – discussion
What support is needed to begin at the local site
2
7/21/2015
Topic One
Response to Instruction/Intervention (RTI)
What is Progress Monitoring - Overview
3
7/21/2015
Progress Monitoring/
Response to Intervention (RTI)
Response to Intervention (RTI):
Is a Graphic Representation Of
Progress Monitoring Data
4
This Graphic Format is easily
understood
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
Goal of Progress Monitoring:
5
To determine whether a discrepancy
exists between expected rate of
Improvement or actual rate of
improvement in a skill when an
empirically supported intervention
is delivered with integrity.
7/21/2015
Introduction


The purpose of this discussion is to Present an
Overview of Progress Monitoring and its relation to
evaluating the effectiveness of Academic
Interventions.
Standardized Procedures plus Normed “Probes”
result in the collection of high quality, researched
based data that is used to evaluate intervention
effectiveness (Response to Intervention RTI). High quality inHigh quality Out

6
RTI’s at this level meet NCLB and proposed IDEIA
Standards.
7/21/2015
Topics of Discussion







7

What is Progress Monitoring/ RTI
Why we should utilize these procedures
When/ How often should this process be conducted
(Based on Needs and 3 Tier Level Approach)
Results/ Reports (Data Driven Decision Making)
AIMSWEB Reading-CBM (Oral Reading Fluency)
Administration/ Scoring Procedures
Considerations and Recommendations Concerning
How To Get Started
Computer Time
Conclusions (Next Steps)
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
Frequent Repeated Measures:
•The Key to RTI is “A Skill, repeatedly
Measured Over Time.”
•The more frequent the measures the
Sooner one is able to ascertain the
Effectiveness of the Intervention.
(Drink Water from a Well)
8
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
(Crews)
Psychometric Demands of
Repeated Measurement of a
Skill:
•The task must be Comparable each time
Performance is Measured.
•Consequently, Many Alternative Forms
(Probes) Must be Available.
•Alternative Forms must be Reliable/Valid and
)
9 Related to the area of concern(ORF=Reading Comp.
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
(Crews)
Practical Demands of Repeated
Measurements:
•The measurement must be “do-able” in the
field.
•As a result each instance of measurement
must be BRIEF, COST EFFECTIVE, and
•EASY TO SCORE
10
7/21/2015
The ABC’s of Progress
Monitoring
Why Highland Pacific needed Progress Monitoring
What we initially set out to do
What we are currently doing
Brad McDuffy
11
7/21/2015
The ABC’s of Progress
Monitoring
What we do in the learning center model
The data we collect
How we use the data to inform instruction
Vicky Rogers
12
7/21/2015
AIMSWEB Practice - John
Login
Download sample probes
Oral Reading Fluency sample
Scoring & Interpretation of data
Charting the data
13
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
(Crews)
STEPS in Accomplishing the GOAL:
1. Determine current level of performance
(Baseline Performance). This is the middle score of
three separate measurements. (ORF=Also Find
Instructional Level)
2. Determine a Goal Point or rate of improvement
given an empirically supported intervention is being
implemented with integrity.
3. Plot a Goal Line (Connect the baseline point to the
goal point).
14
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
(Crews)
STEPS in Accomplishing the GOAL:
4. Plot Performance During Intervention. (At least six
repeated measurements of performance).
5. Determine the trend of performance change during
intervention (Plot a Trend Line).
6. Compare the slope of the goal line and the trend
line. (Create Decision Rules for Determining how to
Proceed with the Intervention.{ 3day rule / trend
analysis})
15
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
(Vanderwood)
Sample Decision Rules:
•
•
•
16
If the slope of the trend line is flatter than the slope
of the goal line, make an instructional change in the
student’s program. (Not making enough progress)
If the slope of the trend line is steeper than the
slope of the goal line, adjust the student’s goal and
draw a new goal line. (More progress than
anticipated)
If the slope of the trend line is the same as the
slope of the goal line, make no changes.
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
Some Examples of Presentation of
Progress Monitoring Data:
Graphic Representation of Data is One of
Most Understandable Methods To Present
RTI Data Presentation.
17
7/21/2015
18
05/26/2005
05/25/2005
05/24/2005
05/23/2005
10
05/22/2005
05/21/2005
05/20/2005
15
05/19/2005
05/18/2005
05/17/2005
05/16/2005
05/15/2005
RTI's
05/14/2005
05/13/2005
05/12/2005
05/11/2005
05/10/2005
05/09/2005
05/08/2005
05/07/2005
05/06/2005
20
05/05/2005
05/04/2005
05/03/2005
05/02/2005
05/01/2005
04/30/2005
04/29/2005
04/28/2005
04/27/2005
04/26/2005
04/25/2005
04/24/2005
04/23/2005
04/22/2005
04/21/2005
04/20/2005
04/19/2005
Words Read Correctly
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
25
Zorro's RTI Reading
Trend
Line
Goal
Line
Baseline
Intervention 1
Intervention 2
Linear (Intervention 1)
Baseline
5
0
Number of Weeks/Session Dates
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
Instructional Sheet For the Parent
G r a p h in g P r o g r e s s M o n it o r in g C h a r t s :
W h a t G o e s O n a G ra p h :
G e n e r a l C o n v e n t io n s
C o r r e s p o n d in g C h a r t In fo r m a t io n
L a b e l fo r t h e H o r iz o n t a l A x is (X -a x is )
P r o g r e s s M o n ito r in g D a te s
L a b e l fo r t h e V e r t ic a l A x is (Y -a x is )
N u m b e r F r u e q u e n c y V a lu e
P h a s e C h a n g e L in e
L in e s e p a r a in g b a s e lin e d a ta fr o m in te r v e n tio n
d a ta ; a ls o u s e d w h e n in te r v e n tio n is m o d ifie d
Goal
T h e fir s t " x " is p la c e d a t m e d ia n b a s e lin e d a ta p o in t
S e c o n d " x " is p la c e d a t th e p o in t w h e r e th e s tu d e n t
is e x p e c te d to b e p e r fo r m in g a t a fte r a c e r ta in
p e r io d o f tim e o r a t th e e n d o f a n in te r v e n tio n
G o a l L in e o r A im lin e
A lin e c o n n e c t in g t h e " x 's " s t a t e d a b o v e r e p r e s e n t s
s t u d e n t 's e x p e c t e d o r d e s ir e d r a t e o f p r o g r e s s
In g e n e r a l th e g o a l lin e s la n ts u p c o n c e r n in g a c a d e m ic s
a n d s la n ts d o w n w h e n b e h a v io r is a n is s u e
T r e n d L in e
R e p r e s e n t s s t u d e n t 's e s t im a t e d r a t e o f a c t u a l
p ro g re s s b a s e d o n c h a rte d p e rfo rm a n c e d a ta
S o m e B a s ic P r o c e d u r e s :
F o r B a s e L in e / C u r r e n t L e v e l o f P e r fo r m a n c e , U s e M e d ia n (M id d le ) S c o r e o f t h e T h r e e D a t a P o in t s
M in im u m o f S ix t o E ig h t P r o g r e s s M o n it o r in g R e s p o n s e T o In t e r v e n t io n (R T I) D a t a P o in t s N e e d e d
19
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
AIMSWEB Progress Monitoring
Template
The Chart is easy to construct on line and
copies can be saved on your computer.
20
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
MARS Progress Monitoring Template
s
7
e
vg
A
0
5
/0
6
P
/2
0
re
0
7
0
0
/2
4
/0
0
3
/0
2
/2
0
0
6
1
0
1
1
/1
2
/2
0
0
6
0
0
/2
0
9
/1
0
/2
8
7
0
A
/0
e
vg
to
is
H
0
is
H
3
ry
2
ry
to
is
H
H
is
to
ry
1
t
0
6
P r o g r e s s M o n ito r in g D a te s
7
G V M S In te r v e n tio n L e v e l/P r o g r e s s M o n ito r in g C h a r t (S tu d e n t N a m e : J o e S m ith )
0
C lu s t e r ( 6 w e e k s ; 5 M a t h )
B e n c h M a rk s (3 L a n g A rts )
B e n c h M a rk s (3 M a th )
CELDT
I n t e n s e L e v e l I n t e r v e n t io n ( R e d
Zone)
0 .2
C lu s t e r ( 6 w e e k s ; 5 L a n g A r t s )
P r o g r e s s R p t s ( R e a L it )
P ro g re s s R p ts (L a n g A rts )
T ie r L e v e l I I I ( Q - 1 )
P ro g re s s R p ts (M a th )
L e v e l I/II A -B /III
H ig h L e v e l I n t e r v e n t io n ( Y e llo w Z o n e - B )
T R I R p t C a r d G r a d e s ( R e a L it )
T R I R p t C a rd G ra d e s (L a n g A rts )
0 .4
0 .6
T R I R p t C a rd G ra d e s (M a th )
T ie r I I B ( Q - 2 )
S T A R R e a d in g T e s t
L o w L e v e l I n t e r v e n t io n ( Y e llo w Z o n e A)
B a s e lin e P o in ts
C A T 6 R e a d in g T o t a l
C A T 6 L a n g u a g e T o ta l
P r o g r e s s M o n ito r P ts
T ie r I I A ( Q - 3 )
C A T 6 M a th T o ta l
C S T E n g l- L a n g A r t s ( G r a d e 6 )
C S T A v g e S c o re s
0 .8
C S T E n g l- L a n g A r t s ( G r a d e 7 )
C o r e C u r r ic u lu m ( G r e e n
Zone)
T r e n d lin e
( B a s e lin e + P r o g r e s s
M o n it o r P t s )
1
C S T E n g l- L a n g A r t s ( G r a d e 8 )
C S T M a th (G ra d e 6 )
C S T M a th (G ra d e 7 )
C S T M a th (G ra d e 8 )
T ie r L e v e l 1 ( Q - 4 ) ( Q - 5 )
1 .2
21
T o t a l P r o g r e s s M o n it o r in g I n d . S c o r e 1 / P M I S 1
T o t a l P r o g r e s s M o n it o r in g I n d . S c o r e 2 / P M I S 2
L in e a r ( T o t a l P r o g r e s s M o n it o r in g I n d . S c o r e 1 / P M I S 1 )
7/21/2015
Progress Monitoring/
Response to Intervention(RTI)
(Continued)
Another Progress Monitoring Template
Base Line 1
Intervention Measure 1
Intervention Measure 2
Intervention Measure 3
Base Line 2
8
Base Line 3
10
Int. Date 1
Int. Date 2
Int. Date 3
Int. Date 4
Int. Date 5
Int. Date 6
Int. Date. 7
Int. Date 8
Int. Date 9
Int. Date 10
7
8
8
7
6
7
7
5
4
4
3
(Add Trend Line After RTI' s Completed)
Progress Monitoring Chart (Student Name:_______)
12
10
Trend Line
Base Line
8
6
RTI's
Goal Line
4
Intervention Measure 1
Intervention Measure 2
Intervention Measure 3
Goal: Reduce Documented BID's
(Vertical Axis) From____To ____ By
____.
2
Ba
se
Ba Lin
se e
Ba Lin 1
se e
L 2
In ine
t. 3
D
In ate
t. 1
D
In ate
t. 2
D
In ate
t. 3
D
In ate
t. 4
D
In ate
t. 5
In Dat
t. e
D 6
In ate
t. . 7
D
In ate
t.
In Da 8
t. te
D 9
at
e
10
0
Interventions: Relevant Meetings/
Classroom Procedures/ BID's/ BSP/
SST.
Response To Interventions
Measures: Number/ Fruequency of
Documented BID's.
Progress Monitoring Dates
22
7/21/2015
Why We Should Utilize These
Procedures
23
7/21/2015
First For the Benefit of our
Kids
Research Shows that:



24
(Vanderwood)
Improvement in Reading Skills Levels generalizes
out to Improvement also being noted in other
Academic areas.
A significant amount of research supports a causal
relation between phonological awareness and
additional higher reading skill levels.
Gains in Reading Skills (phonological awareness)
can be obtained with teaching and these gains
directly affect the ease of subsequent reading skills.
7/21/2015
For the Benefit of our Kids
(Continued)




25
(Vanderwood)
It has been found (Juel/1988) that the probability of a
child who was a poor reader in 1st grade continues to
remain a poor reader in 4th grade (Probability=.88).
Reading Fluency is highly correlated to reading
comprehension measures.
We can conclude that Early Problem Identification
and Intervention is key to overcoming reading
deficits.
Interventions in higher grades is also beneficial in
overcoming reading deficits.
7/21/2015
Third
OUR SANITY



26
These ideas enhance the sense of power and
influence that we all have on educational process
(both behavioral and academic).
More and more of our peers seem to agree with
these views in the field of education.
The next train is about to leave the station and we
can either ride together to our next destination with
our peers on that train or walk alone.
7/21/2015
Topic Three
When/ How Often Should This Process
Be Conducted
(Based on Needs and 3 Tier Level
Approach)
27
7/21/2015
Recommended General
Approach Towards
Implementation



28
Need to Prioritize (What resources do we
have in the way of People Power, Money, and
Time we have to devote to this process.
What are the needs of our kids and what is
the best way of meeting those needs based
on present resource availability (Medical
Model).
Realize it is imperative that we get started,
that our initial efforts are only the beginning
of this process, and it need not be “perfect”
(Goal/ Benchmark,Smaller Steps Approach).
7/21/2015
When/ How Often Should This Process Be Conducted
(Continued)
AIMSWEB 3 Tier Progress
Monitoring
TIER 2 STRATEGIC MONITOR
Assess at-risk students or
other students of concern
monthly and monitor the
effectiveness of instructional
changes
29
TIER 3 PROGRESS MONITOR
Monitor progress frequently towards
individualized goals for at-risk
students or ‘best practice’ IEP goals
for students receiving special
TIER 1 BENCHMARK
Assess all students three times per
year for universal screening (early
identification), general education
progress monitoring, and AYP
7/21/2015
When/ How Often Should This Process Be
Conducted(Continued)
Need Determines Frequency of
Progress Monitoring/RTI

Tier I: Benchmark
Three Times a Year

Tier II: Strategic Monitor
At least One Time Monthly/ Prefer Twice Monthly

Tier III:Progress Monitor
At least One Time Weekly, more frequently as
needed
30
7/21/2015
Topic Four
Results/ Reports
(Data Driven Decision Making)
31
7/21/2015
Results/ Reports(Continued)
Previously Shown RTI Charts
One Example of A Report that Facilitates
Decision Making that is Data Driven
32
(Crews)
7/21/2015
Results/ Reports(Continued)
AIMSWEB Reports Another
Example of Data Driven Decision Making
33

Rank Orders Students
by Performance

Color-Codes Individual
Educational Needs

Provides Instructional
Decisions to Think
About
7/21/2015
Results/Reports (Continued)
AIMSWEB Tier 1 Benchmark –
Student Report
Winter Benchmark Scores
•Benchmark Report Can
Confirm Significant
Improvements Have Been
Made, Such as Response
to Intervention (RTI)
•Growth from At Risk to
Low Average
34
7/21/2015
Results/Reports (Continued)
AIMSWEB Tier 1 Benchmark –
Student Report
Spring Benchmark Scores
Documents What
Worked for At Risk
Students
35
7/21/2015
Results/ Report (Continued)
AIMSWEB Tier 2 Strategic Monitor
Individual Student Report
Allows for More Frequent
Evaluation of Progress
for Both Individual
Students and Programs
Designated to Impact
Achievement for Those
At Risk
36
7/21/2015
AIMSWEB Tier 1 Benchmark - Grade Report
Average Score by School Year
Allows Evaluation
Of Year to Year
Improvement
37
7/21/2015
Results/ Reports(Continued)
MARS Spread Sheets
38

Scores are adjusted from different sources to
facilitate Data Interpretation/Decision Making

The Summary Data Sheets and Reports are in a
Three Tier Format

Short Form and Long Form Summary Data Sheets

Data Presently Hand Entered
7/21/2015
Results/ Reports(Continued)
Spread Sheets Another Example of
Data Driven Decision Making (MARS)
At-Risk Data Base System
School Name:
CSTELA CSTMATHCAT RT CAT LT CAT MT AIMSWEB LA/ B1 LA/B2 LA/B3 Math/B1 Math/B2 Math/B3 Total
Name
ID #
(PMIS)
OARS LA OARS LA OARS LA OARS MT OARS MT OARS MT
Theme# Theme# Theme# Theme# Theme# Theme#
Zorro JonJon zzzzzz
0.215
0
0 0.6 0.809 0.809 0.215
0 0.809
0
1
1 0.45475
Student
2
0.215 0.215 0.215 0.6 0.6 0.6 0.519 0.6 0.6 0.6 0.6 0.6 0.497
Student
3
0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809 0.809
Student
4
0
Student
5
0
39
7/21/2015
Results/ Reports(Continued)
Spread Sheets Can Be Depicted in
Graph Form
GVMS Intervention Level/Progress Monitoring Chart (Student Name: Joe Smith)
Cluster (6 weeks; 5Lang Arts)
05
/0
6/
20
07
03
/0
4/
20
07
01
/0
2/
20
07
Cluster (6 weeks; 5Math)
11
/1
2/
20
06
09
/1
0/
20
06
07
/0
8/
20
06
3
Hi
st
Av
ge
Hi
st
or
y
2
Hi
st
or
y
Hi
st
or
y
1
Progress Monitoring Dates
Bench Marks (3Lang Arts)
Bench Marks (3Math)
CELDT
0
Intense Level Intervention (Red Zone)
Progress Rpts (Rea Lit)
Progress Rpts (Lang Arts)
Tier Level III (Q-1)
0.2
Progress Rpts (Math)
High Level Intervention (Yellow Zone-B)
TRI Rpt Card Grades (Rea Lit)
TRI Rpt Card Grades (Lang Arts)
TRI Rpt Card Grades (Math)
0.4
Level I/II A-B/III
STAR Reading Test
Tier II B (Q-2)
CAT 6 Reading Total
Low Level Intervention (Yellow Zone-A)
0.6
CAT 6 Language Total
Baseline Points
Progress Monitor Points
CST Avge
0.8
CAT 6 Math Total
CST Engl-Lang Arts (Grade 6)
Tier II A (Q-3) Scores
CST Engl-Lang Arts (Grade 7)
Core Curriculum (Green Zone)
CST Engl-Lang Arts (Grade 8)
CST Math (Grade 6)
CST Math (Grade 7)
1
CST Math (Grade 8)
Total Progress Monitoring Ind.Score1/PMIS1
1.2
40
Tier Level I (Q-4)/(Q-5)
Total Progress Monitoring Ind.Score2/PMIS2
7/21/2015
Results/ Reports(Continued)
Other Data Systems
Can also Facilitate the Data Decision
Making Process
“Data Director” System
 Helps organize data from a data base
system into report formats that facilitates
the Data Decision Making Process.
 Take Data to be considered from data bank
{Student Group (specific class,grade etc.)},
and develop Report Formats tailored to
individual user’s needs.
41
7/21/2015
Conclusions
Brad
Lessons learned
Goals for 2007-2008
42
7/21/2015
Conclusions (Next Steps)
Team Members Take Ten Minutes to:
 Discuss Summarize Initial Actions
Required For Your Site to Get Started
 Summarize any Follow Up Action Items and
Support You need
43
7/21/2015