Transcript Slide 1

I-RtI
Network
Making Progress
Monitoring Work at Tier 2
February, 2014
Facilitated/Presented by:
Insert name(s) here
The Illinois RtI Network is a State Personnel Development Grant (SPDG) project of the Illinois State Board of
Education. All funding (100%) is from federal sources.
The contents of this presentation were developed under a grant from the U.S. Department of Education, #H325A100005-12.
However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not
assume endorsement by the Federal Government. (OSEP Project Officer: Grace Zamora Durán)
Making
What
Check-in
connections
Applying
Review
One of the best
ways to remember
something is to
test yourself.
Outcomes
Connection to Illinois Professional
Teaching Standards
This content is directly tied to
Standard 7: Assessment
The competent teacher understands and uses
appropriate formative and summative
assessments for determining student needs,
monitoring student progress, measuring
student growth, and evaluating student
outcomes.
I-RtI Network
PROGRESS MONITORING: COMMON
UNDERSTANDING AND PURPOSE
Illinois RTI Network, 2013
Why do we
spend time
monitoring
the
progress of
students
receiving
Tier 2
support?
What do we expect to see?
What Should We See?
• Improved student performance
– Effect Size: ½ to 1 Standard Deviation
improvement
•
•
•
•
Improvement among lowest achievers
Teachers providing more effective feedback
Teachers making better decisions
Students who are more aware of their own
performance
I-RtI Network
BASICS IN MONITORING STUDENT
PROGRESS
RtI is the Practice of:
1. Providing high quality
instruction/intervention
matched to student
needs and
2. Using learning rate over
time and level of
performance to
3. Make educational
decisions.
A Multi-tiered System of Supports
Three tiers of
support are
stacked resources
so that they are
layered and
aligned with each
other
1. Start with core and standards based instruction
2. Identify prerequisite skills and examine where
deficits exist
3. Then allocate resources to help close the
achievement gap for all of our Learners
Core
learning
outcomes
Assess
Adjust
instruction
Problem
Solve
Do deficits exist
within individuals or
groups of students?
A Coordinated Assessment System
• Purpose (within a problem
solving system)
– Screening
– diagnostic
– Monitoring progress of
student learning
– Program evaluation
Systematic Use of Progress Monitoring Data
Plan Evaluation
Problem Identification
Did the plan work?
What is the problem?
Adequate progress?
Was the goal met?
Do we need to continue
or modify the plan?
What is expected?
What is occurring?
What is the context?
Plan Development
Problem Analysis
What will we do?
Why is it happening?
What is the goal?
Based on why, what will we do differently?
What has the highest likelihood of
success?
How will we monitor progress & fidelity?
What resources are needed? Illinois RtI Network
When does it occur?
With whom?
Can’t do or Won’t do?
Curricula, Instruction?
Environment?
Progress Monitoring Data
At Tier 2, Progress
Monitoring data can
be used to …
Uses of Progress Monitoring Data
1) To estimate the rates of improvement which
allows for comparison to peers
2) To identify students who are not demonstrating
or making adequate progress so that
instructional changes can be made
3) To compare the efficacy of different forms of
instruction – in other words, which instructional
approach or intervention led to the greatest
growth among students.
National Center on Response to
Intervention Website, 2010
Rates of Improvement
ID Students In Need of More Support
Compare Intervention Effectiveness
Summary of Uses
• As Part of a Tiered System of
Supports
• Data for Problem Solving at Tier 2
• To Identify Rates of Improvement
• To Identify Students In Need of
More Support
• To Analyze the Effectiveness of
Interventions
Time to Reflect
Where do I see these uses present in my
district?
Where could my district or school
strengthen our process?
What is one thing I could do to begin to
address this?
I-RtI Network
TOOLS AND EVALUATION
•Progress
monitoring has a
scientific base in
assessment with
over 30 years of
research.
•Technical aspects
of these tools
make them
adequate for
monitoring the
academic progress
of students.
Progress Monitoring Standards
• Brief
• Valid, reliable, and evidence based
• Sufficient number of alternate, equivalent
forms
• Sensitive to student growth
• Benchmarks of expected skill growth
• Specified rates of improvement
Psychometric Standards
1.
2.
3.
4.
5.
6.
7.
Reliability
Validity
Sufficient number of alternate forms
Sensitivity to learning
Evidence of instructional utility
Specification of adequate growth
Description of benchmarks for adequate endof-year performance or goal-setting process
Validated Forms of PM
• CBM differs from most approaches to
classroom assessment in two important ways
(Fuchs & Deno, 1991).
1. First, CBM is standardized so that the behaviors
to be measured and the procedures for
measuring those behaviors are prescribed and
have been shown to be reliable and valid.
2. Second, with CBM, each weekly test is of
equivalent difficulty and represents what the
teacher wants the student to be able to do well
at the end of the year.
Tools Chart
I-RtI Network
TOOL EVALUATION & NETWORKING
Illinois RTI Network, 2013
Progress Monitoring Rubric
Header on cover page
Iowa Department of Education
Progress Monitoring Rubric (Revised 10/24/11)
Why use Progress Monitoring Tools:
They quickly and efficiently provide an indication of a student’s response to instruction. Progress monitoring tools are sensitive to student growth
(i.e., skills) over time, allowing for more frequent changes in instruction. They allow teachers to better meet the needs of their students and
determine how best to allocate resources.
What feature is most critical:
Sufficient number of equivalent forms so that student skills can be measured over time. In order to determine if students are responding positively
to instruction, they need to be assessed frequently to evaluate their performance and the rate at which they are learning.
Descriptive info on each work group’s section
Information Relied on to make determinations: (circle all that apply, minimum of two)
Manual from publisher
NCRtI Tool Chart
Buros/Mental Measurement Yearbook
On-Line publisher Info.
Outside Resource other than Publisher or Researcher of Tool
Name of Progress Monitoring Tool:
Grades: (circle all that apply)
Name of Criterion Measure:
K
1
Skill/Area Assessed with Progress Monitoring Tool:
2
3
4
5
6
Above6
How Progress Monitoring Administered: (circle one)
How Criterion Administered: (circle one)
Group
Group
or
or
Individual
Individual
Iowa Department of Education’s
Evaluation of PM Tools
– # of forms and equivalence
– Cost and time for student
– Scientific adequacy: criterion measure, validity of
level, reliability of level and ROI
– Alignment with core
– Training needed
– Time for administration, scoring, data retrieval
– Computer options
Checklist for Evaluating Tools
Name of Measure:
Academic Area :
Level:
Yes or No
(Circle one)
Y N
1. Does the measure provide a reliable score? Research evidence demonstrates that
the measure is reliable (e.g. similar results are obtained when different people give
the measure or when alternate forms of the measure are given at different times).
Y N
2. Does the measure yield a reliable trend of student progress (reliability of slope)?
The measure has been shown to accurately identify and differentiate the trend or
rate of improvement of different students over time.
Y N
3. Is the measure a valid indicator of the academic skill of interest? The measure
provides meaningful information about the academic ‘construct’ being assessed
(i.e. the cognitive skills or processes needed to complete the academic skill(s) of
interest).
Using a Checklist to Evaluate Progress
Monitoring Tools
• Choose a progress monitoring tool that you
are currently using for PM
• Evaluate the tool using the 8 questions on the
Checklist
• Write a one sentence summary of the
strengths and weaknesses of the tool
• Share how the this evaluation process
occurs in your district
What are You using?
I-RtI Network
CHALLENGES
Problem: It doesn’t always fit
What to do?
What do instructors currently use to measure:
Basic Skills: Basic Math, Reading, Writing
Complex Skills: Reading Comprehension,
Extended Writing
Content Based Knowledge: Factual
Information, Specialized Vocabulary
Extending Knowledge, Skills/Problem
Solving: Demonstrating High Level Reasoning
High Quality Instruction
Formal Formative
Informal Formative
Interim
Progress Monitoring
Summative
Student Growth
Student Outcomes
A Coordinated Assessment System
• Monitoring progress of
student learning
– PM
– Formal Formative
– Interim
– Informal Formative
– Summative
Progress Monitoring Standards
•
•
•
•
Brief
Necessary characteristics of any
high quality assessments that
Valid
measure student growth.
Reliable
Repeatable: a sufficient number of alternate,
equivalent forms
• Sensitive to student growth
• Specified rates of improvement
Time to Reflect
How has this changed my understanding
of monitoring progress of student
learning?…
One thing I would like to have clarified
would be…
One thing I could do to apply this
learning is…
I-RtI Network
PROGRESS MONITORING PROCESS
Illinois RTI Network, 2013
Progress Monitoring Process
 Choose appropriate PM tools
 Create a year long schedule for the collection
of PM data
Schedule regular meetings with agendas to
evaluate the data
Establish a written data evaluation process
with decision rules
Establish written practices to ensure fidelity of
administration, scoring and interpretation
Take a Look at the Process
• To consider whether a practice has
been implemented with fidelity, the
practice must first be clearly defined
(Century, Rudnick & Freeman, 2010).
• By clearly defining a practice,
expectations are spelled out
creating an understanding of what
needs to be accomplished. From
that understanding, educators are
able to reflect on the integrity of
their work and then plan for next
steps to improve implementation
(Fixsen, Blasé, Horner & Sugai, 2009).
I-RtI Network
ISSUES AND TROUBLE SHOOTING
Illinois RTI Network, 2013
Issue: Lack of Adequate Progress
Look at Fidelity in
several areas:
1. Research Based &
Matched to
Student Need
2. Delivered as
Designed
3. Sufficient Time
4. Student
Engagement
5. Training Provided to
Interventionist
52
Issue: Middle and High School
• Reading:
– PM using a 3 minute maze passage
• Writing:
– PM using a 5 minute writing sample
• Content Area Knowledge:
– PM using a 5 minute vocabulary matching of
terms with definitions
Issue: Communication of Results
• Teachers
• Students
• Parents
• Stakeholders
Issue: Lack of PM Data
• Students are receiving instruction, but
progress monitoring data are not being
collected consistently.
• Data are being collected, but not being used
appropriately.
• Decisions are not made using explicit decision
rules.
Issue: Frequency
• Does the how often do you make decisions
about changes to instruction or grouping?
• Does your data collection keep pace with this
timeline? Does it go over? Does it not keep
up?
Issue: PD and Support
• Do teachers and assessment personnel
understand the PM process?
• Who provides training and ongoing support?
• What process is used to evaluate the tools and
process systematically?
• Who decides what to add and want to
eliminate as evidenced by data?
Issue: Goal Setting
• Three Methods
– Standards Based Goals
• Indicates that a student who meets the expectation is likely to
meet standards (either grade level or state test).
– Norm Referenced Goals
• Indicates that a student who meets the expectation is likely to be
successful working with in a group with typical students.
– Growth Referenced Goals (critical for use in PERA
decisions)
• Indicates that a student who meets the expectation is making
progress relative to their own level of achievement, even though
it may still be substantially below either normative information
or standards.
Setting goals is a value driven activity.
Issues Discussion
• Lack of Adequate
Progress
• Middle School/High
School
Measurement
• Communication of
Results
• Lack of PM Data
• Frequency of
Assessment
• PD and Ongoing
Support
• Goal Setting
Activity
Time to Reflect
One thing I learned during this section
is…
One thing I would like to have clarified
would be…
One thing I could do to apply this
learning is…
Key Ideas In Progress
Monitoring Tool Evaluation
1. PM should result in greater
student achievement.
2. PM tools, CBM and other
types of tools, should adhere
to scientific standards.
3. An RtI/MTSS system should
include a process for
evaluating tools.
Action
Planning