Transcript Progress Monitoring Power Point Presentation

Overview of Progress Monitoring Training Session
Part of a training series developed to accompany the AIMSweb
Improvement System.
Purpose is to provide the background information and data collection
procedures necessary to organize and conduct frequent, direct and
continuous measurement of students’ skill improvement.
Designed to accompany:
• Administering and Scoring of Reading Curriculum-Based Measurement for Use in
General Outcome Measurement Training Workbook.
• AIMSweb Progress Monitor - Strategies for Writing Individualized Goals in General
Curriculum and More Frequent Formative Evaluation
• Standard Progress Monitoring Reading Assessment Passages.
• AIMSweb Progress Monitor Web-Based Software.
• Training Video.
Big Ideas About Frequent Formative Evaluation Using General
Outcome Measures and the Progress Monitoring Program
One of the most powerful interventions that schools can use is
systematic and frequent formative evaluation.
Benchmark Assessment is not enough for some students because
they may be in ineffective programs too long.
The solution is to write individualized goals and determine a feasible
progress monitoring schedule.
The core of frequent progress monitoring is Survey-Level Assessment,
goal setting using logical educational practices, and an analysis of
student need and resources for determining progress monitoring
frequency.
Formative Assessment
Process of assessing student achievement during instruction to
determine whether an instructional program is effective for individual
students.
•
When students are progressing, keep using your instructional
programs.
•
When tests show that students are not progressing, you can
change your instructional programs in meaningful ways.
•
Has been linked to important gains in student achievement (L.
Fuchs, 1986) with effect sizes of .7 and greater.
Systematic formative evaluation requires the use of:
Standard assessment tools…
That are the same difficulty and that are…
Given the same way each time.
More Severe Achievement Problems and/or More Resource
Intensive Programs Require More Frequent Formative Evaluation
Benchmark Testing (3 - 4 x
Per Year) is not enough for
some students.
With Very Low Performers, Not Satisfactory to Wait This Long!
Programs That are More Resource Intensive…
Title I, English Language Learning, Special Education
Should monitor student outcomes more frequently than the
Benchmark Testing schedule.
Formative Evaluation of Vital Signs Requires Quality Tools
Technical adequacy (reliability and validity);
Capacity to model growth (able to represent student achievement
growth within and across academic years);
Treatment sensitivity (scores should change when students are
learning);
Independence from specific instructional techniques (instructionally
eclectic so the system can be used with any type of instruction or
curriculum);
Capacity to inform teaching (should provide information to help
teachers improve instruction);
Feasibility (must be doable).
Formative Evaluation is Impossible without Clear Goals
Goals Make Progress Decisions Easier
Current Goal Setting Practices Are Unsatisfying
Do you like these IEPs?
I do not like these IEPs
I do not like them Jeeze Louise
We test, we check
We plan, we meet
But nothing ever seems complete.
Would you, could you
Like the form?
I do not like the form I see
Not page 1, not 2, not 3
Another change
A brand new box
I think we all
Have lost our rocks!
Improving the Process of Setting Goals for Formative Evaluation
Set a few, but important goals.
Ensure goals are measurable and linked to validated formative
evaluation practices.
Base goal setting on logical educational practices.
Few But Important Goals
Goal Smorgasboard!
•
•
•
•
•
•
•
•
•
•
•
Student will perform spelling skills at a high 3rd grade level.
Student will alphabetize words by the second letter with 80% accuracy.
Student will read words from the Dolch Word List with 80% accuracy.
Student will master basic multiplication facts with 80% accuracy.
Student will increase reading skills by progressing through Scribner with
90% accuracy as determined by teacher-made fluency and
comprehension probes by October 199x.
To increase reading ability by 6 months to 1 year as measured by the
Woodcock Johnson.
Student will make one year's growth in reading by October 1990 as
measured by the Brigance.
Student will be a better reader.
Student will read aloud with 80% accuracy and 80% comprehension.
Student will make one year's gain in general reading from K-3.
Students will read 1 story per week.
Improving the Process of Setting Goals for Formative Evaluation
Set a few, but important goals.
Ensure goals are measurable and linked to validated formative
evaluation practices.
Base goal setting on logical educational practices.
Reduce the Number of Goals to a Few Critical Indicators
Reading
In (#) weeks (Student name) will read (#)
Words Correctly in 1 minute from randomly
selected Grade (#) passages.
Spelling
In (#) weeks (Student name) will write (#)
Correct Letter Sequences and (#) Correct
Words in 2 minutes from randomly selected
Grade (#) spelling lists.
Math Computation In (#) weeks (Student name) will write (#)
Correct Digits in 2 minutes from randomly
selected Grade (#) math problems.
Written Expression In (#) weeks (Student name) will write (#) Total
Words and (#) Correct Writing Sequences when
presented with randomly selected Grade (#)
story starters.
Ensure the Goals are Measurable and Linked to Validated
Formative Evaluation Practices
Goals should be based on quality tests like CBM.
Based on validated practices such as how often, how many samples,
etc.
Base Goal Setting on Logical Educational Practices
Leonardo currently reads about 15 words correctly from Grade 5
Standard Reading Assessment Passages. He reads Grade 1reading
passages successfully; 68 correct words per minute, which is how
well end of first grade students read this material.
Goal Setting Strategies
Current Performance Information based on Survey-Level Assessment
(SLA).
Know the Time Frame for the Goal.
Determine a Future Performance Level.
Conducting a Survey Level Assessment
Students are tested in successive levels of general curriculum,
beginning with their current expected grade placement, until a level at
which they are successful is determined.
Reading Survey-Level Assessment for Kate, A Third-Grader
Grade Reading
Assessment
Passages
Fall
Performance
Level
Passage 1
(WRC/E)
Passage 2
(WRC/E)
Passage 3
(WRC/E)
Median
(WRC/E)
3
35/6
37/5
50/8
37/6
At Risk
Reader
2
60/4
58/3
42/7
58/4
Average
Reader
Setting the Time Frame, Goal Level Material, and Criterion
Time Frame
End of Year (At Risk or Grade-Level Expectations)
In 18 Weeks…
Annual IEP Goals (Special Education)
In 1 year… (or) In 32 Weeks…
Setting the Goal Material
Logical Task-• Matching or Not Matching Expected Grade Placement
•
Title 4 Fourth Grader--Grade 4 Material?
•
Grade 4 Special Education Student--Grade 4 Material?
When Grade-Level Expectations Are Not Appropriate
Consider the Severity of the Discrepancy
Consider the Intensity of the Program
Determining the Criterion for Success
Local Benchmark Standards.
Linkage to High Stakes Tests.
Normative Growth Rates.
Developing Your Own Sample of Standards.
Benchmark Standards Advantages and Disadvantages
Advantages
Disadvantages
Easily Understood
Uncomfortable, Especially in Low
Achievement Environments
Can Indicate When Student No
Longer Needs Specialized
Instruction
Issues of “Equity”
Determining Least Restrictive
Environment (LRE)
Linkage to High Stakes Standards Advantages and Disadvantages
Advantages
Disadvantages
Reduces Problems of Equity when
Local Achievement is Low
Need Linkage to High Quality High
Stakes Test
Increases Usefulness of High
Stakes Tests
Linkage Must Be Established
Empirically
Helps Specify Long-Term Targets
(What Grade 2 Student Needs to
Read to Be Successful on Grade 6
Test)
Adoption of Assumption that the
Attainment of the Target Guarantees
Passing High Stakes Test
Normative Growth Rates
Criterion for Success = Score on SLA + (Grade Growth Rate times #
of Weeks)
Score on SLA (30)+ (Ambitious Grade Growth Rate (2.0) times # of
Weeks (32)
Or
30 + (2.0 * 32) or 30 + 64 = Annual goal of 94 WRC
Growth Rate Standards Advantages and Disadvantages
Advantages
Easily Understood
Disadvantages
May Underestimate What Can Be
Attained with High Quality
Instruction
Developing Your Own Sample of Standards
Developing a Small Local Norm
Benchmark Standards Advantages and Disadvantages
Advantages
Same Advantages as Benchmark
Standards
Disadvantages
Same Disadvantages of Benchmark
Standards
Small Sample Size
How Frequently to Assess?
Balancing IDEAL with FEASIBLE
The End