Transcript Will Growth Models Improve School Accountability and NCLB

Will Growth Models Improve School
Accountability and NCLB/AYP?
Results From New Research
Survey and Analysis of Current AYP Growth Proposals
Kimberly O'Malley – Pearson Educational Measurement
Bill Auty – Education Measurement Consulting
AYP Growth Proposals
NCLB Alignment Elements
●
●
●
The accountability model must ensure that all students
are proficient by 2013-14 and set annual goals to
ensure that the achievement gap is closing for all
groups of students.
The accountability model must not set expectations for
annual achievement based upon student background
and school characteristics.
The accountability model must hold schools
accountable for student achievement in
reading/language arts and mathematics.
AYP Growth Proposals
Foundational Elements
●
●
●
●
The accountability model must ensure that all students in the tested grades
are included in the assessment and accountability system. Schools and
districts must be held accountable for the performance of student subgroups.
The accountability model includes all schools and districts.
The State’s assessment system, the basis for the accountability model, must
receive approval through the NCLB peer review process for the 2005-06
school year. In addition, the full NCLB assessment system in each of grades
3-8 and in high school in reading/language arts and math must have been in
place for two testing cycles.
The accountability model and related State data system must track student
progress.
The accountability model must include student participation rates in the state
assessment system and student achievement on an additional academic
indicator.
Cross-Cutting Issues from Peer
Reviewers
●
States should:
●
●
●
●
incorporate available years of existing achievement data, instead
of relying on only two years of data,
align growth timeframe with school grade configuration and
district enrollment,
make growth projections for all students, not just those below
proficient, and
hold schools accountable for same subgroups as they did under
status model.
States should not:
●
●
use wide confidence intervals,
●
reset growth targets each year, and
●
average scores between proficient and non-proficient students.
Current Status
●
In sum,

the alignment and foundational elements appear to
be necessary for approval

the cross-cutting issues appear more as guiding
principles than requirements

USDE has allowed some creativity in proposals
Typology of Growth Models
●
Improvement
●
Difference Gain Scores
●
Residual Gain Scores
●
Linear Equating
●
Transition Matrix
●
Multi-Level
Characteristics of Growth Model Types
●
●
●
●
●
●
●
●
Database
Common Scale
Confidence Interval
Missing Scores
Alternate Tests
Growth Question Answered
Student Performance Standards Explicitly
Included in Definition of Growth
Non-Linear Growth
Table of Growth Model Characteristics
Improvement
Difference Residual
Linear
Gain Scores Gain Scores Equating
Transition
Matrix
Multi-level
Data Requirements
Database of matched
student records over
time (Student ID)
N
Y
Y
Y
Y
Y
Requires common scale
N
Y
N
N
N
Y
Independent
Groups
t-Test
Model Error
Variance
Model Error
Variance
NA
Model Error
Variance
Includes students with
missing scores
Y
N
N
N
N
Y
Includes Results From
Alternate Tests
(Different scales)
N
N
N
N
Y
N
Psychometric Issues
Confidence Interval
Growth Question
Answered
Did this year's Is the gain for
How much
students do
a group higher growth was
better than last or lower than produced by a
year's students?
average?
group?
Are students in
a group
How much of a
Did students
making
group's growth
stay at the
adequate
is the result of
same
progress across group-level
percentile?
performance
effects?
levels?
Student Performance
Standards Explicitly
Included in Definition
of Growth
Y
N
N
N
Y
N
Handles non-linear
growth
N
N
Y
N
Y
Y
Local Applications
●
Oregon School Boards Assoc. Bridges project

Leadership Training to Improve Student
Achievement

Providing Data Analysis to Pilot Districts

Range of Capacities

Partnership With Data Provider
Implementer's Guide to Growth
Models
●
Sequel to Policy-Maker's Guide to Growth
●
CCSSO ASR project
●
Collaboration of member states
●
Publication later this year
Summary
●
Next steps

Updates to paper
●
●
●
●
Corrections
Impact
Additional proposals
Implementer’s guide

Practical Advice for Implementing Growth Models

Available Later this Year