Transcript 20110209-02 PVAAS presentation 2-9-11

PVAAS – Pennsylvania
Value-Added
Assessment System
http://www.portal.state.pa.us/portal/server.pt/community/pa_valueadded_assessment_system_(pvaas)/8751
The TWO PVAAS Methodologies
Looking
Forward/Planning…
PVAAS Projection Reports
For Individual Students
and Cohorts of Students
Looking Back/Evaluation…
Value-added Growth Reports For
Cohorts of Students
Today
NO individual student growth measure
PVAAS IS:
• A statistical analysis of existing
assessment data (PSSA).
• A realistic measurement of a school’s
progress/growth over time.
• Progress data to add to achievement
data as schools are analyzing their
work with students and making
improvement plans.
PVAAS IS NOT:
•
Another test.
•
A method for providing a
measure of growth for an
individual student.
Current Data Collection
Approaches
High Achievement
Average Achievement
Low Achievement
Achievement is based
on each student’s ending point.
PVAAS Adds Another Variable
To Achievement…Progress
Below
Growth
Standard
At or Above
Growth
Standard
Progress is based
on each student’s starting point.
Viewing Achievement
and Progress Together
Scatter Plots
Scatter Plot Quadrants
and Their Meanings
Achievement and Growth - 4th Math
Underperforming
and Slipping
Excelling
% Proficient or Advanced
Achieving but
Slipping
56
Underperforming
but Improving
0
-20
0
PVAAS Growth Measure
20
Please review this document in
your folder…
Example: Regional Scatter Plot
% Proficient or Advanced - PSSA
Achievement vs. Growth - 4th Grade Math
56
0
PVAAS Growth Value
Scatter Plot Comparison
State vs. IU 12 vs. YCSD
Math and Reading
Grades 4 – 8
Statewide to IU Region to District
Grade 4 Math
Statewide
Grade 4 Math
IU 12
Grade 4 Math
York City SD
Grade 4 Math
Statewide to IU Region to District
Grade 4 Reading
Statewide
Grade 4 Reading
IU 12
Grade 4 Reading
York City SD
Grade 4 Reading
Statewide to IU Region to District
Grade 5 Math
Statewide
Grade 5 Math
IU 12
Grade 5 Math
York City SD
Grade 5 Math
Statewide to IU Region to District
Grade 5 Reading
Statewide
Grade 5 Reading
IU 12
Grade 5 Reading
York City SD
Grade 5 Reading
Statewide to IU Region to District
Grade 6 Math
Statewide
Grade 6 Math
IU 12
Grade 6 Math
York City SD
Grade 6 Math
Statewide to IU Region to District
Grade 6 Reading
Statewide
Grade 6 Reading
IU 12
Grade 6 Reading
York City SD
Grade 6 Reading
Statewide to IU Region to District
Grade 7 Math
Statewide
Grade 7 Math
IU 12
Grade 7 Math
York City SD
Grade 7 Math
Statewide to IU Region to District
Grade 7 Reading
Statewide
Grade 7 Reading
IU 12
Grade 7 Reading
York City SD
Grade 7 Reading
Statewide to IU Region to District
Grade 8 Math
Statewide
Grade 8 Math
IU 12
Grade 8 Math
York City SD
Grade 8 Math
Statewide to IU Region to District
Grade 8 Reading
Statewide
Grade 8 Reading
IU 12
Grade 8 Reading
York City SD
Grade 8 Reading
11th Grade Scatterplots
The PSSA-PVAAS Scatter Plots visually display
the following data from the statewide
distribution of PSSA results for 2009-10:
•
Achievement
– The mean predicted performance of the cohort.
– The actual mean performance of all students in the state is
displayed for reference
•
Growth
– School Effect ≈ Observed Mean – Predicted Mean
– Standardized School Effect = School Effect relative to its
error – displays degree of confidence
The following two slides explain the meaning
of each of the regions of an 11th grade
The Scatter Plot
Coordinate System for Grade 11
Vertical Axis
Mean
Predicted
Scaled Score
2010 Mean
Scaled Score
Growth Descriptor
Boundaries (±2SEs)
Horizontal Axis
Growth Measure in
Standard Errors
Scatter Plot Regions
Higher
and Their Meanings for Grade
11
Performing
Met Expectations
Higher
Performing
Below
Expectations
Lower Performing
Below
Expectations
Higher
Performing
Above
Expectations
Lower Performing
Above
Expectations
Lower Performing
Met Expectations
Statewide to IU Region to District
Grade 11 Math
Statewide
Grade 11 Math
IU 12
Grade 11 Math
York City SD
Grade 11 Math
Statewide to IU Region to District
Grade 11 Reading
Statewide
Grade 11 Reading
IU 12
Grade 11 Reading
York City SD
Grade 11 Reading
Overview of PVAAS
Reporting
PVAAS Reporting



District Level Reporting
School Level Reporting
Grade Level Reporting
•
•
•
•



Reading (Grades 4-8, 11)
Math (Grades 4-8, 11)
Science (Grades 4, 8, 11)
Writing (Grades 5, 8, 11)
Subgroups Reporting (ex. IEP, ED, ELL)
Performance Levels (Advanced, Proficient, Basic,
Below Basic)
Student Level Reporting
• Reading, Math, Science, Writing (grades vary based on
content area)
Have all of our groups of
students experienced one
year of academic growth
during the last school year?
Value-added
Growth Reports
PVAAS Value-added Cohort
Growth Descriptors – Grade 4 – 8
Math and Reading
Favorable Indicator
Caution Indicator
Estimated gain at or above growth standard.
Students in this cohort have made at least one year
growth.
All schools can achieve this rating.
Estimated gain below growth standard but by less than
one standard error.
Students in this cohort have grown less than the
standard.
Estimated gain below growth standard by more than one
but less than two standard errors.
Students in this cohort have fallen behind their peers.
Stronger Caution
Strongest Warning
Estimated gain well below growth standard by more than
two standard errors.
Students in this cohort have made little progress.
District/School Report Example
PVAAS Predictive School Report
How are our best students, our
average students, our
struggling students growing?
Performance Diagnostic
Report
Growth Standard Methodology
What the whiskers tell us…
Exceeded Growth Standard;
More than One Year’s Growth
Green
Met the Growth Standard;
Made One Year’s Growth
Yellow
Growth
Standard
(One Year’s
Growth)
Below Growth Standard;
Less than One Year’s Growth
Rose
Predictive Methodology
What the whiskers tell us…
ABOVE Expectations -Observed Score Was Greater
Than Predicted Score
NDD – No Detectable Difference
Between Observed Score and
Predicted Score
Observed –
Predicted
(One Year’s
Growth)
BELOW Expectations -Observed Score Was Less
Than Predicted Score
Performance Diagnostic Report
Performance Diagnostic Report
What is the likelihood that a
student will be proficient on a
future PSSA exam?
PVAAS Projections
Student Projection
Methodology
• A separate longitudinal modeling
process focused on the performance
scores of individual students.
• The model uses all of the available
data (both math and reading) to project
future PSSA performances in Math,
Reading, Science and Writing.
• Projections can be calculated for an
individual student or groups of
students.
PVAAS Student Projection Report
Identified Students Meeting Search Requirements
District (Single Grade)
Projection Summary Report
PVAAS Fall 2009 Reporting:
Projections to Basic, Proficient, Advanced
Math and Reading:
 3  4 or 5
 4  5 or 6
 5  6 or 7
 6  7 or 8
 7  8
 8  11
Writing:
 3  5
 4  5
 5  8
 6  8
 7  8
 8  11
Science:
 3  4
 4  NA
 5  8
 6  8
 7  8
 8  11
Grade PSSA
Grade PSSA
Grade PSSA
Last Taken
Last Taken
Last Taken
Approved:
PA’s Proposal for Growth
Model as Means to Make
AYP