Transcript Quick Thoughts on Value Added in Ohio

Enquiring mines wanna no . . .
Who is it?
Coleman Report
“[S]chools bring little influence
to bear upon a child’s
achievement that is
independent of his background
and general social context.”
(Coleman et al., 1966, p. 325)
It is part of the “picture” . . .
but let’s look a picture of achievement . . .
The Achievement “Lens”
 Provides a measure of what students know and
are able to do relative to the Ohio standards,
benchmarks and grade level indicators.
 Achievement tests are an important measure of
the academic learning students have
accumulated since birth.
 Creates a set of criteria for a statewide
accountability system.
But Issues Arise When We Connect
School Quality only to achievement
 What part of academic achievement is
due to SES and what part is due to the
school?
 Achievement scores alone have
problematic diagnostic value
 A judgment of school quality is flawed if
it is based solely on the achievement
levels of its students. (see next slide)
Achievement Levels
Proficiency Bar — 75% passage rate
2
3
4
5
6
Grade Levels
7
8
The focus of this lens is progress . . .
The School’s Most Important Product . . .
The Progress “Lens”
 Uses existing achievement measures to do the analysis.
 Is a reliable measure of student academic growth from one
test to the next, i.e., spring to spring.
 Has substantial diagnostic value as well as accountability
value.
 Provides data on
 average progress of students in each subject and grade
level.
 average progress of students at different prior
achievement levels.
 comparisons of progress across curricular areas.
How is it Done?
1.
We collect all of the individual student data that is available for
students.
2.
All of these data points are used to develop individual student
academic profiles.
Profile for
“Student One”
2001
2002
2003
2004
SAT9
OPT4
SAT9
OPT6
3rd
4th
5th
6th
Reading
Math
489
551
220
230
535
595
218
238
Science
525
215
565
220
Social
Studies
575
240
605
252
Copyright © 2003. Battelle for Kids
3.
Each student’s profile is added to a pool that contains the profiles of all
students, present and past, who have taken the same year-end assessments.
Student Profile 1
Student Profile 2
Student Profile 3
Student Profile 4
The Pool
Student Profile 5
Student Profile 6
Student Profile 7
Student Profile 8
Student Data
Student Profile 9
Student Profile 10
Student Profile 11
Student Profile 12
Includes the profiles of all students from this
year and from past years who have taken the
same year-end assessments
Student Profile 13
Student Profile 14
Student Profile 15
Student Profile 16
Student Profile 17
Student Profile 18
Student Profile 19
Student Profile 20
Copyright © 2003. Battelle for Kids
4. The profiles of other students with similar performance histories are then
used to create statistically reliable projected scores for each student .
Student Projection1
Student Projection 2
Student Projection 3
The Pool
Student Projection 4
Student Projection 5
Student Projection 6
Student Projection 7
The actual results of other students who have
profiles very similar to Student One are used to
create a statistical projection of where Student
One is likely to be at the end of a given
academic year.
Student Projection 8
Student Projection 9
Student Projection 10
Student Projection 11
Student Projection 12
Student Projection 13
Student Projection 14
Student Projection 15
Student Projection 16
Student Projection 17
Student Projection 18
Student Projection 19
Student Projection 20
5. Mean projections and mean student scores are calculated.
Student Projection1
Student Score 1
Student Projection 2
Student Score 2
Student Projection 3
Student Score 3
Student Projection 4
Student Score 4
Student Projection 5
Your School
Student Score 5
Student Projection 6
Student Score 6
Student Projection 7
Student Score 7
Student Projection 8
Student Score 8
Student Projection 9
Student Score 9
Student Projection 10
Student Score 10
Student Projection 11
Student Score 11
Student Projection 12
Student Score 12
Student Projection 13
Student Score 13
Student Projection 14
Student Score 14
Student Projection 15
Student Score 15
Student Projection 16
Student Score 16
Student Projection 17
Student Score 17
Student Projection 18
Student Score 18
Student Projection 19
Student Score 19
Student Projection 20
Student Score 20
Mean Projected Score
Mean Student Score
Copyright © 2003. Battelle for Kids
6.
The mean student score is compared to the building’s mean predicted score
to determine the School Effect.
Mean Student Score - Mean Predicted Score
with additional statistical reliability factored in
= School
Effect
Copyright © 2003. Battelle for Kids
Ohio’s Value-Added Measure

Measures the contribution of a school or district
to the progress of its students on test scores
 Scores are measured in “Normal Curve
Equivalent” gains
 “0” gain represents the typical or “expected” gain
 “Value-Added” is based on the 2006 – 07
distribution of scores
 Scores use a measure of precision (1 Standard
Error) to help describe the Value-Added
classification
16
Ohio’s Value-Added Measure
There are three classification “bands”
using “gain score” and 1 Standard Error
• + Above expected growth (Green)
•  Met expected growth (“one year of growth in
one year of time”) (Yellow)
• - Below expected growth (Red)
17
Gain Score, Standard Error and Classifications
Relative to Value-Added Standard
10
Exceed
(+)
Exceed (+)
Meet ( )
0
Meet (
)
Not
Meet ()
-10
Gain = 4
SE = 3
Gain = 4
SE = 5
Gain = 3
SE = 1
Gain = -4
SE = 5
Gain = -3
SE = 2
Not
Meet
(-) = -5
Gain
SE = 1
How Value-Added Impacts
The Ohio School Report Card Scores
Ohio’s Report Card System
. . . Ohio's Accountability Scheme
The yearly “Report Card” is
Ohio’s way to insure
accountability among its
schools and communicate this
accountability to the public.
The Ohio Department of
Education (aka, ODE)
publishes an annual guide for
understanding the Report
Card.
Ohio School Performance Designations
 Schools and school districts “earn” designations from the
State of Ohio based on their performance. These
designations are:
 Excellent with Distinction
 Excellent
 Effective
 Continuous Improvement
 Academic Watch
 Academic Emergency
 The School Year Report Card contains the evidence on which
these designations are based and includes four components .
. . One of these components is Value Added
State Report Card
(component 4 – Value Added . . .)
Value-added results are calculated for schools with grades 4-8 in reading
and mathematics. Additionally, a composite is displayed on the front of
the report card. The composite impacts the district’s or school’s rating.
Results will be displayed using the following symbols:
 ( + ) A plus symbol indicates that a district or school has
achieved more than one year of expected growth in student
progress.
 ( √ ) A check symbol indicates that a district or school has
achieved one year of expected growth in student progress.
 ( - ) A minus symbol indicates that a district or school has
achieved less than one year of expected growth in student
progress.
State Report Card
(component 4 – Value Added - continued . . .)
Consequences:
 A district or school that achieves, on the composite result,
an “Above Expected Growth” for two consecutive years may
increase its rating by one category. For instance, a school
otherwise rated as Effective may be raised to Excellent if it
achieves a “+” designation for two consecutive years.
 Districts and schools that show “Below Expected Growth”
for three consecutive years will have their designation
reduced.
Measurement of Learning Types
Achievement Status
Achievement Gain
Is the ultimate metric for
student post-secondary
opportunities
Simply put, achievement status
conveys the degree to which a
student or cohort demonstrates
performance mastery
Is significantly related to student
family background
Much of the foundational research
using this form reported
discouraging school influence on
student achievement
Measures the progress
students make between
two points in time
Often use
Standardized
Test Scores
Is largely unrelated to student
background factors
Though the calculations require
involved data structures, capacity, and
statistical technology, the concept is
straightforward – student progress
Reflect School
Quality
Both are important
to parents, teachers,
and community
Compares year to year performances
of unlike cohorts on School Report
Card
The more recent research using this
metric found the school’s / teacher’s
influence to be paramount for
facilitating student academic
progress
A usable gauge for measuring
school program effectiveness for
anyway that students are
grouped
Research Nuggets
Teachers matter
 The difference between having a high and a low quality
teacher 3 years in a row is roughly 50 percentile points.
 Differences within schools are typically greater than
differences across schools.
 Only the most effective teachers - the top 20 per cent –
are providing instruction that produces adequate gain in
high-achieving students, while students in the lower
achievement levels profit from all but the least effective
teachers.
 Having a high quality teacher throughout elementary
school can substantially offset or even eliminate the
disadvantage of low socio-economic background.
And Finally . . . .
https://ohiova.sas.com/evaas/signin.faces
Use the above link to explore school districts and
buildings throughout Ohio to view how they
performed on the Value-Added standard.