Transcript PVAAS Public Site Tour

Pennsylvania Value-Added
Assessment System (PVAAS):
PVAAS Public Site Tour
PVAAS Statewide Team for PDE
Fall 2011
Session Description




This 60 minute session will provide an overview
of the PVAAS public reporting site.
All data reported on the public site will be
described as to the intent and meaning of the
results. This includes the district and school level
data, the School Search feature and the webbased scatter plots.
Materials and resources to use as tools for
communication about PVAAS will be reviewed.
Opportunities for questions and discussion will be
provided.
Agenda






Achievement and Progress Data
What’s New?
Available Data & Features
Website Demonstration
Suggested Resources for
Communicating PVAAS
Questions
Achievement vs. Progress
Progress
Achievement


The final result of an
academic experience
Highly correlated with
demographic factors,
such as socioeconomic
status




Affected by factors
outside the school
Dependent upon what
happens as a result of
schooling
Not correlated with
demographic factors
Is the concept
underlying valueadded analysis and
reporting
Achievement vs. Progress
Progress
Achievement



Measures students’
performance at a
single point in time

Compares student
performance to a
standard

Critical to students’
post-secondary
opportunities

Measures students’
progress across
time/years
Compares student
performance to
his/her own prior
performance
Critical to ensuring
students’ future and
continued academic
success
Stair-Step Expectations
Grade Three
Grade Four
Grade 3 Test
Grade 4 Test
Grade Five
Grade 5 Test
In a perfect world students would start school at
about the same academic level. Proficiency levels
could simply be achieved by students gaining age
appropriate knowledge and skills each year.
Differentiated Reality
Grade Three
Grade Four
Grade 3 Test
Grade Five
Grade 4 Test
Grade 5 Test
Students begin school at different places, and they
often progress at different rates. Yet all students
are expected to achieve the same level of
academic proficiency each year.
The Need for Progress Measures
Grade Three
Grade Four
Grade 3 Test
Grade Five
Grade 4 Test
Grade 5 Test
To measure school effectiveness in this
“differentiated” world, we need to pay attention not
only to proficiency levels but also to how much
progress students make in a given year.
What is Value-Added?



Imagine a child’s
physical growth curve
Parent or doctor
measures the child’s
height at various ages
A graph can be
constructed to
illustrate the height of
the child as shown
here
What is Value-Added?



Let’s apply the same
process to education
This chart measures the
growth, or progress, for
a group of students at
various grade levels
This is value-added 
following the growth, or
progress, of students
over time to estimate
growth during a year of
schooling.
With PVAAS, Pennsylvania educators
get an indicator as to whether they are
making appropriate academic
progress for their students!
Value-Added is…



A statistical analysis used to measure a district’s
or school’s impact on the academic progress
rates of groups of students from year to year.
Conceptually, a growth measure is
approximately the difference between current
achievement (current results) and prior
achievement (prior results) with achievement
being measured by an appropriate assessment,
such as the PSSA.
PVAAS is NOT a simple comparison of
two scores!
Value-Added & PVAAS



Pennsylvania’s model for value-added is called
PVAAS - the Pennsylvania Value-Added
Assessment System.
PVAAS is based on the EVAAS Methodology –
the Education Value-Added Assessment
System.
The EVAAS methodology has been nationally
reviewed and published.
Benefits of Value-Added


Offers an objective, more accurate way to measure
the influence districts and schools have on students’
academic progress:
Administrators and Teachers can:
• Monitor the progress of all groups of students from lowachieving to high-achieving ensuring growth
opportunities for all students
• Make more informed, data-driven decisions about where
to focus resources to help students make greater
progress and perform at higher levels.
• Align professional development efforts in the areas of
greatest need.
• Identify best practices and implement instructional
strategies and programs that best meet the needs of all
students.
Key Concepts in Understanding
PVAAS Reporting
 PVAAS reporting reflects the
effectiveness of your district’s or school’s
Standards-Aligned System
 PVAAS reporting reflects the district or
school’s system regarding curriculum,
assessment, and instruction
Overview of PVAAS
Public Reports &
Features
Achievement + Growth
Achievement results (PSSA) and
growth results (PVAAS) must be
used together to get a complete
picture of student learning.
To view the achievement results
for Pennsylvania's public
districts/schools, go to:
http://paayp.emetric.net/
PVAAS Login Page
https://pvaas.sas.com
Public Home Screen
PVAAS Public Reports
Use of Reports Tab to Select & View Reports
PVAAS Public Reports
Use of Tests Tab to View Reports at Different
Grade Levels for the District and School
Value-Added Summaries
PVAAS Public Reports

Value-Added Summary Reports
• District/LEA and School Level data only
• Math and Reading

Grades 4-8 and 9-11
• Science and Writing, Grades 9-11

School Search Capability
• Allows users to find and view the progress of
local schools, charter schools, and full-time
CTCs across Pennsylvania.
• Can search for similar schools based on grade
levels tested, various demographics,
Intermediate Unit (IU) region and/or county.
PVAAS Public Reports
(continued)

Web-based scatter plots
• Math, Reading

Grades 4-8 & Grades 9-11
• Science and Writing, Grades 9-11
Reports on Public Site


The data on the PVAAS public site
come directly from the reports on the
district/school password-protected
site.
This session will include a cross-walk
between these two sites to show the
source of the data on the public
reporting.
Purpose of District & School
PVAAS Data
• Provides users with information to
assist them in evaluating the overall
effectiveness of a district/LEA or
school on the academic progress of
groups of students.
• This report is NOT a report on
teacher effectiveness!
What is New?
Since Feb 2011 Public Reporting Release:
• New: Web-based Scatterplots
• New: Color Scheme/Legends
• New: Addition of Science and Writing
for Grades 9-11
• New: Rollover Text Feature
• New: Redesigned School Search Feature
Web-based Scatter Plots
Accessing Scatter Plots

The scatter plot report can
be accessed directly
within the orange
“Reports” tab
Customizing Scatter Plots
Users can
switch
subjects
Users can
also switch
the the X and
Y Axis
Roll-Over Feature!
Website
Demonstration
https://pvaas.sas.com
District and School
Value-Added Reports
PVAAS Value-added
Growth Descriptors
Grades 4-8 Math & Reading
Grades 9-11 Math, Reading, Science & Writing
Example of District Value-Added
Summary Report
Grades 4-8, Math & Reading
Example of School Value-Added
Summary Report
Grades 4-8, Math & Reading
Where can I find this information
on other reports?
Public Site: School Value-Added Summary Report
Restricted Site: School Value-Added Report
Example: Growth Measure over Grades
Relative to the Growth Standard
Growth Measure over Grades Relative to the Growth Standard (on the public
report) is the SAME as the Growth Measure over Grades Relative to the Growth
Standard on the School Value-Added Report (password-protected site)!
3.2
4th Grade
Growth
0.0
5th
Grade
Growth
2
1.6 Growth Measure over Grades
Relative to Growth Standard
What is the Growth Measure over
Grades Relative to the State?
• Represents the average growth across the grade levels
served between 4 and 8 compared to the average
progress of all students in Pennsylvania at the same
grade levels.
• It is the average academic growth of the district’s or
school’s students, compared to the academic growth of
students statewide.
• Answers the question, “How much did the
district/LEA/school impact the academic progress of its
students compared to the progress of other students in
Pennsylvania in those same grade levels?”
Example: Growth Measure over Grades
Relative to the State
Growth Measure over Grades Relative to the State (on the public report) is the
SAME as the Growth Measure over Grades Relative to the State on the School
Value-Added Report (password-protected site)!
1.6
?
Growth Measure over Grades
Relative to Growth Standard
for the school
Average Growth over
Grades Relative to Growth
Standard for the State
0.9 Growth Measure over
Grades Relative to the State
What is Growth Measure over Grades Relative
to the Growth Standard for the State?
2.9
-1.4
Grade 4 State
3-Yr-Avg
Grade 5 State
3-Yr-Avg
2
.75 Average Growth over Grades
Relative to Growth Standard for the State
Example: Growth Measure over Grades
Relative to the State
Growth Measure over Grades Relative to the State (on the public report) is the
SAME as the Growth Measure over Grades Relative to the State on the School
Value-Added Report (password-protected site)!
1.6
.75
Growth Measure over Grades
Relative to Growth Standard
for the school
Average Growth over
Grades Relative to Growth
Standard for the State
0.9 Growth Measure over
Grades Relative to the State
Example of District Value-Added
Summary Report
Grades 9-11, Math, Reading, Science & Writing
Example of School Value-Added
Summary Report
Grades 9-11, Math, Reading, Science & Writing
Where can I find this
information on other reports?
Public Site: School Value-Added Summary Report
Restricted Site: School Value-Added Report
Is it appropriate to compare the amount of
progress made by a district/school to
another district/school?
• Without taking the Standard Error into account, it is NOT
possible to directly compare growth measures across
districts/schools.
• The color-coding of the growth measures does in fact take
into account the Standard Error.
o Note the link to the color code legends on each report.
• The Average Growth Index found on the School Search
report feature takes the Standard Error into account and
allows a more direct comparison across schools.
SCHOOL SEARCH FEATURE
Purpose of School Search
• Users can find and view the
progress of public schools across
Pennsylvania and search for similar
schools based on grade levels
tested, various demographics,
Intermediate Unit (IU) region,
and/or county.
Which schools are included
when I use School Search?


Schools with at least one tested grade in
common as the “reference school” you
selected.
Example: Your reference school is a
grade 6-8 school
• Other schools included in the search may
include grade 6-7 schools, grade 7-8 schools,
K-6 schools, etc. – unless you specific a grade
range.
Accessing School Search
PVAAS School Search is
accessed the via links at the
top of the page or using the
orange Reports tab
Predictive Text to Assist in
Locating Schools
Example: the user enters the word “Pink”
and school names containing the word
“Pink” appear as options to select.
School Search Results
School Search Results
Finding Similar Schools
Comparison Schools
Website
Demonstration
https://pvaas.sas.com
How are schools compared?


PVAAS: Average Growth Index
What is an Index?
• A numerical scale used to compare variables
with one another or with some reference
number

Analogy: Consumer Price Index
• A measure of the average change over a period
of time
• Statistical Indicator
• Reflects patterns

The PVAAS Average Growth Index allows
viewers to compare growth across schools.
What is the Average Growth Index?
•
A measure of student progress across the tested grade
levels in a school.
• This index is a value based on the average growth across
grade levels and its relationship to the standard error so that
comparison among schools is meaningful. If the standard
error is not accounted for, users might get a skewed picture
of the relative effectiveness of different schools.
• For grades 4 through 8, the Average Growth Index is
calculated by dividing the Growth Measure over Grades
Relative to the Growth Standard by the corresponding
Standard Error.
• For grades 9 through 11, the Average Growth Index is
calculated by dividing the Growth Measure by the
corresponding Standard Error.
Example: Average Growth Index,
Grades 4-8
Average Growth Index
= Growth Measure over Grades Relative to the Growth Standard divided by
the Standard Error
What does the Average Growth
Index mean?
•Average Growth Index (AGI) = 0: The average achieving
student in this school met the standard for PA Academic
Growth.
•Average Growth Index (AGI) > 0: The average achieving
student in this school exceeded the standard for PA
Academic Growth. A larger AGI provides more evidence that
the average achieving student in this school exceeded the
standard for PA Academic Growth.
•Average Growth Index (AGI) < 0: The average achieving
student in this school did not meet the standard for PA
Academic Growth. A smaller AGI provides more evidence
that the average achieving student in this school did not
meet the standard for PA Academic Growth.
Why can I not find a district/school
on the PVAAS public site?
Reporting was suppressed for the following
schools:
• Districts/schools with less than 10 students

PA public reporting requirements
• Districts/schools serving one grade level




Act 104 legislative intent was district and
building level data
Example: Grade 11 only school
Example: Grade 11-12 schools
Example: Grade 6 only school
Suggested Resources
for Communicating
PVAAS
PVAAS Help Menus on
Public Reporting
PDE PVAAS
Webpage
•Help Menus
•PowerPoint
Presentations
with Trainer
Notes
• Resource
Materials
• Video Clips
• District Case
Studies
Suggested Resources

Guide for PVAAS Public Reporting 2011
•
•
•
•
•
•
Differences between Achievement and Progress
Concept of Value-Added
Benefits of Value-Added
Description of Reporting
Frequently Asked Questions
Glossary of Key Terms & Acronyms
Available on the PVAAS login page!
24/7 Suggested Resources

Archived webinars and PowerPoint
presentations detailing the public reporting
site

One-page handout titled “What is PVAAS?”

PVAAS Podcasts, FREE on iTunesU
• Introduction to PVAAS
• Value-Added Reporting
• School Search
Why PVAAS?
• Provides users with information to
assist them in evaluating the overall
effectiveness of a public district or
public school on the academic
progress of groups of students.
Questions?
Comments?
Questions:
PVAAS Materials or
Statewide Implementation
[email protected]
717-606-1911
PVAAS Report Web Site
https://pvaas.sas.com
www.pde.state.pa.us
333 Market Street
Harrisburg, PA 17126