Transcript PVAAS - Nov 13 2014 final
PAIU CC Meeting PVAAS Update
November 2014 Kristen Lewald, Ed.D.
Statewide PVAAS Director for PDE Lancaster-Lebanon IU13
Topics
Guide for Reflection on Teacher Specific Reporting
Demo Access Accounts
Common Questions from Admins and Teachers
Feedback from Teachers
Feedback from 499 Superintendents/Designee
Help Menu System
PVAAS Reporting/Website Updates
NEW: Guide for Reflection
PVAAS Teacher Specific Reporting
Guide for Reflection: Process for Review and Planning
Steps to Review Reports and Develop Plan of Action
Guide for Review and Planning
Two Sets of Guiding Questions:
Organized Based on Domains 1. Planning and Preparation 2. Classroom Environment 3. Instruction
Organized Based on CIAO!
1. Curriculum 2. Instruction 3. Assessment 4. Organization
Demo Account Access
PVAAS Teacher Specific Reporting
Demo Account Access
Share with LEAs
Common Questions from Admins and Teachers PVAAS Teacher Specific Reporting
Why No Report for a Teacher Who Rostered?
PVAAS Teacher Specific Reporting Common Questions
Students Included in Reporting
To actually be included in PVAAS Teacher Specific reporting, a student MUST:
• • • • • • Have a PSSA or Keystone score from the most recent year NOT be a foreign exchange student NOT be a first-year ELL student NOT be Proficient or Advanced on a PRIOR Keystone exam NOT be claimed at less than 10% instructional responsibility NOT have tested with the PASA (alternate assessment)
Minimum N Counts
Must have at least 11 students
Must have an “Active N Count” of at least 6 students/6.0 FTE Students
Why No Individual Student Growth Measure?
PVAAS Teacher Specific Reporting Common Questions
Why No Individual Student Growth Measure?
Student History Report
Growth is about a GROUP of students NOT an individual student.
Error would be too large for an individual student to know if they made growth or not.
This report should NOT be used or interpreted as growth for an individual student!
Why No Predicted Score NOW?
PVAAS Teacher Specific Reporting Common Questions
Why No Predicted Score Available NOW for SY14-15?
A student’s predicted score is based on the student’s own testing history and on the average performance of students in the same cohort statewide who have a similar testing history. In other words, for each assessment we analyze with the predictive model, we look at the past testing histories of students across the state who took that assessment in the most recent year. Because the predicted score is based on the performance of other students statewide who took the assessment in the same year, it’s not possible to generate the predicted score prior to the assessment. What would the score tell you that a PVAAS projection/% of likelihood does not?
Why Different?
School Value-Added vs. Teacher Value-Added Reporting
PVAAS Teacher Specific Reporting Common Questions
Why Different?
School Value-Added vs. Teacher Value-Added Reporting
2 Primary Reasons: PVAAS School Reporting uses full academic year/Oct 1 as a requirement for including students PVAAS Teacher Value-added reporting uses % Student + Teacher Enrollment and % Shared Instruction to determine the weighting of each student on each teacher’s PVAAS teacher Specific Reporting
Why Different?
Composite v. Teacher Value Added
PVAAS Teacher Specific Reporting Common Questions
What is the Composite?
A combined growth measure across a teacher’s PVAAS reported subjects/grades/courses within SY13-14. For SY13-14, we only have one year of reporting so the Composite Score is a combined growth measure across a teacher’s PVAAS reported subjects/grades/courses for SY13-14 only. In future years, the Composite Score will represent a combined growth measure across a teacher’s PVAAS reported subjects/grade/courses across years, up to and including 3 consecutive school years.
1 year Composite 2 Year Composite 3 Year Composite/3 Year Rolling Average 1 Pager in Develop ment
Teacher Value Added Summary
The composite is a combined measure of all tested subjects, grades, and Keystone content areas in which a teacher received a Value Added report.
Composite: How Calculated
To calculate the composite, a simple average is taken of all of the teacher's individual index values for up to three years. Then the average is multiplied by the square root of the number of individual index values that went into the average. This step is a necessary step that accounts for the fact that more data was used to generate the average than was used to generate each individual index, which affects the standard error for the composite.
Why Different?
Composite v. Teacher Value-Added
The growth color indicators are categories or ranges of indicators of growth. We cannot “average” categories. In other words, green + red ≠ yellow; likewise green + dark blue ≠ light blue.
For PVAAS, the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.
Why Different?
Composite v. Teacher Value-Added
It is all
about the amount of evidence
in the student assessment results – the assumption is that the achievement level of the teacher's group of students is maintained (green) UNLESS there is enough evidence in the assessment data to say otherwise.
The
more data available, the more evidence
OR not. we have to see if the group of students exceeded the growth standard There is
more evidence when all data was combined
for a composite score. With data from multiple subjects (or multiple years when that’s available) included in the Composite, there’s more evidence/more data.
The composite is not really an “average,” but rather
Academic Growth
.
an accumulation of the evidence/data towards meeting, exceeding, or falling short of the standard for PA
Why Different?
Teacher Value-Added v. Diagnostic Reporting
PVAAS Teacher Specific Reporting Common Questions
Why Different?
Teacher Value-Added vs. Diagnostic Reporting
Diagnostic Reports do NOT factor in the percent of Instructional Responsibility Students are weighted equally Reflect the growth of a group of students who may have had more than one teacher with instructional responsibility in that specific subject/grade or course
Methodology Questions
PVAAS Teacher Specific Reporting Common Questions
More Info About Methodology
PVAAS Methodologies, Measuring Growth & Projecting Performance – PVAAS login page, under Professional Development Virtual Learning Modules – eLearning Link Value Added Reports – Math/Reading Value Added Reports – Science/Keystones Teacher Specific Reporting EVAAS Statistical Models in PA – PDE PVAAS site, Methodology Page
Full Day TOT Sessions for Admins PVAAS Teacher Specific Reporting
Participation
450 LEAs
Over 1700 Participants
Misunderstanding
PVAAS 3 Year Rolling Average 3 Consecutive Years of PVAAS Teacher Reporting Any state assessed subject/grade/course Does not need to be in the same subject/grade/course
Half-Day Teacher Sessions at IUs PVAAS Teacher Specific Reporting
Half-Day Sessions with Teachers – We Need Your Support!
Before session
Setup of session – registration, handouts, technology Make sure all participants can get online Make sure all participants log onto PVAAS site - check username and password and PPID before training starts Download session PPT
Welcome Participants to your IU/Orient to Facility During Session
Rotate around room during online work Gather questions if not comfortable answering Engage with participants Ideas for Follow-up IU services
After Session
Gathering and Packing Up Materials Copies of Session Feedback Forms Email Final Registrant List
Feedback from Teachers
PVAAS Teacher Specific Reporting
What will you tell your colleagues about PVAAS?
Emails to 499 Superintendents/Designees
PVAAS Teacher Specific Reporting
Asked 4 Questions
How is it going overall? Were most of the reports in sync with what principals expected? Any/many surprises for the principals?
Any questions or information needed by you or your team?
“Best” Teacher
Qualities such as: students engaged in teacher’s classroom, using technology with students, positive feedback from students and/or parents, meets with students before/after school, goes to a student’s activities and sporting events, contributes to the school community, contributes to the district community, etc.
Could a teacher do all of these things listed above, but students may not be academically growing? is that possible?
Why?
Could a teacher do all of these things listed above AND their students ARE academically growing? is that possible?
Why?
NEW: PVAAS Help Menu System
PVAAS Teacher Specific Reporting
New PVAAS Help Menu System System
Updates to PVAAS Keystone Reporting
PVAAS District/School Reporting
Updates to Both PVAAS Reporting Sites:
PVAAS Restricted/Password Protected Site: Thursday, Nov. 20 th
Updates to Keystone Reporting
PVAAS Public Site: Thursday, Dec. 4 th
Updates: PVAAS Password Protected Site
• • • • • • Keystones district/school value-added reports Keystones district/school performance diagnostic reports Keystones district/school quintile diagnostic reports PVAAS SPP/AGI reports PVAAS DPP/AGI reports Changes to students included on drill down in above reports based on new business rules for Keystone reporting