Transcript Measuring” What We Do: The Changing Landscape of Teacher

Measuring Up: Effective
Strategies for Teacher
Evaluation
Patricia A. Popp, Ph.D.
State Coordinator, Project HOPE-VA
Clinical Associate Professor
[email protected]
Xianxuan Xu, Ph.D.
Post-Doctoral Research Associate
[email protected]
School of Education
The College of William and Mary
Teacher Evaluation in an Era of
Accountability
• Currently, ESEA flexibility has been granted to 34 states
and the District of Columbia. As part of the flexibility
requirements, the states were required to establish new
teacher evaluation systems that factor in student
achievement progress for statewide implementation by
the end of the 2013-2014 school year.
•
All states that received Race to the Top funding are
undertaking substantial reforms with teacher evaluation.
• “The quality of an education system cannot exceed the
quality of its teachers” (Barber & Mourshed, 2008).
Share of At-Risk Students on PISA
(Not reaching PISA baselines): Reading
25
21
20
15
11
10
10
8
8
5
5
0
Canada
Finland
Shanghai
Singapore South Korea
United
States
Share of At-Risk Students on PISA
(Not reaching PISA baselines): Math
25
21
20
15
11
10
10
8
8
5
5
0
Canada
Finland
Shanghai Singapore
South
Korea
United
States
Percentage of Country’s Students in
PISA Top Performing Groups: Reading
20
18
16
14
12
10
8
6
4
2
0
19
16
15
13
13
10
Canada
Finland
Shanghai
Singapre
South
Korea
United
States
Percentage of Country’s Students in
PISA Top Performing Groups: Math
60
50
50
40
36
30
20
18
26
21
10
10
0
Canada
Finland
Shanghai
Singapre
South
Korea
United
States
What do we do
about it?
How Long Do Students Attend School?
Days in the Academic Year
Country
Days in an Academic
Year
Canada
Finland
Singapore
Shanghai
South Korea
United States
Average: 188
187
200
180
204
180
How Long Do Students Attend Schools?
Minutes in the Day
Location
Canada
Finland
Shanghai
Singapore
South Korea
United States
Minutes in a School
Day
304
240
390
330
264
402
How Much Do We Spend?
Location
Canada
Finland
Annual Expenditures Per Pupil
8,045
7,216
Shanghai
Singapore
South Korea
N/A
N/A
6,663
United States
10,259
Student/Teacher Ratio
Country
Average
Student/Teacher Ratio
Canada
Finland
Shanghai
Singapore
South Korea
United States
25
19
39
35
36
24
Source: Available at worldbank.org.
Which factor is a strong predictor of
student achievement gains?
Class size
Classroom heterogeneity
School resource differences
It’s the teacher.
Sources: Wright, Horn, & Sanders, 1997;
Hanushek, Kain, & Rivkin, 1998,
Influences on Student Achievement:
Explained Variance
Teachers
30%
Home
5-10%
Students
50%
School
5-10%
Peers
5-10%
Source: Hattie, J. Teachers make a difference: What is the research evidence.
Retrieved 20Nov08 from http://acer.edu.au/documents
Dallas Research: Teacher Quality
4th gr. Math Achievement
4th grade
1st grade
100
90
80
70
60
50
40
30
20
10
0
Highly Effective
Ineffective
Dallas, Texas data: 2800-3200 students per cohort
Comparison of 3 “highly effective” & 3 “ineffective” teachers (Jordan, Mendro, & Weerasinghe, 1997)
Dallas Research: Teacher Quality
4th gr. Reading Achievement
70
60
50
40
30
20
10
4th
Grade
1st
Grate
0
Highly Effective
Ineffective
Time in School Year Needed to Achieve
the Same Amount of Learning
75th Percentile
Teacher
25th Percentile
Teacher
0
1/4
1/2
3/4
1
Years Needed
Leigh, Economics of Education Review (2010)
Time in School Needed to Achieve the
Same Amount of Learning
T ime in School Year Needed to Achieve the Same Amount
of Learning
90th Percentile
Teacher
10th Percentile
Teacher
0
1/4
1/2
3/4
Years Needed
Source: Leigh, A. (n.d.). Estimating teacher effectiveness from two-year changes in
students’ test scores. Retrieved May 22, 2007, from http://econrsss.anu.edu.au/~aleigh/.
1
Sequence of Effective Teachers
Low
Low
Low
High
High
High
52-54
percentile
points
difference
Sanders & Rivers (1996)
Sequence of Effective Teachers
Low
Low
High
High
High
High
13
percentile
points
difference
Sanders & Rivers (1996)
Revised Teacher
Evaluation System
in Virginia: An
Overview
Primary Purposes of the Evaluation
System
• Improve student achievement through the quality of
instruction by assuring accountability for classroom
performance
• Contribute to the successful achievement of the goals
and objectives defined in a school division’s
educational plans
• Provide a basis for instructional improvement through
productive teacher appraisal and professional growth
• Share responsibility for evaluation between the teacher
and the evaluation team in a collaborative process that
promotes self-growth, instructional effectiveness, and
improvement of overall job performance
A Flawed System
Problem No. 1: Observation equals evaluation
Problem No. 2: Likely to rely on intuition, not evidence, to
make judgments about teacher performance
Problem No. 3: One size fits all
Problem No. 4: Don’t communicate
Problem No. 5: Fragmented evaluation process
Problem No. 6: Irrelevant evaluation
Problem No. 7: One-point rating scales
Problem No. 8: No impact evaluation
Question 1
What is the basis of the
teachers’ evaluation?
Main Components
Performance
Standard
Standard 2: Instructional Planning
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective
strategies, resources, and data to meet the learning needs of all students.
Sample Performance Indicators
Examples may include, but are not limited to:
The teacher:
2.1
2.2
2.3
2.4
2.5
Performance
Indicators
Uses student learning data to guide planning.
Plans time realistically for pacing, content mastery, and transitions.
Plans for differentiated instruction.
Aligns lesson objectives to the school’s curriculum and student learning needs.
Develops appropriate long- and short-range plans, and adapts plans when needed.
Proficient
Exemplary
In addition to meeting the
standard, the teacher actively
seeks and uses alternative
data and resources and
consistently differentiates
plans to meet the needs of all
students.
Proficient is the expected
level of performance.
The teacher plans using
the Virginia Standards of
Learning, the school’s
curriculum, effective
strategies, resources, and
data to meet the needs of
all students.
Developing/
Needs Improvement
The teacher inconsistently
uses the school’s curriculum,
effective strategies,
resources, and data in
planning to meet the needs
of all students.
Performance
Appraisal
Rubric
Unacceptable
The teacher does not plan, or
plans without adequately
using the school’s curriculum,
effective strategies,
resources, and data.
Performance Standards
Professional
Knowledge
Professionalism
Instructional
Planning
Learning
Environment
Instructional
Delivery
Assessment of
and for Student
Learning
Student
Academic
Progress
Teacher Performance Standard 1:
Professional Knowledge
The teacher demonstrates an understanding
of the curriculum, subject content, and the
developmental needs of students by providing
relevant learning experiences.
Teacher Performance Standard 2:
Instructional Planning
The teacher plans using the Virginia Standards
of Learning, the school’s curriculum, effective
strategies, resources, and data to meet the
needs of all students.
Teacher Performance Standard 3:
Instructional Delivery
The teacher effectively engages students in
learning by using a variety of instructional
strategies in order to meet individual learning
needs.
Teacher Performance Standard 4:
Assessment of and for Student Learning
The teacher systematically gathers, analyzes,
and uses all relevant data to measure student
academic progress, guide instructional content
and delivery methods, and provide timely
feedback to both students and parents
throughout the school year.
Teacher Performance Standard 5:
Learning Environment
The teacher uses resources, routines, and
procedures to provide a respectful, positive,
safe, student-centered environment that is
conducive to learning.
Teacher Performance Standard 6:
Professionalism
The teacher maintains a commitment to
professional ethics, communicates effectively,
and takes responsibility for and participates in
professional growth that results in enhanced
student learning.
Teacher Performance Standard 7:
Student Academic Progress
The work of the teacher results in acceptable,
measurable, and appropriate student academic
progress.
Question 2
How will teacher
performance be
documented?
Multiple Data Sources
Data Source
Evaluator
Informal Observations

Formal Observations

Teacher
Student Surveys

Portfolios/Document Logs

Self-Evaluation

Measures of Academic Progress Reviews/approves
Selects/develops
Measures of Academic Progress
Percentage of Evaluation
based on Student Growth
Percentiles (SGPs)
Percentage of Evaluation
based on Other Growth
Measures
20
20
Teachers who support
instruction in reading and
mathematics for whom
SGPs are available
No more than 20
20 to 40
Teachers who have no
direct or indirect role in
teaching reading or
mathematics in grades
where SGPs are available
N/A
40
Teachers
Teachers of reading and
mathematics for whom
SGPs are available
Student Achievement Goal Setting
Step 1:
Determine
needs
Step 2:
Create specific
learning goals
based on preassessment
Step 3:
Step 4:
Create and
Monitor student
implement
progress through
teaching and
ongoing formative
learning strategies
assessment
Step 5:
Determine
whether the
students
achieved the
goal
What are the Purposes of
Student Achievement Goal Setting?
 Focus on student results
 Explicitly connect teaching and learning
 Improve instructional practices and teacher
performance
 Tool for school improvement
Question 3
How will teacher
performance be rated?
Evaluations
Interim Evaluation
• Used to document evidence of meeting standards
• Does NOT include rating of performance
Summative Evaluation
• Comes at end of evaluation cycle
- One year for probationary teachers
- Three years for continuing contract teachers
• Assessment of performance quality
- Four point rating scale
- Performance rubric for every standard
Evaluating Performance
Category
Description
Definition
Exemplary
The teacher maintains
performance, accomplishments,
and behaviors that consistently and
considerably surpass the
established standard.
Exceptional Performance
Proficient
The teacher meets the standard in
a manner that is consistent with the
school’s mission and goals.
Effective Performance
Developing/
Needs
Improvement
The teacher often performs below
the established standard or in a
manner that is inconsistent with the
school’s missions and goals.
Below Acceptable Performance
Unacceptable
The teacher consistently performs
below the established standards or
in a manner that is inconsistent with
the school’s missions and goals.
Ineffective Performance
• Sustains high performance over period of time
• Behaviors have strong positive impact on
learners and school climate
• Serves as role model to others
• Meets the requirements contained in job
description as expressed in evaluation criteria
• Behaviors have positive impact on learners and
school climate
• Willing to learn and apply new skills
• Requires support in meeting the standards
• Results in less than quality work performance
• Leads to areas for teacher improvement being
jointly identified and planned between teacher
and evaluator
• Does not meet requirements contained in job
description as expressed in evaluation criteria
• Results in minimal student learning
• May result in employee not being
recommended for continued employment
Sample Performance Appraisal Rubric
Standard I: Professional Knowledge
The teacher demonstrates an understanding of the curriculum, subject
content, and the developmental needs of students by providing relevant
learning experiences.
Proficient
Exemplary
Proficient is the expected
level of performance.
In addition to meeting
The teacher
the standard, the teacher demonstrates an
consistently
understanding of the
demonstrates extensive
curriculum, subject
knowledge of the subject content, and the
matter and continually
developmental needs of
enriches the curriculum. students by providing
relevant learning
experiences.
Developing/Needs
Improvement
The teacher
inconsistently
demonstrates
understanding of the
curriculum, content, and
student development or
lacks fluidity in using the
knowledge in practice.
Unacceptable
The teacher bases
instruction on material
that is inaccurate or outof-date and/or
inadequately addresses
the developmental needs
of students.
Summative Rating
Summative ratings should apply the rating for each of the
seven performance expectations, with the most significant
weight given to Standard 7 - Student Academic Progress.
• Weight each of the first six standards equally at 10
percent each
• Weight Standard 7 – Student Academic Progress at 40
percent
Lessons Learned
 Most practitioners believe new models provide specific
measures of teacher effectiveness that are useful for
distinguishing effective from less effective teachers.
Proficient
Exemplary
Proficient is the expected
level of performance.
In addition to meeting the
standard, the teacher actively
seeks and uses alternative
data and resources and
consistently differentiates
plans to meet the needs of all
students.
The teacher plans using the
state’s standards, the
school’s curriculum, effective
strategies, resources, and
data to meet the needs of all
students.
Developing/Needs
Improvement
The teacher inconsistently
uses the school’s curriculum,
effective strategies,
resources, and data in
planning to meet the needs
of all students.
Unacceptable
The teacher does not plan, or
plans without adequately
using the school’s curriculum,
effective strategies,
resources, and data.
Lessons Learned
 Most practitioners believe that new models have the potential
to improve teaching and learning by providing useful feedback
that can be used to diagnose and guide teacher improvement.
Formal Observation Post-Conference
Teacher Self-Assessment
Student Learning Objectives
Student Surveys
Lessons Learned
 With more rigorous assessment to identify problems and
recognize excellence, investments in teacher development can be
better related to school and division goals for improvement.
Evaluation
Strengths and
Weaknesses
Identified
Targeted
Professional
Development
Lessons Learned
 Practitioners appreciate the value in using multiple data sources to
provide evidence of performance standards.
Lessons Learned
 Practitioners believe that new models set up realistic expectations
for teacher performance, and they reflect the most important
elements of effective teaching.
Lessons Learned
 The evaluation framework is valid in terms that the process
standard ratings of teacher have a moderate ability to predict the
student academic progress. In addition, there is a significant
correlation between each of the six process standards and student
academic progress.
Lessons Learned
 New models can be time-consuming to implement.
 Practitioners mistrust the validity of student progress models as
a measure of student growth.
 Teachers believe that the nature, quality, and credibility of the
evaluation process vary depending on the qualifications of
the evaluators.
Concluding Thoughts on Transforming
Teacher Evaluation
State
• Convene stakeholders across the state to design, implement,
and improve evaluation system.
• Develop validated and reliable evaluation measures.
• Provide incentives and on-going support to school divisions.
• Make knowledge of new developments in teacher evaluation
part of leadership and teacher preparation programs.
• Make evaluation count.
Concluding Thoughts on Transforming
Teacher Evaluation (Continued)
Schools and School Divisions/Districts
• Clear expectations of the WHAT will be evaluated.
• Clear communication of the evaluative criteria for
effective performance.
• Instruments and procedures for teachers to provide
evidence from multiple sources of the HOW of their
performance.
• Increase the use of evaluation results to inform
professional development.