Transcript Pay for Performance

The DPS/DCTA Pay for Performance Pilot
An Introduction and Update
Brad Jupp, Design Team
Shirley Scott, Design Team
Presented to the 2002 CPRE National Conference on Teacher
Compensation and Evaluation
November 22, 2002
Agenda
1)
2)
3)
The Pay for Performance Pilot -- an
Introduction and Update
Some Recent Findings
Eight Building Blocks for Positive
School District Change Based on
Labor/Management Partnership
The Pay for Performance Pilot -- An
Introduction and Update
1.
2.
3.
4.
5.
6.
7.
8.
The Basics
The District and Association
The Schools
The Teachers and Principals
The Objectives
Student Learning
District Capacity
How the Pilot Will Conclude
An Introduction and Update -- The
Basics
The Premise:
Teacher compensation
should be based, in part, on
the academic growth of the
students that they teach.
An Introduction and Update -- The
Basics
The Pilot in a Paragraph
The DPS/DCTA Pay for Performance Pilot
investigates setting measurable student
achievement objectives. It investigates offering
teachers bonuses for meeting those objectives.
It will conclude in March 2004, when the
Design Team will make a recommendation to
the teachers and the Board of Education. The
final recommendation will include a radically
redesigned salary structure, not just a system of
bonuses.
An Introduction and Update -- The
District and Association
August 1999 -- “A landmark agreement”
 April 2000 -- a four year pilot
 April 2001 -- the Joint Task Force on
Teacher Salary

 A climate of tactical thinking about teacher
compensation

May 2002 -- “Partnership Agreement”
An Introduction and Update -- The
District and Association
Collective Bargaining spring and summer 1999
 The district position
 teacher experience steps should be earned based on student
achievement
 an attractive salary schedule should be a part of the trade-off
• entry pay tops the market
• “squared range” with a fixed but diminished ratio between cells
• MA60 becomes the top earning education column

The association position
 The salary system is attractive, but the objective setting process is
untested, so run a pilot on the objective setting process.
 National research (anchored by the National Commission for
Teaching and America’s Future) points to the importance of skills and
knowledge pay
An Introduction and Update -- The
District and Association

The Memorandum of Agreement on Pay
for Performance, August 1999
Objective setting for bonuses
Schools enter by election
Comparative model -- three approaches
Design Team
Outside expert to evaluate the pilot
An Introduction and Update -- The
Schools

Sixteen of Denver’s 135 schools
participate in the pilot. Schools joined
the pilot by successful faculty election
Twelve elementary schools, including eleven
that have been in the been in the pilot since
October 1999
Two middle schools
One high school and one “high school
education complex”
he
ll
an
ua
l
an
n
bi
Rat in g
Scho ol s
P ercent Free Lu n ch
m
oo
r
ry
Co
r
lo
so
n
Tr
ay
Ed
i
n
al
Je
ff e
rso
So
uth
T
Ce
nt
en
ni
El
lis
d
la n
Oa
k
s
an
Ph
ili
p
Co
lum
lf a
x
Co
Fa
irv
iew
M
M
La
ke
itc
Ho
ra
ce
M
Percent Free Lunch
Excellent
High
Average
Average
Low
Low
Low
Low
Low
Low
Low
Low
Unsatisfactory
Unsatisfactory
Unsatisfactory
Unsatisfactory
An Introduction and Update -- The
Schools
Pay for Performance School Ratings Compared with Percent Free Lunch
Ra ting s
1 20
1 00
80
60
40
20
0
An Introduction and Update -- Teachers
and Principals

There were 18 principals and 4 assistant
principals in the pilot in the 01-02 school year.
Two principals have been in the pilot since its
beginning, a turnover rate similar to the overall
turnover rate of principals in DPS.

There were 633 teachers in the pilot in the 0102 school year.
124 of the original 307 pilot teachers remain in their
same schools, a turnover rate similar to the overall
turnover rate in DPS.
An Introduction and Update -- The
Objectives

Teacher objectives are the practical cornerstone
of the pay for performance pilot.
Teachers set two objectives and receive a bonus of
$750 if they meet the objective.
There was no clear compensation theory underlying
the model -- in other words, there was not
agreement that the bonus was to motivate, reward,
punish or create an incentive.
During the pilot, the work of the Design Team
evolved into the work of implementing this objective
setting system.
An Introduction and Update -- The
Objectives

Objectives contain two bodies of information,
measurement content and learning content.
Measurement content includes the technical
information -- assessment, population, interval and
expected gain -- that is used to set and measure
progress toward expectations
Learning content includes the major elements of the
teacher’s strategic thinking about student
expectations -- rationale, student learning priorities
and teaching strategies.
An Introduction and Update -- The
Objectives
Total
Objectives Objectives
Objectives
Met
Not Met
01-02
01-02
01-02
Number of
Objectives
Percentage
of
Objectives
1266
1113
157
100%
88%
12%
An Introduction and Update -- Student
Learning

Evidence that student achievement or student learning
has improved in pilot schools, or is better than student
learning in schools outside the pilot, is mixed.
 Design Team research shows there is no obvious correlation
between percentage of objectives met and student achievement
levels or improvement in student achievement levels at the
school level.
 CTAC research shows that there is correlation between the
quality of objectives and student performance on independent
measures of student achievement.
 CTAC research shows that pilot schools outperformed
comparison schools on some tests but not on others.
• CTAC research has not correlated those differences in
performance to objective setting or pay for performance.
An Introduction and Update -- District
Capacity

District capacity to support the objective
setting process has grown over the course of the
pilot
 The district has provided teachers and administrators on-line
access to a wide range of student achievement information
• In September 2002, OASIS had an average of 50 hits a day.
 The district has placed the objective setting in a web-bsed
environment
• In the 01-02 school year 632 teachers (100% of
participants) completed their objectives on line.
 The district is aligning local, state and federal accountability
initiatives through a common set of performance indeicators.
An Introduction and Update -- District
Capacity

The district and association have engaged in at
least 6 different efforts at alternative
compensation since 1994
 Market incentives for ELA S teachers
 Salary freezes for teachers with unsatisfactory principal
evaluations
 Tuition supplements and extra pay for teachers with National
Board Certificates
 Differentiated pay for Teachers in Residence
 Market incentives to attract and retain hard to recruit
positions
 Extra pay for instructional coaches in the literacy program
An Introduction and Update -- How the
Pilot will Conclude






August 2002 -- the final year that teachers and principals set
objectives in 16 schools
April 2003 -- The Joint Task Force on Teacher Compensation
presents its first draft recommendation of a teacher compensation
system
October 2003 -- The Joint Task Force presentts its revised draft of
a recommendation.
December 2003 -- The Community Training and Assistance Center
presents its research study on the pilot
January 2004 -- DPS and DCTA will develop a collective
bargaining agreement based on the recommendations of the Joint
Task Force and the CTAC research study
March 2004 -- the teachers and the Board of Education vote on the
collective bargaining agreement
Some Recent Findings

Sources
1. Evidence from teachers gathered by the
DCTA
2. Research by the Community Training and
Assistance Center (CTAC)
3. Research by Ciruli and Associates
4. Evidence from teachers, principals, schools
and DPS gathered by the Design Team
5. The experience of the Design Team
Findings to Date -- Three Areas
1.
2.
3.
It is a sound practice to set measurable objectives based
on student growth and based in the teacher’s discipline
Educators are clearly resistant to the concept of pay for
performance, but remain open-minded about teacher
compensation in general and the results of the Pay for
Performance Pilot in specific, and have mixed feelings
about the compensation system used in the pilot.
Educators are unclear about the relation between teacher
objectives and other accountability initiatives, like
accreditation, school improvement planning and,
especially teacher evaluation
1. It is a sound practice to set objectives
a)
b)
c)
d)
CTAC Research shows that when teacher objectives are clear about
learning content, complete and cohesive, and express high and
measurable expectations, they correlate with higher student
achievement on independent assessments.
Educators and CTAC research report that the greatest potential of the
objective setting practice is the professional conversation between
principal and teacher to focus on student growth expectations
Teachers and principals in the pilot report that the objective setting
process has worked best where there is effective principal leadership
Teachers and principals report that they rely on training, clear
procedures, and information about how to quantify student growth
expectations in order to set high quality objectives..
1. It is a sound practice to set objectives
e)
f)
g)
h)
Teachers and principals report that they need flexibility to adjust
teacher objectives to meet the unique needs of the students they
teach.
Teachers and principals report that they need timely and reliable
access to meaningful student growth data if they are to set rigorous
objectives.
Teachers and principals report that it is easier for teachers and
principals to set high quality objectives when there are performance
standards for students and well-aligned assessments that measures
student growth.
The Design Team and CTAC have reported that there has been little
supervision of the objective setting process by anyone other than the Design
Team, whether by district education staff (like area or level superintendents)
or by central departments (like Planning or Curriculum).
1. It is a sound practice to set objectives
i)
While the Pay for Performance Memorandum of Understanding between
DPS and DCTA provides for extraordinary dispute resolution procedures
for teachers and principals, they have not been used during the life of the
pilot.
2. Educators have mixed feelings about the
compensation system used in the pilot
a)
b)
c)
d)
Research by CTAC, Gonder, Baird and Ciruli Associates all shows
that teachers are resistant to the concept of “pay for performance”
In spite of this obvious resistance, research by CTAC shows not only
that the majority of educators are open-minded about the results of
the pilot, but also that a greater portion of pilot educators describes
themselves as open-minded
Research by CTAC shows that many educators believe that pay for
performance is inevitable.
Research by DCTA and CTAC indicates that teachers do believe that
some portion of their pay or some portion of their evaluation should
be accurately measured student growth.
2. Educators have mixed feelings about
the compensation system used in the pilot
e)
f)
g)
Research by CTAC mines a much richer body of teacher
perceptions that shows a common basis of educator
expectations based on what the Design Team Calls
“worst fear elements” and “perceived elements of sound
practice.”
Research by CTAC also points to places where pilot
teachers than non-pilot teachers are more optimistic when
they anticipate a future pay for performance system.
The Design Team has explored teacher attitudes in
systematic faculty meetings held to gather teacher
perceptual data and found that teacher attitudes about
receiving bonuses for meeting objectives are ambivalent.
3. Educators are unclear about how teacher
objectives fit with other accountability
measures
a)
b)
c)
d)
e)
The Design Team has found that teachers and administrators are confused
about the relation between objective setting and teacher evaluation.
The Design Team and CTAC research have found that the difference
between administrator pay for performance and teacher pay for performance
has confused principals and teachers.
The Design Team and CTAC research have found that the current school
improvement planning process has little consistent impact the setting of
teacher objectives.
The Design Team and CTAC research have found that there is little or no
connection between school accreditation and other school based
accountability initiatives, such as school improvement planning and teacher
objective setting.
The Design Team and CTAC research have found that there is little or no
connection between Colorado School Accountability Reporting and other
school based accountability initiatives, such as school improvement planning
and teacher objective setting.
Eight Building Blocks for Positive School District
Change Based on Labor/Management Partnership
1.
2.
3.
4.
5.
6.
7.
8.
Saying yes was more powerful than saying no.
Neither the district nor the association will be perfectly
ready to begin.
Changing teacher compensation will not necessarily lead
to urban school reform.
Do not be afraid to work with student achievement.
There must be shared commitment to complete all reform
efforts.
Current district and union capacity is not sufficient.
Be prepared to pay.
Evaluate everything.
Thank You.
DPS/DCTA Pay for Performance
Design Team
http://denverpfp.org
[email protected] or
[email protected]
303.764.3629