Transcript Document
National Conference on
Teacher Compensation and Evaluation
Presentation of the Teacher Advancement Program
by
Lewis C. Solmon
Senior Vice President and
Director of Teacher Advancement Program
Milken Family Foundation
November 21, 2002
NCLB: Qualifications for Teachers
Any new teacher hired must meet the requirements of a
“highly qualified” teacher.
States must establish a plan to ensure that by the end of
2005-06 all teachers in core academic subjects must be
highly qualified. The plan must include annual measurable
increases towards the goal.
States and districts must begin to report progress toward
ensuring all teachers are highly qualified by 2005-06.
NCLB: “Highly Qualified”
Fully licensed or certified
No waivers or emergency credentials
At least a bachelor’s degree
Demonstrated subject matter
knowledge though state test
Teaching skills also demonstrated
through state test (elementary)
Nothing Matters More Than a
Quality Teacher
Rivers longitudinal work found that
average achieving students assigned to 4
years of ineffective teachers had only a
40 percent chance of passing the
Tennessee high school exit examination.
The same students assigned to 4 years of
effective teachers had an 80 percent
chance of passing.
Why Don’t People Choose Teaching?
Salaries not competitive
Costs of training not warranted by salary
Women have more career opportunities now
Little collegiality
Little respect from community
Often unpleasant, dangerous environment
Everyone gets same pay
Teachers Who Leave
20% of teachers leave within 3 years
50% of urban school teachers leave within 5
years
Twice as likely to leave with no induction
program
Twice as likely to leave with top scores on
high- stakes exams
Teacher Quality Efforts
Past Efforts
New Ideas
New Ways to Attract Teachers
Increase Salaries
School Debt Forgiveness
Housing Subsidies
Perks
PR Campaign
New Recruitment Strategies
Accelerated Teacher Education
More Rigorous Training
Drawbacks to Programs for Attracting
& Retaining High Quality Teachers
small
isolated efforts
not school-centered
poorly designed
poorly implemented
rather than systemic reforms
solve one problem only to create another
Teacher Advancement Program (TAP)
GOAL OF TAP:
Increased Student Achievement
METHOD FOR GETTING THERE:
Maximize Teacher Quality
HOW TO DO THAT:
Comprehensive Reform to Attract, Motivate and
Retain High Quality Teachers
TAP is a Comprehensive Reform
ELEMENTS OF THAT REFORM:
1. Multiple Career Paths
2. Market-Driven Compensation
3. Performance-Based Accountability
4. Ongoing, Applied Professional Growth
5. Expanding the Supply of High Quality Educators
NOTE:
Schools can implement TAP in conjunction with:
Effective curricula
New management methods
Community involvement initiatives, etc.
Teacher Advancement Program
The expansion of the pool is achieved by:
Initial academic degree and teaching certification
attainable in four years
Alternative certification through assessments
and classroom demonstration
Outstanding retired teachers continue working on a
part-time basis as faculty fellows
Multi-state credentialing
Portable, private pension plans
Opportunity for national certification
TAP – Both Old and New
Multiple Career Paths
Career Ladders
Performance Pay
Odden, Denver, Cincinnati
Assessment
Sanders, Danielson, NBPTS
Professional
Development
???
Expanding the Pool
N.J. Alt. Cert, Troops to
Teaches, Teach for America
TAP is unique because it ties both teacher performance
assessment & student value added to teacher compensation
& supports that with a unique professional development
program.
Models for the Teaching Profession:
Career Advancement
Traditional Model
TAP Model
Single Career Path
Multiple Career Paths
Teacher Position Only
Senior, Mentor & Master Positions
Requiring the same level of:
• professional qualifications
• responsibility
• authority
• assessment rigor
Requiring increasing levels of:
• professional qualifications
• responsibilities
• authority
• assessment rigor
Improving Teacher Quality and
Career Advancement
Currently, significant career advancement
in the teaching profession requires moving
out of the classroom and out of teaching.
Models for the Teaching Profession:
Compensation
Traditional Model
Salary Schedule Drives
Compensation
Lock-step salary
determined only by years of
experience and training
units accrued
TAP Model
Performance and
Responsibility Drive
Compensation
Salary determined by level of
responsibilities and
effectiveness of performance
Teacher Advancement Program
Higher pay is granted for the following:
If the teacher’s primary field is difficult to staff,
and if the teacher is in a hard-to-staff school
Higher teacher training levels and relevant degrees
Excellent teacher performance, as judged by
experts
Different functions/additional duties
High student achievement
Other plans reject pay based on
Judgment of others
Student achievement/ test scores
Subject specialty
How TAP Compensation System
Has Evolved
Performance awards
bonus earned each year.
not cumulative
constrained by available funds
augment salaries by $5,000 or less.
supplements traditional step & column scale.
No one earns less than in traditional compensation
system, even for poor performance.
Opportunity for all teachers to get a bonus of some
amount. NOT: only the top X% will receive bonuses.
Bonuses are criterion referenced, not relative. Any
teacher who meets a standard receives the bonus.
How TAP Compensation System
Has Evolved
50% of the bonus is awarded for skills and knowledge.
50% is based upon student achievement (value-added):
30% school-wide for all teachers
20% based on achievement of individual teacher’s
students
Teachers who score well on skills and can earn bonuses
even if students’ scores do not improve.
If teachers work more days, they must get paid for them
at least at their former daily rate.
Since subject specific tests are often unavailable, the
student achievement element of the bonus for high
school teachers is complicated.
Models for the Teaching Profession:
Professional Accountability
TAP Model
Traditional Model
Performance-based
Accountability
Uneven Accountability
• Idiosyncratic evaluation
standards & procedures
•
TAP standards, procedures
and performance rubrics
• Rewards and sanction
unrelated to evaluation
outcomes
•
Hiring, advancement and
compensation tied to
evaluation
• Support provided for
deficiencies only
•
Support provided for growth
Past Teacher Accountability Versus
TAP Teacher Accountability Efforts
Past Efforts
TAP
Two Performance Levels
Teaching Performance
Standards
Five Performance Levels
Evaluation Excludes Student
Achievement
Evaluation Includes School
and Classroom Achievement
One Evaluator
Multiple Evaluators
Evaluation Supports
Deficiencies Only
Evaluation Supports
Professional Growth
Performance Independent of
Compensation
Performance Tied to
Compensation
Checklist of Teaching Behaviors
TAP Performance-Based
Accountability Summary
Performance Indicators
Teacher Skills,
Knowledge, and
Responsibilities
Classroom- level
Achievement Gains
School
Achievement Gains
Designing and Planning
Performance Standards
Standardized
Tests
Standardized
Tests
Implementing Instruction
Performance Standards
Standards-based
content tests
Standards-based
content tests
Classroom Environment
Performance Standards
Performance
Assessments
Responsibility Standards
Measuring Classroom and School
Wide Value-Added Achievement
Base decisions on value-added gains
Use the TAP value-added statistical model
Set leveled criteria for school gains and
classroom gains (13%, 8%, 4%, Years
Growth, Negative gain)
Test every year
Use reliable and valid tests
Tie student level data to teacher each year
Models for the Teaching Profession:
Professional Growth
Traditional Model
TAP Model
Ongoing Applied
Professional Growth
Inservice/Course-based
Professional Development
•
Individual commitment, intermittent
activities
•
Goals and activities tied to personal
and financial interests of the
individual
•
Unconnected to evaluation
•
Schoolwide commitment, weekly,
site-based, teacher lead activities
•
Goals and activities tied to state
standards, local SIP & analysis of
student learning outcomes
•
Used to support and reinforce
evaluation growth goals
Current TAP Demonstration Sites
Arizona
6 schools
South Carolina
7 schools
Colorado
5 schools
Arkansas
9 schools
Indiana Archdiocese
4 schools
Active consideration:
Louisiana
Florida
Nevada
Ohio
(+3)
Unions accepting TAP
“Bottom up” not “top down”
Involves teachers at every step
Require >75% of faculty vote
TAP seen as fair
Does not replace traditional salary schedule
Any teacher who qualifies can get award
Implement slowly, gain confidence of teachers
TAP is a whole program
The Cost of TAP
Incremental costs = 6% of budget OR $400/student
No current teacher worse off
Salary supplements for Master & Mentor teachers
New teacher positions
New specialists hired
Senior teachers’ summer professional growth
Turnover savings not kept by school
Traditional salary schedule increases in place
Bonus pool must be > current certain raises
New Sources of Funds
Current district/school budgets
New state appropriations
Ballot initiatives
Private foundations
Federal Funds
Expected Final Outcome
Improved Student Achievement
Data
TAP Schools were matched to Controls based on:
• Achievement , school size, % students receiving
free lunch, configuration, and urban/rural
classification
4 TAP Schools
• 1,114 TAP Student 2000-2001
• 1,277 TAP students 2001-2002
8 Comparison Schools
• 2009 students 2000-2001
• 1,372 students 2001-2002
2000 Baseline Data TAP vs. Controls
TAP Schools
Control Schools
70
64
59
60
50
41
40
39
39
42
41
37
30
20
10
0
Total
Achievement
Reading
Achievement
Mathematics
Achievement
Percent Std.
Free Lunch
Analyses
• Value-added assessment
• Statistical model to measure
growth in student achievement
from pre-to-post-testing
• Each student must have 2 consecutive
years of test data from a reliable and
valid test
• Data needs to be linked to school, and
ideally, teachers each year
Interpreting Results
1. Gap Reduction
• each school is given an achievement target to
reach, and their goal is to reduce the gap between
their initial achievement and the target each year
2. Level of Certainty
• Statistics involves the study of probable
occurrences
• Whenever a statistical result is reported, so too is
the likelihood of achieving that result
Gap Reduction
Example
If my school’s pretest was the 50th percentile rank and
their posttest was the 55th percentile rank, they made a
5 percentile point gain.
Their gain to the target, however, would really be 14
percent, because the school has 35 percentile rank
points to make up (85-50), and dividing 5 by 35 is .14,
or 14 percent.
Level of Certainty
Level of Certainty
For TAP teachers/schools we want to be at least 70
percent certain that their classroom achieved a gain.
• Teacher #1 achieved a gain under that criteria
• Teacher #2 did not
We use statistics to calculate a certainty level
associated with the gain for each teacher and each
school.
Research Question # 1
1. Do TAP schools improve student
achievement on a yearly basis?
Research Conclusion 1
The average TAP school gain per year
was 11.5 percent, or 23 percent to
standard over two years.
Research Question # 2
2. Do TAP schools outperform
comparable schools on a yearly
basis?
Research Conclusion 2
Over the course of two years, TAP
schools out-gained their controls by
approximately 13 percent.
Research Question # 3
3. Do a greater proportion of
teachers in TAP schools achieve
student learning gains than
teachers in comparable schools?
% of Teachers Whose Classrooms
Achieved Gains
TAP Teachers
Comparison School Teachers
60
50
48
46
38
36
40
30
20
10
0
2001
2002
Research Conclusion 3
In both 2001 and 2002, 10 percent
more teachers in TAP schools
compared to controls achieved student
learning gains.
Research Question # 4
4. Does each individual TAP school
outperform its comparable
control schools?
Research Conclusion 4
• In 2001, three of the four TAP schools gained
significantly more in Reading, Language, and
Math than their control schools.
• In 2002, two of the four TAP schools gained
significantly more in Reading, Language and
Mathematics than their control schools.
Research Question # 5
5. Do TAP schools that strictly
adhere to implementing the five
TAP principles produce greater
student achievement gains than
TAP schools that implement the
principles with less rigor?
Implemenation Score
TAP School Implementation Scores
160
140
120
100
80
60
40
20
0
147
90
1
144
94
87
2
140
3
4
TAP School Num ber
5
6
Research Conclusion 5
• Two of the three TAP schools (3 and 5) with
rigorous implementation are attaining large
achievement gains.
• TAP School 3, the school with the highest
implementation score, is out-gaining its control
schools by 20 percent in the first year, and 31
percent in the second year, for a total of 51
percent in two years.
Research Conclusions
1. All TAP schools posted achievement gains in both years
they implemented the reform.
2. TAP schools significantly out-gained similar comparison
schools by a total of 13 percent over two years.
3. Significantly more TAP teachers’ classrooms’ achieved
student-learning gains than teachers in control schools.
4. Schools that rigorously implemented TAP produced
student achievement gains that were larger than TAP
schools that only moderately implemented the reform,
and much larger than control schools that did not
implement TAP.
A Final Note
Although, all of the TAP schools realized
student achievement gains in the first two
years of the reform, the magnitude of
those gains differed across schools (e.g.
from an 11 percent gain to target to a 51
percent gain to target over two years).
A Final Note
Studies of comprehensive school reform show that
the reform is more likely to be successful when:
1. It is supported by teachers
2. There is strong principal leadership
3. There is a stable and committed district
4. The schools receive on going assistance from
developers
5. The reform is implemented in smaller rather
than larger schools
Intermediate Outcomes
Teachers opt for new system vs. existing system
Changes in types of individuals applying
Number of applicants
Differences in characteristics of people hired
Changes in teacher retention rates
Changes in which teachers stay in classroom
Survival rates in the first five years
Changing nature of collective bargaining
Stakeholder perceptions of staff quality & professionalism
Teacher satisfaction data
The Stories We’ve Heard
New teachers have support unheard of in other
schools and in the past.
Veteran teachers are leaving comfortable school
environments to be at TAP schools (often lower
SES).
Teachers are collaborating and communicating
much more.
The Stories We’ve Heard
Seeing a great deal of flexibility and change
among teachers’ classroom practices.
Developing/studying the rubrics helps teachers
learn about and implement effective classroom
practices.
Immediate feedback from evaluations enables
teachers to enhance their performance.
The Stories We’ve Heard
Analyses of student test scores serves as guide
for individualized instruction.
Teachers are working harder than ever before,
but a lot better.
Professional growth at the school site on a
regular basis sends an affirming message to
teachers that their development is important.
TAP is the BEST Solution
Systemic program
Strong induction program
Excellent & relevant professional
growth
Fair & understandable assessment
system
TAP is the BEST Solution
Increased responsibility & leadership
Commensurate compensation
WITHOUT fully leaving classroom
Involves teachers at every step
Gains teacher support for making change
TAP is the BEST Solution
Anticipates high individual rewards
Pays well for teaching well
Teachers, in part, evaluated by what students learn.
Peer evaluation is fair and honest
www.mff.org/tap
[email protected]
Research Questions
1. Do TAP schools improve student achievement on a
yearly basis?
2. Do TAP schools outperform comparable schools on a
yearly basis?
3. Do a greater proportion of teachers in TAP schools
achieve student learning gains than teachers in
comparable schools?
4. Does each individual TAP school outperform its
comparable control schools?
5. Do TAP schools that strictly adhere to implementing
the five TAP principles produce greater student
achievement gains than TAP schools that implement
the principles with less rigor?
Gap Reduction
Demanding one uniform gain for every student
irrespective of where they start is unfair
• When you start at the 80th percentile it is more difficult to
make -- say 10 percentile rank gain-- than someone who
started at the 50th percentile
Demanding that a student makes up the distance from
where they start to a target each year takes into account
every student’s initial status, making the gain relative to a
starting point
We set the target at the 85th percentile rank (primarily for
statistical reasons relating to the test’s scale)
Remember
1. TAP’s value-added assessment is a
GAP REDUCTION MODEL
2. There is always probability
associated with achievement gains
Interpreting MFF
Productivity Profiles
1. Vertical axis is the certainty level
2. Horizontal axis is the percent gain to
standard (e.g. 85th percentile rank)
3. 70 percent certainty line
4. School growth curves
2000-2001
1.0
TAP School 3
Confidence
0.8
0.6
TAP School 1
TAP School 2
0.4
0.2
TAP School 4
0.0
0
10
20
30
% Gain
40
50
2001-2002
1.0
3
Confidence
0.8
1
0.6
0.4
6
5
0.2
4
2
0.0
0
10
20
% Gain
30
40
50
2000-2001
1.0
Confidence
0.8
0.6
TAP Schools
0.4
70% Confident
Control Schools
of 11.0% Gain
70% Confident
of 4.8% Gain
0.2
0.0
0
10
20
30
% Gain
40
50
2001-2002
1.0
Confidence
0.8
0.6
TAP Schools
70% Confident
0.4
of 12.0% Gain
0.2
Control Schools
70% Confident
of 5.2% Gain
0.0
0
10
20
30
% Gain
40
50
2000-2001
1.0
0.8
Confidence
Confidence
0.8
0.6
0.4
2000-2001
1.0
Control School
TAP School 1
70% Confident
70% Confident
of 9.5% Gain
of 12.7% Gain
TAP School 2
0.6
70% Confident
of 4.8% Gain
0.4
Control Schools
0.2
0.2
70% Confident
of 0% Gain
0.0
0.0
0
10
20
30
40
0
50
10
20
% Gain
2000-2001
1.0
40
50
2000-2001
1.0
0.8
0.6
Confidence
0.8
Confidence
30
% Gain
TAP School 3
70% Confident
of 25% Gain
0.4
TAP School 4
0.6
70% Confident
of 8.0% Gain
0.4
Control Schools
Control Schools
70% Confident
70% Confident
of 8.5% Gain
0.2
0.2
0.0
0.0
0
10
20
30
% Gain
40
50
of 6.0% Gain
0
10
20
30
% Gain
40
50
2001-2002
Implementation: 38
1.0
0.8
0.8
Control Schools
0.6
0.4
Control Schools
TAP School 1
70% Confident
70% Confident
of 11.5% Gain
of 13.0% Gain
Confidence
Confidence
2001-2002
Implementation: 37
1.0
0.6
70% Confident
of 1.6% Gain
TAP School 2
0.4
70% Confident
of 6.4% Gain
0.2
0.2
0.0
0.0
0
10
20
30
40
50
0
10
20
30
% Gain
2001-2002
Implementation: 58
1.0
50
2001-2002
Implementation: 39
1.0
0.8
0.8
TAP School 3
0.6
Confidence
Confidence
40
% Gain
70% Confident
of 31% Gain
Control Schools
0.4
70% Confident
0.6
Control Schools
0.4
70% Confident
TAP School 4
of 0.0% Gain
of 13% Gain
70% Confident
of 2.7% Gain
0.2
0.2
0.0
0.0
0
10
20
30
% Gain
40
50
0
10
20
30
% Gain
40
50
2001-2002
1.0
3
Confidence
0.8
1
0.6
0.4
6
5
0.2
4
2
0.0
0
10
20
% Gain
30
40
50