Transcript Principal Leadership Institute: Teacher Quality and

Teacher Evaluation in Newark:
Evaluator Training
August 21 or 22, 2013
1
The Irreplaceables explores retention through the experience of the nation’s
best teachers, who urban schools desperately need to keep.
Who Are the Irreplaceables?
The “Irreplaceables” are teachers so successful
that they are nearly impossible to replace.
Estimates of Irreplaceables percentage based on teachers with value-added or growth data; District A high performers: 21%; District B high performers: 20%;
District C high performers: 20%; District D high performers: 18%; Student impact estimates calculated following the methodology of Hahnel and Jackson
(2012). Source: District data from SY 2009-10 and SY 2010-11.
/2
When an Irreplaceable leaves a low-performing school, the school is almost
guaranteed to hire a less-effective replacement.
Likelihood of Replacing a High Performer with a Teacher of Similar Quality
When a great teacher leaves a school,
it can take 11 hires to find one teacher of comparable quality.
Estimates based on teachers with value-added or growth data; Low performing schools include schools in the lowest quintile of proficiency by school level;
Percentage of high-performing potential replacements in all schools - District A: 12%; District B: 17%; District C: 15%; District D: 15%; Low-performing schools District A: 12% ; District B: 10%; District C: 3%; District D: 9%. Source: District data from SY 2008-09 and SY 2009-10.
/3
Yet most schools retain Irreplaceables and low performers at fairly similar
rates.
School Retention Rates by Teacher Performance, 2009-10
Struggling teachers remain for too long: Most have more than 9 years
of experience and plan to stay for at least another 10 years.
Chart: School retention defined as teachers remaining at their school from one year to the next. Bottom statement: Median years of experience 9-10 years
across districts; Percentage planning to stay more than 10 years 48-62% across districts. Source: District data from SY 2008-09 through SY 2010-11
/4
Low performers rarely improve significantly. Even three years later, most
perform worse than the average first-year teacher.
Performance Comparison of New Teachers and
Low Performers over Three Years
40 percent of teachers with 7+ years of experience are less
effective than the average first-year teacher
Chart: Median percentile ranks by population scores in District C; Populations defined in 2007-08. Bottom statement: District A: 44% veterans less
effective; District C: 39% veterans less effective. Source: District data from SY 2008-09 through SY 2010-11
/5
There are simple, low-cost steps principals can take that double the time
Irreplaceables plan to remain at their schools.
Low-Cost Retention Strategies for Irreplaceables
Top teachers who experience two or more of these retention strategies plan
to keep teaching at their schools for nearly twice as long (2-6 more years).
Low-cost retention strategies defined as those that influence planned school retention of Irreplaceables. Source: District and survey data.
/6
However, Irreplaceables report receiving little recognition or attention at the
school level – often on par with the lowest performers.
Teachers Reporting Recognition at School
Principals used 7 of 8 top retention strategies at similar rates
for high and low performers.
Source: District B data and survey data. Trends confirmed across districts.
/7
The Framework for Effective Teaching is at
the core of the evaluation system
8
Our goal today: Ensure all are ready to
implement the teacher evaluation system as a
tool for differentiated management
Today, we will:
 Reflect on SY12-13 and discuss what is changing
for SY13-14
 View instruction and norm on the framework
 Conduct a deep dive into the student goal setting
process
 Discuss details of evaluation requirements and
prepare to implement as the school year begins
9
REFLECTIONS ON 2012-13
TEACHER EVALUATION
PROCESS
10
Activity: Think-Pair-Share
Reflecting on implementation of teacher evaluation
this past year, write down:
• 1 to 2 successes from this year
• 1 to 2 challenges from this year
Turn to your neighbor and share what you wrote down
11
Quick Quiz
What percentage of our teachers received an annual
evaluation by the end of the year?
12
Completion rates for observations, midyears, and annuals evaluations were strong
93%
76%
At least 1
At least 1
Met
observation observation observation
(4/2012)
(7/1/13) requirements
(7/1/13)
1
3
94%
91%
Confidential - Do Not Distribute
72%
Mid-year
Review
(2/22/13)
77%
Annual
Annual
Evaluations Evaluations
(7/2012)
(8/13/2013)
Completion rates for observations and evaluations
are consistently high across networks
Annual Evaluations
Observations
93% 92%
99% 98%
97%
90%
94% 93%
95% 92%
65%
56%
Brad
(n = 961)
1
4
Mitch
(n = 580)
Peter
(n = 535)
Roger
(n = 478)
Tiffany
(n = 484)
Central
(n = 108)
Notes: Observation completion rates measured as percent of teachers with required number of
observations;
Central includes Office of Special Education, Early Childhood, Title I Office, Master Teachers, and
anyone else evaluated by central office staff except staff on long term leave
Confidential - Do Not Distribute
Data current as of 7/1/2013
Completion rates for annual evaluations and
observations less consistent across schools
 52 schools have annual evaluation or completion
rates above 95%
Number of schools and departments
Completion rate
Annual Evaluations
Observations
100%
35
28
95 – 99%
17
22
90 – 94%
11
9
80-89%
8
6
Less than 80%
6*
12*
Note: Only 1 school has an annual evaluation completion rate below 80% and only 6 schools have
observation completion rates below 80%
1
5
Data current as of 7/1/2013
Quick Quiz
What percentage of teachers in the EWPS pool did
not receive a rating in 2012-13?
16
Teachers in the EWPS pool receive lower ratings
compared to the district distribution
• 29% of teachers in the 2012-13 EWPS pool (and 21% of teachers
projected 2013-14 EWPS) did not receive annual evaluations
2%
48%
5%
11%
50%
69%
30%
20%
12-13 EWPS Pool
(n=153)
1
7
32%
13%
13-14 EWPS Pool
(n=339, projected)
16%
4%
District
Highly Effective
Effective
Partially Effective
Ineffective
Teachers on long-term leave have lower ratings
compared to other teachers
•
11% of teachers on long-term leave have completed annual evaluations
•
23% have the required number of observations and 26% have been
observed at least once
4%
7%
10%
16%
33%
45%
68%
Highly Effective
69%
50%
15%
Effective
Partially Effective
Ineffective
30%
21%
17%
11%
4%
Long term leave
(n = 57)
Observations
1
8
District
(n = 4832)
Long term leave
(n = 57)
District
(n = 2969)
Annual Evaluations
Quick Quiz
What percentage of observations had a partially
effective rating?
What percentage of annual reviews had a partially
effective rating?
19
Observations and Annual Ratings
Distribution, 2012-13
Observations
Annual Evaluations
Highly Effective
2
0
Effective
Partially Effective
Ineffective
Quick Quiz
What was the percentage of teachers rated highly
effective last year (2011-12)?
What was the percentage of teachers rated highly
effective this year (2012-13)?
21
Compared to last year, % HE annuals decreased
and % PE annuals increased
2011-2012
2012-2013
69% 69%
17%
16%
11%
Highly Effective
2
2
10%
Effective
Partially Effective
4% 4%
Ineffective
Annual ratings vary across networks
9%
72%
13%
10%
62%
63%
14%
17%
21%
75%
75%
20%
8%
70%
23%
14%
1%
9%
7%
6%
4%
3%
3%
2%
Brad
Mitch
Peter
Roger
Tiffany Central
(n = 890) (n = 574) (n = 521) (n = 447) (n = 461) (n = 70)
2
3
Highly Effective
Effective
Partially Effective
Ineffective
Ratings vary even more across schools:
Some schools still rated few teachers as partially
effective or ineffective
Observations:
> 27 schools had no ineffective observation ratings.
> 3 schools had no partially effective or ineffective observation
ratings.
> 34 schools assigned more than 80% of observation ratings
in the top two categories.
Annual Evaluations:
> 28 schools had no ineffective annual ratings.
> 4 schools had no partially effective or ineffective annual
ratings.
> 37 schools assigned more than 80% of evaluation ratings in
the top two categories.
2
4
Quick Quiz
What competencies receive the greatest proportion
of partially effective and ineffective ratings?
25
On Annuals, Competencies 2 and 4 have the
highest share of PE and IE ratings
19%
10%
20%
67%
66%
11%
21%
70%
65%
66%
20%
16%
13%
13%
11%
3%
3%
2%
2%
2%
CompetencyCompetencyCompetencyCompetencyCompetency
1
2
3
4
5
2
6
Highly Effective
Effective
Partially Effective
Ineffective
Ratings distribution is fairly consistent across
different evaluator types (slightly higher for DCs)
6%
8%
7%
13%
10%
10%
Highly Effective
Effective
19%
3%
17%
5%
17%
3%
14%
3%
Department Chair
73%
Vice Principal
69%
21%
4%
Observations
(n = 4832)
2
7
65%
Principal
Principal
5%
71%
Department Chair
24%
67%
Vice Principal
65%
Annual Evaluations
(n = 2969)
Partially Effective
Ineffective
Ratings are slightly higher both from observations
and mid-years to annual evaluations
Annual Evaluation
Mid-Year
Highly Effective Effective
2
8
Partially
Effective
Ineffective
Highly Effective
71%
29%
0%
0%
Effective
3%
90%
8%
0%
Partially Effective
0%
21%
71%
8%
Ineffective
0%
3%
36%
62%
Note: Percentages calculated as percent of row total; red numbers indicate inflation, blue
numbers indicate deflation
TRANSITION TO
NETWORKS
29
The five competencies set the standard for
teacher practice in NPS
Students
sustain
focus on a
specific
objective
that moves
them toward
mastery.
Instructional
strategies
challenge all
students and
provide
multiple
pathways to
mastery.
A learningfocused
environment
of shared
high
expectations
promotes
mastery.
Students
show
evidence of,
and teacher
monitors,
growth.
The teacher
demonstrates
commitment
to excellence
and to the
professional
growth of
his/her
school and
peers.
30
Framework language changes to increase
clarity and user-friendliness
12-13 version
13-14 version
• Some indicators used the
phrase “and/or”, particularly in
PE or I ratings
• Simplified to use either “and”
or “or” to make rating these
indicators easier and more
consistent
• The distinction between
performance levels in some
indicators wasn’t very clear
• Indicator 3e did not include
reference to the teacher
modeling high expectations
• Language was changed to
make distinctions clearer
• Indicator 3e revised to include
the expectation that the
teacher is a model of high
expectations
31
Framework language changes to increase
clarity and user-friendliness
12-13 version
13-14 version
• Highly Effective in indicator 3b
asked for students to demand
persistence of each other.
• This phrase in Highly Effective
was removed
• Competency 4 over-time
indicators were revised to
better align with the new
student learning goals in the
IPDP.
32
Most remaining content of the Framework did
not change. But does require additional
training to ensure consistency across
evaluators.
• All/nearly all, most, some, few
• Calling out a teachers’ physical classroom space
• Explicitly listing professional standards
• Making an explicit description of attendance metrics in
Competency 5
33
Let’s reflect on the changes to the framework
and evaluation system this year
• Consider each competency and the changes (both in
language and evidence collection) between 2012 and 2013
• On your handout
• Note what these changes signify about the intent of the
competency
• Note what these changes signify about the implementation of
the competency
• Share your thoughts with your table mates
• Be prepared to share your group’s thoughts with the larger group
about one competency
34
This year, there are many ways to collect
evidence to evaluate and support teachers
Rigor &
Inclusiveness
Culture of
Achievement
Commitment
to Personal &
Collective
Excellence
What Can Be Seen in
Quantitative Data
What Can Be
Seen in
Artifacts
What Can Be
Observed
Lesson Design
& Focus
Student
Progress
Towards
Mastery
35
BREAK
36
VIEWING AND RATING
INSTRUCTION
37
Viewing instruction:
Using an observation template or your own method, transcribe
what happens in the following teaching clip.
After you view the video, categorize your evidence and assign
ratings on the NPS framework. Then, note your ratings on the flip
charts at the front of the room.
3
8
Video linked here
Let’s review our ratings and discuss
Where are we aligned in ratings?
Where are there outliers? What evidence can you share for
these outlier ratings?
What do we need to do to ensure we and our school leaders
are aligned?
What feedback would you give this teacher? What is the
highest-leverage thing she could do to improve?
3
9
LUNCH
40
ASSESSING STUDENT
LEARNING IN TEACHER
EVALUATION
41
Student learning is already a core part of our
framework
42
Main Takeaways
1. Competency 4 defines how we approach the use the assessment of
student learning in teacher evaluation.
2. The Individual Professional Development Plan (IPDP) is the way we
will document how a teacher addresses Competency 4.
3. The IPDP requires setting student learning goals and teacher goals
related to those student learning goals.
4. The goal setting process in the IPDP documents what effective
teachers already do.
5. The IPDP should serve as a communication tool for teachers and their
administrators on goals.
6. The specificity of student goals should be determined by the strength
of available tools and resources (e.g., curriculum, assessments).
7. The Corrective Action Plan (CAP) has the same process as the IPDP
for setting student learning goals and a more rigorous process for
setting and tracking teacher goals.
8. All IPDPs and CAPs will be entered into an online system.
43
This year’s Individual Professional Development
Plan (IPDP):
• Provides a tool for goal setting for teachers and administrators
around both student learning goals and the teacher’s own
development goals
• Leverages over-time indicators and Common Core planning in
the goals teachers set for students and themselves
• Is a tool for teachers and administrators to communicate about
goals and growth areas throughout the year
44
The IPDP content includes:
Student Learning Plan:
• Areas of Focus
• Student End Points
• Student Starting Points
• Instructional Tools and Resources
Professional Growth Plan:
• Growth Areas
• Action Steps for Teachers
Let’s review a sample IPDP form together
45
The IPDP form is created in the beginning of the year
and examined at conferences throughout the year
Goal-Setting
Conference
Observations and
Conferences
Annual
Conference
Mid-Year
Conference
Observations and
Conferences
46
The final assessment of progress towards goals
happens at the annual conference
Specifically, evaluators reflect their assessment of whether or not
the teacher met his or her learning and professional development
goals in the over-time indicators in Competency 4:
• 4d. Using Data: Teacher tracks assessment data to understand each
student’s progress toward mastery and uses results to guide planning
and instruction
• 4e. Understanding of Growth: Teacher can articulate specifically (and
with evidence) whether or not each student has internalized gradelevel standards and, if not, what s/he still needs to learn.
• 4f. Progress Toward Goals: Data reflect that students are mastering
the objectives of the focus areas, leading toward mastery of gradelevel standards.
47
Teachers rated PE or I at the end of SY12-13 will have a
Corrective Action Plan (CAP), which replaces their IPDP
How is the CAP different from the IPDP?
CAP is more robust to ensure that:
- Struggling teachers are getting the support they need and
-The district is collecting sufficient evidence to support tenure charges if necessary
The CAP includes 2 extra steps in setting professional development
goals:
1) Establishing metrics & processes to monitor progress
2) Articulating the steps administrators will take to support teachers’ development
There are several follow-up steps required by state law
Additional observations
Mid-year conference
Observations by multiple observers
48
TRANSITION
49
IMPLEMENTING TEACHER
EVALUATION: 2013-14
50
Goal-Setting
Conference
Observations and
Conferences
Annual
Conference
Mid-Year
Conference
Observations and
Conferences
51
GOAL-SETTING CONFERENCE
52
Goal-Setting Conference at Beginning of Year
GoalSetting
Conference
• Set student learning and professional development
goals
• Finalize IPDPs or CAPs for all teachers
By 9/15 for
teachers on a
CAP; by 9/30
for all others
53
Evaluators’ first focus should be on creating a
CAP for teachers rated PE or I last year
In addition to the requirements mentioned earlier,
a strong CAP is:
Very explicit in indicators and competencies that are areas
of focus for the teacher
Very clear on the plan for development, including the role
of both the teacher and the administrator
Is wholly co-developed by evaluator and teacher.
CAPs must be on file for all relevant teachers by September 15.
54
Some evaluators submitted CAP as part
of the Annual in the Zoho system
 25% of PE teachers and 50% of IE teachers have
CAP already
Did not receive a
CAP
Received a CAP
367
121
54
54
Partially Effective
(n = 488)
Ineffective
(n = 108)
These evaluators and teachers should re-visit their CAP
when they input into the new BloomBoard system.
5
5
CLASSROOM OBSERVATIONS
56
Observations requirements have changed but
reinforce best practices
Factor
SY12-13
SY13-14
Length of
Observations
Formal: Full class
PPO: No guidance
Long: 40 minutes
Short: 20 minutes
# of Observations
3 formal for non-tenured
1 formal for tenured
Undefined PPOs
3 for non-tenured (length depends
on years of experience)
3 for tenured teachers (all short)
# of Observers
One observer required per teacher
Each non-tenured teacher and
teacher with CAP must have at
least two observers
Timing
Just happen at some point in the
school year
At least one observation must
occur in each semester
Pre-Obs
Conferences
Recommended for an announced
observation
One observation each year must
be announced with a pre-conf
within 7 days before observation
Post-Obs
Conferences
Formal post-observation required
after formal observation within 10
days
Post-observation required for all
observations within 15 days (for
tenured Ts not on CAP, can be
“informal”)
57
Quick Quiz
What percentage of teachers were observed by
more than one evaluator?
58
Consideration: The use of multiple observers
 13-14SY Requirement: All non-tenured teachers and a teacher with a
CAP must be observed by more than evaluator
 Note: A co-observation counts toward this requirement
 However, only 10% of teachers were observed by more than one
evaluator in 2012-13
90%
1
9%
1%
2
Number of unique observers in 2012-13
(excludes Peer Validators)
3
 Bottom line – This is a shift from evaluator practice last
year.
5
9
Quick Quiz
How many observations were announced this year?
60
Consideration: Pre-observation conferences
 13-14SY Requirement: Each year, every teacher must have at
least one announced visit with a pre-observation conference
within 7 days before the observation
 In 12-13SY:
 70% of observations were announced and
 77% of those announced observations included a pre-observation
conference
 Bottom Line – Not a big shift from previous evaluator
practice.
6
1
Consideration: Post-conference timing
 13-14SY Requirement: Post-Conferences must occur within 15 days of
any observation
 Note: But for tenured teachers not on a CAP, these can be “informal” post-
conference through the BloomBoard system
 In 12-13SY, 88% of post-obs conference occurred within 15 days
Percent of observations that take
place within a given timeframe
72%
13%
Less than 10
days
5%
10 to 15 days 15 to 20 days More than 20
days
 Bottom line – Not a big shift from last past practice.
6
2
10%
Consideration: Timing of observations
 13-14SY Requirement: At least one observation must occur in each
Number of observations
semester
 In 12-13SY, the majority of observations took place in the first semester
100
90
80
70
60
50
40
30
20
10
0
6
3
Start of the 3rd marking period
SCORING
64
Scoring remains the same in SY2013-2014
4 points
4 points
4 points
4 points
1, 0, -2 or -6
Based on a preponderance of evidence, evaluators:
• Assign a rating of Highly Effective, Effective, Partially Effective or
Ineffective on Competencies 1-4
• Assign a rating of Exceeds Expectations, Meets Expectations, Slightly
Below Expectations, or Significantly Below Expectations on
Competency 5
The evaluation rating is determined based on the teacher’s total
score on all 5 competencies out of 17 points
65
However, scoring will be slightly different for
short vs. long observations
Short Observations
Long Observations
Evaluators will report:
• Ratings at the indicator and
competency level for which
they have sufficient evidence
• An overall rating for the
observation based on
preponderance of evidence No
formula to determine full rating
Evaluators will report:
• Ratings at the indicator level
for which they have sufficient
evidence
• Ratings required for each
competency based on
preponderance of evidence
• Rating is determined by adding
up competency ratings
66
MID-YEAR REVIEWS
67
The Mid-Year Conference is a chance to review
evidence and assess progress towards goals
GoalSetting
Conference
• Set student learning and professional development
goals
• Finalize forms, including CAPs for applicable teachers
Mid-Year
Conference
• Review evidence of all indicators and assess
progress towards goals
• Adjust approach to goals to move toward meeting
goals
• Provide a rating based on evidence collected so far
By 9/15 for
teachers on a
CAP; by 9/30
for all others
By 2/15 for all
teachers
68
ANNUAL REVIEWS
69
At the Annual Conference, evaluators review
evidence and assign ratings to Competencies
GoalSetting
Conference
• Set student learning and professional development
goals
• Finalize forms, including CAPs for applicable teachers
Mid-Year
Conference
• Review evidence of all indicators and assess
progress towards goals
• Adjust approach to goals to move toward meeting
goals
• Provide a rating based on evidence collected so far
Annual
Conference
and
Evaluation
• Assess all evidence, including whether or not student
learning goals were met
• Rate each indicator and competency based on all
evidence to determine final evaluation score
By 9/15 for
teachers on a
CAP; by 9/30
for all others
By 2/15 for all
teachers
By 4/15 for
non-tenured;
5/15 for
tenured
teachers on
CAPs; and
6/15 for all
others
70
Quick Quiz
How many annual evaluations were completed on
time?
71
Consideration: Deadlines for Annuals
 Deadlines will stay the same (with the new addition of the May 15th
deadline for tenured teachers on CAPS)
 74% of annual evaluations are completed on time (for April 15th and
June 15th deadlines)
 100% on time: American History Program, Fourteenth Ave., Ridge St.,
Roberto Clemente, Samuel L. Berliner, Science High
 However, evaluations completed on time received higher ratings:
70%
On time
67%
Not on time
21%
12%
15%
8%
Highly Effective
7
2
Effective
Partially Effective
4%
3%
Ineffective
Summary of Deadlines
Item
Deadline
CAPs finalized with student learning goals at beginning
of year conference for all applicable teachers
September 15, 2013
IPDPs finalized with student learning goals at beginning
of year conferences for all other teachers
September 30, 2013
Observations
At least one in the first
semester
Mid-Year Reviews complete for all teachers
February 15, 2014
Observations
At least one in the
second semester
Annual Reviews complete for non-tenured teachers
(whether or not on CAPs)
April 15, 2014
Annual Reviews complete for tenured teachers on CAPs May 15, 2014
Annual Reviews complete for all other tenured teachers
June 15, 2014
73
Supports for implementing the evaluation
system
BloomBoard: Sophisticated
data collection
Early September
Teacher Evaluation Guidebook:
Clear policies and procedures
Early September
Successful
Implementation
Instructional Resource Center
and BloomBoard: Easy access
to supplemental resources
Now
Peer Validation: Additional
support for evaluators and
teachers
October
74
School Improvement Panels (SIP)
Membership:
 Each SIP must include the school’s principal, vice principal, and a
teacher.
 SIPs may have more members, but at least 1/3 of the SIP’s members
must be teachers.
The deadline for finalizing SIP membership is August 31, 2013.
Function:
 Oversee the mentoring and evaluations of teachers, conducting a midyear evaluation of a teacher who may receive an IE or PE rating at the
Annual
 Identify professional development opportunities for instructional staff
members that are tailored to meet the unique needs of the students and
staff of the school.
 Request Peer Validators
7
5
In summary, while there are some changes, the
core elements of evaluation remain the same,
What stays the same
What changes
• Framework at the core of every
teacher’s observation
•
A beginning-of-year conference
to start the year focused on
growth and development
•
Better, more detailed tools with
some new tools for structured
goal setting of both adult and
student learning goals
•
Requirements for the # of
observations captured in the
online system
•
A more sophisticated data
system providing better tools
and resources
• Multiple observations for each
teacher allow ample
opportunities to collect
evidence
• Mid-year and end of year
review conferences to check in
on goals
• Online data entry for real-time
reporting and monitoring
76
REFLECTION AND WRAP UP
77
Who are the Irreplaceable teachers in your
building?
• How do you know?
• What strategies could you employ to support those
teachers?
• How do you help your other teachers become
Irreplaceable?
• How can implementing the Framework for Effective
Teaching support you in identifying, developing and
retaining Irreplaceable teachers?
78
Next Steps: Your critical role in ensuring your teachers
understand the evaluation system this year.
Date
Activity
First week of September
Teacher Evaluation Guidebook distributed to
all teachers and staff
Early September
Online trainings on using the BloomBoard
data system
By mid-September
Present training information to teachers in a
one-hour meeting (potentially with A Supt,
SATQ)
79
SURVEY
Please complete the exit survey and drop it on the
table in the back of the room before you leave.
80