Transcript Multiple Measures of Teacher and Principal Effectiveness

Teacher Effectiveness
August 29 & 30, 2013
Facilitators:
Betsy A. Baker, Ed.D.
Susan J. Bickford, Ph.D.
7/17/2015
pbevan
1
Project Development - Goal

To develop a teacher effectiveness model that:

will reform the way we evaluate teachers

and identify critical components of teacher training
and professional growth
7/17/2015
pbevan
2
Project Development - Background

$800,000 Gates Foundation grant to facilitate the
development of statewide policy, tools and processes to
evaluate teachers and principals in which student
achievement is a significant factor affecting
performance ratings

PDE is closely following the work of the Pittsburgh
Public Schools – PPS recipients of $40 million Gates
Foundation grant that is more comprehensive in scope
but similar in redesigning evaluation policy, tools and
processes
7/17/2015
pbevan
3
2010-2011 Pilot I Sites
Allentown School District
Cornell School District
Mohawk School District
IU5 – Northwest Tri-County




2011-2012 Phase II Sites
102 Pilot II LEA Sites
•
•
•
•
Moshannon Valley Area School District
West Branch Area School District
State College Area School District
Bellefonte Area School District
2012-2013 Phase III Sites
•
•
•
•
•
•
•
•
•
Bald Eagle
Curwensville Area
Glendale
Harmony Area
Moshannon Valley
Penns Valley Area
Central Intermediate Unit # 10
Keystone Central School District
Philipsburg-Osceola School District
7/17/2015
pbevan
4
Teachers Without Eligible PVAAS Score
Observation/Evidence
Building Level Data
Danielson Framework Domains
1. Planning and Preparation
2. Classroom Environment
3. Instruction
Indicators of Academic Achievement
Indicators of Closing the Achievement Gap, All
Students
Indicators of Closing the Achievement Gap, Subgroups
Academic Growth PVAAS
Effective 2013-2014
Effective 2013-2014 SY
Building Level
Data, 15%
Observation/
Evidence,
50%
Elective Data/SLOs
Elective Data,
35%
Optional 2013-2014 SY
Effective 2014-2015 SY
District Designed Measures and Examinations
Nationally Recognized Standardized Tests
Industry Certification Examinations
Student Projects Pursuant to Local Requirements
Student Portfolios Pursuant to Local
Teachers with Eligible PVAAS Score
Teacher Effectiveness System in Act 82 of 2012
School Building Data
Effective 2013-2014 SY
Observation/Evidence
Effective 2013-2014 SY
Danielson Framework Domains
1. Planning and Preparation
2. Classroom Environment
3. Instruction
4. Professional Responsibilities
Indicators of Academic Achievement
Indicators of Closing the Achievement Gap, All Students
Indicators of Closing the Achievement Gap, Subgroups
Academic Growth PVAAS
Other Academic Indicators
Credit for Advanced Achievement
School Building
Data, 15%
Teacher Specific
Data, 15%
Observation/
Evidence,
50%
Teacher Specific Data
PVAAS / Growth 3 Year Rolling Average
1. 2013-2014 SY
2. 2014-2015 SY
3. 2015-2016 SY
Elective Data/SLOs
Elective
Data, 20%
Optional 2013-2014 SY
Effective 2014-2015 SY
District Designed Measures and Examinations
Nationally Recognized Standardized Tests
Industry Certification Examinations
Student Projects Pursuant to Local Requirements
Student Portfolios Pursuant to Local Requirements
Non Teaching Professional Employee
Effectiveness System in Act 82 of 2012
Effective 2014-2015 SY
Observation/Evidence
Danielson Framework Domains
1. Planning and Preparation
2. Educational Environment
3. Delivery of Service
4. Professional Development
Student
Performance
20%
Student Performance of All Students in the
School Building in which the Nonteaching
Professional Employee is Employed
District Designed Measures and Examinations
Nationally Recognized Standardized Tests
Industry Certification Examinations
Student Projects Pursuant to Local Requirements
Student Portfolios Pursuant to Local Requirements
Observation/
Evidence
80%
7
Principal Effectiveness System in Act 82 of 2012
Effective 2014-2015 SY
Observation/ Evidence
Domains
1. Strategic/Cultural Leadership
2. Systems Leadership
3. Leadership for Learning
4. Professional and Community
Leadership
Building Level Data
Indicators of Academic Achievement
Indicators of Closing the Achievement Gap, All Students
Indicators of Closing the Achievement Gap, Subgroups
Academic Growth PVAAS
Other Academic Indicators
Credit for Advanced Achievement
Building
Level Data
15%
Observation
/ Evidence
50%
Correlation
between
Teacher PVAAS
scores and
Teacher
Danielson
rating
15%
Elective
Data/ SLOs
20%
Correlation Data Based on
Teacher Level Measures
PVAAS
Elective Data/SLOs
District Designed Measures and Examinations
Nationally Recognized Standardized Tests
Industry Certification Examinations
Student Projects Pursuant to Local Requirements
Student Portfolios Pursuant to Local Requirements
Pre-Observation – Domain 1 and 4
Concluding about the Lesson
2 days before: Teacher provides evidence using Lesson Plan Form
The goal is for the teacher to be Distinguished in Domain I, Planning and Prep.
Teacher and Evaluator discuss evidence provided; Evaluator Collects additional Evidence through
Questioning
Where
your group
mark
During did
the Observation
– Domains
1, 2the
and 3lesson at
or above proficient?
Evaluator arrives early – Walks the Walls
Evidence Collected during the lesson: Avoid Opinions
Evaluator provides Teacher with Evidence Collected during the Observation
Wherefordid
your group
mark 1,below
Preparing
Post-Observation
– Domains
2, 3 and 4
proficient?
Evaluator provides Teacher with Evidence Collected during the
Observation
Teacher self-assesses using highlighter and rubric –and gives to
evaluator
Evaluator assesses and marks all agreed upon – leaves areas of
concern blank to discuss
(No need to discuss the areas of agreement; plan to talk about
the areas of concern only)
Post-Teaching Collaborative Assessment – Domains: 1, 2, 3 and 4
Teacher and Evaluator discuss agreed upon items
Evaluator invites teacher to discuss areas of disagreement
7/17/2015
Teacher develops self-assessment summary
pbevan
9
7/17/2015
pbevan
10
TEACHER ELECTIVE DATA
STUDENT LEARNING OBJECTIVES (SLO)
SLO is a process to document a measure of Educator
Effectiveness based on student achievement of content
standards. SLOs are a part of PA’s multiple-measure,
comprehensive system of Educator Effectiveness
authorized by Act 82.
COMPONENTS:
1. Classroom Context
2. SLO Goal
3. Performance Indicators (PI)
4. Performance Measures (PM)
5. Teacher Expectations
7/17/2015
pbevan
11
Next Steps – Value-Added Models – (15%)

Mathematical Policy Research Proposal
◦ Develop value-added models (VAMs) for estimating the
contribution of individual teachers and principals to growth in
student achievement.
◦ Provide estimates based on VAMs for teacher and principal
evaluation.
◦ Assess the strength of relationships between VAM-based and
observation-based measures of performance, to inform decisions
about which observation-based measures should be included and
how to evaluate teachers in non-tested grades and subjects.
◦ Develop and calculate summary performance indicators across
value-added measures and between value-added and other
measures.
◦ Synthesize the findings in annual reports that document the
model’s continual development over time.
7/17/2015
pbevan
12
Teacher Specific Data (15%) (PVAAS)

Teacher value-added reports for individual teachers by grade/subject/year
using a robust statistical report.

Teacher diagnostic reports for insight on effectiveness with students by
achievement level and subgroup.

Administrator summary reports for authorized users in a particular school
or district.

Drill down capacity to individual student-level projections based on
classroom rosters.

Teacher-level value-added reports require student-teacher linkages, which
capture the instructor(s) responsible for a student’s learning in the tested
grade/subject. It is critical that this linkage system provide accurate
information that is validated by individual teachers.

Web-based teacher reporting for PSSA grades 4-8

Web-based teacher reporting for PSSA grades 4-8 and Keystone Exams
7/17/2015
pbevan
13
BUILDING LEVEL DATA
SCHOOL PERFORMANCE PROFILE
7/17/2015
pbevan
14
OBSERVATION AND PRACTICE
Teacher Effectiveness Framework (50%)

Principal/Supervisor classroom observations,
including evidence that demonstrates behaviors
associated with improving student achievement:

Domains
◦
◦
◦
◦
Planning and preparation
Classroom environment
Instruction
Professional responsibilities
7/17/2015
pbevan
15
Teacher Effectiveness Framework

4 Domains
◦ 22 Components

4 Possible Ratings for Each Component
◦
◦
◦
◦
Failing
Needs Improvement
Proficient
Distinguished
7/17/2015
pbevan
16
Formative Assessment
How much do we really know about the
Danielson Model and our own district
leadership?
Agree or Disagree
7/17/2015
pbevan
17
Introduction to the Domains
OBJECTIVES: Participants will learn…
 How their thinking about good teaching
compares to the framework we will use
 That the Framework represents good
common sense and much that we already
know about teaching
 The form and content of
Domains 1, 2, 3 and 4.
7/17/2015
pbevan
18
Having an Impact

If we want to impact student achievement
and growth
◦ Then we must impact teaching and learning
◦ And of course, we know what good teaching
is…or do we?
7/17/2015
pbevan
19
Worksheet #1- Pg.2
Wisdom of Practice:
Comparing our thinking about
what is good teaching
7/17/2015
pbevan
20
Wisdom of Practice
What are the qualities of teaching
most tightly tied to student
learning?
Participant Materials
Worksheet #1, Page 2
7/17/2015
pbevan
21
The Domains
1.
Planning and Preparation
2.
The Classroom Environment
3.
Instruction
4.
Professional Responsibilities
(Middle School Video, Danielson Intro)
7/17/2015
pbevan
22
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Setting Instructional Outcomes
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Designing Student Assessments
Domain 2: The Classroom Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Participating in a Professional
Community
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating with Students
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
7/17/2015
pbevan
23
Worksheet #2-pg 3
Identifying the Domains:
Extending our Learning of the
Domains
7/17/2015
pbevan
24
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
a. Demonstrating Knowledge of Content
and Pedagogy
b. Demonstrating Knowledge of Students
c. Setting Instructional Outcomes
d. Demonstrating Knowledge of Resources
e. Designing Coherent Instruction
f. Designing Student Assessments
Domain 4: Professional
Responsibilities
Domain 3: Instruction
a. Reflecting on Teaching
b. Maintaining Accurate Records
c. Communicating with Families
d. Participating in a Professional Community
e. Growing and Developing Professionally
f. Showing Professionalism
7/17/2015
Domain 2: The Classroom Environment
a. Creating an Environment of Respect
and Rapport
b. Establishing a Culture for Learning
c. Managing Classroom Procedures
d. Managing Student Behavior
e. Organizing Physical Space
a. Communicating with Students
b. Using Questioning and Discussion
Techniques
c. Engaging Students in Learning
d. Using Assessment in Instruction
e. Demonstrating Flexibility and
Responsiveness
pbevan 7/17/2015
pbevan
25
25
Features of
A Framework for Teaching
Generic: applies to all grade levels, content
areas
 Not a checklist

◦ Evidence based/reflective
Not prescriptive: tells the “what” of
teaching, not “how”
 Comprehensive: not just what we can see
 Inclusive: Novice to Master teacher

7/17/2015
pbevan 7/17/2015
pbevan
26
26
Domain 2:
Classroom Environment
2a:
Creating an Environment of Respect and Rapport
2b:
Establishing a Culture for Learning
2c:
Managing Classroom Procedures
2d:
Managing Student Behavior
2e:
Organizing Physical Space
7/17/2015
pbevan
27
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Setting Instructional Outcomes
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Designing Student Assessments
Domain 2: The Classroom
Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Participating in a Professional
Community
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating with Students
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
pbevan 7/17/2015
pbevan
28
28
Domain 3:
Instruction
3a:
Communicating with Students
3b:
Using Questioning and Discussion Techniques
3c:
Engaging Students in Learning
3d:
Using Assessment During Instruction
3e:
Demonstrating Flexibility and Responsiveness
7/17/2015
pbevan
29
Worksheet #3a- Pg. 4:
Exploring Domain 3
Create a specific example of your
assigned component
For Example, in Component 3a:
Ms. T says to Joey, “You dummy!”


7/17/2015
Do not restate the rubric
pbevan 7/17/2015
pbevan
30
30
Worksheet #3b- Pg. 5
Generalizing about Levels of
Performance

What do all the examples of Failing have in
common?

What do all the Needs Improvement
examples have in common?

Proficient?

Distinguished?
7/17/2015
pbevan 7/17/2015
pbevan
31
31
Conclusions: Levels of Performance
7/17/2015

Failing: Potential for harm

Needs Improvement: Inconsistent, novice

Proficient: Consistent, competent

Distinguished: Unusually excellent, no one
“lives” here permanently in all components
pbevan 7/17/2015
pbevan
32
32
Components of Domain 3:
Instruction
7/17/2015

3a: Communicating with Student

3b: Using Questioning and Discussion Techniques

3c: Engaging Students in Learning

3d: Using Assessment in Instruction

3e: Demonstrating Flexibility and Responsiveness
pbevan 7/17/2015
pbevan
33
33
Let’s Take a Break – 15 minutes
7/17/2015
pbevan
34
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Setting Instructional Outcomes
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Designing Student Assessments
Domain 2: The Classroom
Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Participating in a Professional
Community
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating with Students
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
pbevan 7/17/2015
pbevan
35
35
Domain 1: Planning and Preparation
1a: Demonstrating Knowledge of Content and Pedagogy
1b: Demonstrating Knowledge of Students
1c: Setting Instructional Outcomes
1d: Demonstrating Knowledge of Resources
1e: Designing Coherent Instruction
1f: Designing Student Assessments
pbevan 7/17/2015
pbevan
36
36
Component 1a: Demo. Knowledge of
Content/Pedagogy
1.
Teacher wrote a scholarly article
2.
Lesson plans/structure/content/relevance
3.
Teacher explanation of probable Students’ misconceptions
4.
Teacher’s answers to student questions during class
5.
Teacher presented a workshop to faculty
6.
Teacher explains the structure of discipline prior to lesson
7.
Teacher tells observer how this lesson fits into the larger unit
8.
Teacher adjusts the lesson midstream based on Students’ misconceptions
9.
Teacher poses different levels of content questions during the lesson
10.
Teacher states how this lesson connects to content standards
7/17/2015
pbevan
37
Component 1a: Demo. Knowledge of
Content/Pedagogy
1.
Teacher wrote a scholarly article
2.
Lesson plans/structure/content/relevance
3.
Teacher explanation of probable Students’ misconceptions
4.
Teacher’s answers to student questions during class
5.
Teacher presented a workshop to faculty
6.
Teacher explains the structure of discipline prior to lesson
7.
Teacher tells observer how this lesson fits into the larger unit
8.
Teacher adjusts the lesson midstream based on Students’ misconceptions
9.
Teacher poses different levels of content questions during the lesson
10.
Teacher states how this lesson connects to content standards
7/17/2015
pbevan
38
STEP # 1: FORM # 1,
Lesson Plan - Pg. 6
Lesson Plan
“Participant Materials”
Read the Domain 1 questions.
 Why are they important?

pbevan 7/17/2015
pbevan
39
39
Lesson Plan Data Collection Tool

Page 6- Lesson Plan

Practice Evidence Collection (Video Clip
High School Social Studies)

Write what is said, not what you think
about it.
7/17/2015
pbevan
40
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Setting Instructional Outcomes
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Designing Student Assessments
Domain 2: The Classroom
Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Participating in a Professional
Community
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating with Students
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
pbevan 7/17/2015
pbevan
41
41
Lunch – 40 minutes
7/17/2015
pbevan
42
Domain 2:
The Classroom Environment
2a: Creating an Environment of Respect and
Rapport
2b: Establishing a Culture for Learning
2c: Managing Classroom Procedures
2d: Managing Student Behavior
2e: Organizing the Physical Space
pbevan 7/17/2015
pbevan
43
43
Participant Materials, Page 8
7/17/2015
pbevan
44
CONCEPT MAP
Domain 2:
Establishing a Culture for
Learning
Big Idea:
Classroom Environment
Component:
3C
Engaging Students
in Learning
UDL
Component:
4C
Communicating with
Families
Culturally
appropriate info
7/17/2015
pbevan
Component:
1B
Demonstrating
Knowledge of
Students
Student background
45
Worksheet #5b- Pg. 9
Focus Questions for Domain 2


Browse Domain 2 of your Rubric
Reflect and answer questions on Worksheet #5
◦ Independently

7/17/2015
Table Share
pbevan 7/17/2015
pbevan
46
46
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Setting Instructional Outcomes
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Designing Student Assessments
Domain 2: The Classroom
Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Participating in a Professional
Community
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating with Students
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
7/17/2015
pbevan
47
Domain 4:
Professional Responsibilities
4a: Reflecting on Teaching
4b: Maintaining Accurate Records
4c: Communicating with Families
4d: Participating in a Professional
Community
4e: Growing and Developing
Professionally
4f: Showing Professionalism
7/17/2015
pbevan
48
Worksheet Step #1:FORM # 1,
Lesson Plan
Worksheet #7, Page 13 Participant Materials
Skim the rubrics in Domain 4,
 Have a table conversation about HOW these
components might impact student learning.
 Self Select an “expert” Group
 Develop a Distinguished response for your
Component.

7/17/2015
pbevan
49
Uses of A Framework for Teaching
Self-Assessment
 Reflection
 Peer Coaching
 Teacher Evaluation
 Mentoring and Induction
 Professional Growth Plans

7/17/2015
pbevan
50
Benefits of Using a Framework
for Teaching
Common Language
 Similar vision for good teaching and how it
can be improved
 Greater validity and reliability in the
teacher evaluation process
 Changes in novice thinking
 Opportunities for collaboration

7/17/2015
pbevan
51
Doing Teacher Evaluation Right
•
5 Rules
•
Evidence
7/17/2015
pbevan
52
Why Evaluate Teaching

Quality Assurance

Professional Learning
7/17/2015
pbevan
53
Doing Teacher Evaluation Right
OBJECTIVES: Participants will learn…

The role of the Framework in teacher
assessment

Best practices in teacher evaluation

How to conduct teacher evaluation to
accommodate quality assurance and
professional learning, too

How to differentiate for novices, experienced
teachers, and at-risk teachers

How to create buy-in for all stakeholders.
(Widget Effect, pp 27-30)
7/17/2015
pbevan
54
5 “Rules” for Teacher
Evaluation
1.
2.
3.
4.
5.
Defensible definition of teaching
Differentiation of evaluative
processes
Evidence-driven process
The role of teacher learning
Transparency
7/17/2015
pbevan
55
Overarching Question

Who does the thinking?

Who does the learning and growing?
7/17/2015
pbevan
56
Rule # 1:
Defensible Definition of Teaching
Start with a
defensible definition
of good teaching that is studied,
and understood, by all
stakeholders.
7/17/2015
pbevan
57
A Framework for Teaching:
Components of Professional Practice
Domain 1: Planning and Preparation
•Demonstrating Knowledge of Content
and Pedagogy
•Demonstrating Knowledge of Students
•Selecting Instruction Goals
•Demonstrating Knowledge of Resources
•Designing Coherent Instruction
•Assessing Student Learning
Domain 2: The Classroom
Environment
•Creating an Environment of Respect
and Rapport
•Establishing a Culture for Learning
•Managing Classroom Procedures
•Managing Student Behavior
•Organizing Physical Space
Domain 4: Professional
Responsibilities
•Reflecting on Teaching
•Maintaining Accurate Records
•Communicating with Families
•Contributing to the School and District
•Growing and Developing Professionally
•Showing Professionalism
Domain 3: Instruction
•Communicating Clearly and
Accurately
•Using Questioning and Discussion
Techniques
•Engaging Students in Learning
•Using Assessment in Instruction
•Demonstrating Flexibility and
Responsiveness
1.3-B
7/17/2015
pbevan
58
The Card Sort
The Framework for Teaching or Danielson
Tools
Use a sticky note
 Write

◦
◦
◦
◦

Domain
Component
Element
Level of Performance
Share with table mates as instructed; get
consensus
7/17/2015
pbevan
59
Rewrite

Select one scenario at your table

Determine tentative Level of Performance

Rewrite at higher and lower levels using
rubric characteristics
7/17/2015
pbevan
60
5 “Rules” for Teacher
Evaluation
1.
2.
3.
4.
5.
Defensible definition of teaching
Differentiation of evaluative
processes
Evidence-driven process
Teacher learning integral
Transparency
7/17/2015
pbevan
61
Rule # 2:
Differentiation of Evaluative Process
Differentiate
the processes of evaluation for:
◦ novices
◦ experienced teachers
◦ teachers at risk
What models are you using in your LEA?
Note PDE Guidelines for Differentiated Supervision
(Requirements Tab)
7/17/2015
pbevan
62
Differentiated Evaluation
Novice/Untenured
Experienced/Tenured
At-Risk
Very close observation
and assessment
Presumption of
professionalism
Not punitive
Formal and informal
observation of
teaching is key +
teacher interviews
+ artifacts
Structured process
1/3yr. Other
years: informals +
teacher interviews+
professional goalsetting
Intensive, extensive
team-based support
based on persistent
unsatisfactory
performance in one
or more components
2 – 4 formal times
per year; multiple
informal
observations
Professional GoalSetting: Choose
from a list of
rigorous, approved
activities
Clear goals,
outcomes, evidence
and timelines anchor
No self-directed
activities
Activities produce
evidence which is
then evaluated
Designed for the
teacher who can,
and wishes, to
improve
7/17/2015
pbevan
63
Overarching Question
Who does the thinking?
 Who does the learning and growing?

7/17/2015
pbevan
64
Formative Assessment

Revisit the Agree or Disagree Assessment
you completed as a pre-test.

Do you wish to make changes to any of
your answers? If so, do it now.
7/17/2015
pbevan
65
5 “Rules” for Teacher
Evaluation
1.
2.
3.
4.
5.
Defensible definition of teaching
Differentiation of evaluative
processes
Evidence-driven process
Teacher learning integral
Transparency
7/17/2015
pbevan
66
Rule # 3:
Evidence Driven Process
Let evidence
-not opinionanchor the process
7/17/2015
pbevan
67
Evidence or Opinion?
1.
2.
3.
4.
5.
6.
7.
The teacher has a warm relationship with the
students.
The teacher said that the South should have won the
Civil War.
The table groups were arranged in 2 x 2 pods.
The materials and supplies were organized well.
Wait time was insufficient for student thinking.
The teacher stated that students have learned to add
2-digit numbers in preparation for today’s lesson.
6 students, questioned randomly, did not know the
day’s learning goals.
7/17/2015
pbevan
68
Evidence or Opinion?
1.
2.
3.
4.
5.
6.
7.
The teacher has a warm relationship with the
students.
The teacher said that the South should have won the
Civil War.
The table groups were arranged in 2 x 2 pods.
The materials and supplies were organized well.
Wait time was insufficient for student thinking.
The teacher stated that students have learned to add
2-digit numbers in preparation for today’s lesson.
6 students, questioned randomly, did not know the
day’s learning goals.
7/17/2015
pbevan
69
Evidence vs. Opinion
Worksheet #6 – Pgs. 11-12
Participant Materials
•
•
•
•
•
is a factual reporting of events,
may include teacher/student actions and
behaviors,
may include artifacts prepared by the
teacher, students or others,
avoids personal opinion or biases,
and is selected using professional judgment
by the observer and/or the teacher.
7/17/2015
pbevan
70
BIAS OR PREFERENCE?

Teaching you do not like is not necessarily
bad teaching.

Teaching can be highly successful in an
approach or style you would not have
personally chosen.

Watch the input of our bias or opinion in
good teaching.
7/17/2015
pbevan
71
Evidence…Observation-based
Assessment: Process and Evidence
Process
Evidence
Pre-Observation
Domains 1 and 4
Standard Lesson Plan with
Components of Domain 1Evidence provided by Teacher
Observation:
Domains 1, 2 and 3
Standard Evidence Collection
Document – Shared with Teachers
Post-Teaching
Domains: 1, 2, 3 and 4
Teacher Self-Assessment, Rubrics
and additions/correction of
evidence gathered
Collaborative Assessment
Domains 1, 2, 3 and 4
Evaluator Rubric and Teacher SelfAssessment Rubric
7/17/2015
pbevan
72
Practice

Watch the pre-observation conference
6th grade middle school math

Collect evidence on the Lesson Plan form
for Domains 1 and 4, Page 13 of the
Participant Materials

Write evidence only.
7/17/2015
pbevan
73
Evidence…Observation-based
Assessment: Process and Evidence
Process
Evidence
Pre-Observation
Domains 1 and 4
Standard Lesson Plan with
Components of Domain 1Evidence provided by Teacher
Observation:
Domains 1, 2 and 3
Standard Evidence Collection
Document – Shared with Teachers
Post-Teaching
Domains: 1, 2, 3 and 4
Teacher Self-Assessment, Rubrics
and additions/correction of
evidence gathered
Collaborative Assessment
Domains 1, 2, 3 and 4
Evaluator Rubric and Teacher SelfAssessment Rubric
7/17/2015
pbevan
74
Types of Observation Evidence

Verbatim scripting of teacher or student comments:
“Could one person from each table collect materials?”

Descriptions of observed teacher or student
behavior:
The teacher stands by the door, greeting students as
they enter.

Numeric information about time, student
participation, resource use, etc.:
Three students of the eighteen offer nearly all of the comments
during discussion.

An observed aspect of the environment:
The assignment is on the board for students to do while roll is
taken.
7/17/2015
pbevan
75
7/17/2015
pbevan
76
7/17/2015
pbevan
77
7/17/2015
pbevan
78
7/17/2015
pbevan
79
Collecting Evidence for D2, D3
Observation Evidence
Watch the lesson, 6th Grade Math
 Collect evidence of what you see and
hear.
 If you aren’t sure where to write the
evidence, just write it.
 This is practice; relax

7/17/2015
pbevan
80
Points about Evidence

All questions are not about 3b

Engagement is about the nature of the
work and who does it

Formative assessments should assess
whether EACH student met the
objectives.
7/17/2015
pbevan
81
Let’s Take a Break

15 minutes
7/17/2015
pbevan
82
Evidence…Observation-based
Assessment: Process and Evidence
Process
Evidence
Pre-Observation
Domains 1 and 4
Standard Lesson Plan with
Components of Domain 1Evidence provided by Teacher
Observation:
Domains 1, 2 and 3
Standard Evidence Collection
Document – Shared with Teachers
Post-Teaching
Domains: 1, 2, 3 and 4
Teacher Self-Assessment, Rubrics
and additions/correction of
evidence gathered
Collaborative Assessment
Domains 1, 2, 3 and 4
Evaluator Rubric and Teacher SelfAssessment Rubric
7/17/2015
pbevan
83
Pre-Observation – Domain 1 and 4
Concluding about the Lesson
2 days before: Teacher provides evidence using Lesson Plan Form
The goal is for the teacher to be Distinguished in Domain I, Planning and Prep.
Teacher and Evaluator discuss evidence provided; Evaluator Collects additional Evidence through
Questioning
Where
your group
mark
During did
the Observation
– Domains
1, 2the
and 3lesson at
or above proficient?
Evaluator arrives early – Walks the Walls
Evidence Collected during the lesson: Avoid Opinions
Evaluator provides Teacher with Evidence Collected during the Observation
Wherefordid
your group
mark 1,below
Preparing
Post-Observation
– Domains
2, 3 and 4
proficient?
Evaluator provides Teacher with Evidence Collected during the
Observation
Teacher self-assesses using highlighter and rubric –and gives to
evaluator
Evaluator assesses and marks all agreed upon – leaves areas of
concern blank to discuss
(No need to discuss the areas of agreement; plan to talk about
the areas of concern only)
Post-Teaching Collaborative Assessment – Domains: 1, 2, 3 and 4
Teacher and Evaluator discuss agreed upon items
Evaluator invites teacher to discuss areas of disagreement
7/17/2015
Teacher develops self-assessment summary
pbevan
84
Middle School Math Observation
Greenberg (pp. 16-20)





Self-Assessment done by Middle School
Math teacher
Read carefully, at your table groups, react
to the teacher’s analysis of the evidence.
Where do you agree?
Where do you disagree?
Do not include your own opinions. Use
only evidence to support the ranking on
the rubric.
7/17/2015
pbevan
85
Mark Components of Agreement

DO mark the components of agreement
on the single copy of the rubric.

Do NOT mark components with which
you are not in agreement. Keep these in
mind when talking with the teacher.

Always let evidence, or the lack there of,
determine the level in the rubric.
7/17/2015
pbevan
86
Post Observation Process
Greenberg video of post
 Compare your assessment to that of
Greenberg and his observers

7/17/2015
pbevan
87
Evidence…Observation-based
Assessment: Process and Evidence
Process
Evidence
Pre-Observation
Domains 1 and 4
Standard Lesson Plan with
Components of Domain 1Evidence provided by Teacher
Observation:
Domains 1, 2 and 3
Standard Evidence Collection
Document – Shared with Teachers
Post-Teaching
Domains: 1, 2, 3 and 4
Teacher Self-Assessment, Rubrics
and additions/correction of
evidence gathered
Collaborative Assessment
Domains 1, 2, 3 and 4
Evaluator Rubric and Teacher SelfAssessment Rubric
7/17/2015
pbevan
88
Types of Observation Evidence

Verbatim scripting of teacher or student
comments:
“Could one person from each table collect materials?”

Descriptions of observed teacher or student
behavior:
The teacher stands by the door, greeting students as
they enter.

Numeric information about time, student
participation, resource use, etc.:
Three students of the eighteen offer nearly all of the
comments during discussion.

An observed aspect of the environment:
The assignment is on the board for students to do while roll
is taken.
2.2-A
7/17/2015
pbevan
89
Building Evaluator Reliability
Reliability refers to similarity of
conclusion/consistency
 Consistency is a function of
consensus-building activities
 Evaluators must practice
consensus building activities
regularly

7/17/2015
pbevan
90
Paradigm Shift
Who Collects/Provides Evidence?
Both teacher and evaluator
Evaluation is not done TO you;
it is done with you and for you
7/17/2015
pbevan
91
Evidence…Observation-based
Assessment: Process and Evidence
Process
Evidence
Pre-Observation
Domains 1 and 4
Standard Lesson Plan with
Components of Domain 1Evidence provided by Teacher
Observation:
Domains 1, 2 and 3
Standard Evidence Collection
Document – Shared with Teachers
Post-Teaching
Domains: 1, 2, 3 and 4
Teacher Self-Assessment, Rubrics
and additions/correction of
evidence gathered
Collaborative Assessment
Domains 1, 2, 3 and 4
Evaluator Rubric and Teacher SelfAssessment Rubric
7/17/2015
pbevan
92
REMINDER: STEPS IN THE
PROCESS
•
•
•
•
•
•
•
•
Teachers get a copy of the evidence immediately following
the lesson
Teachers may add to the evidence
Teachers use the evidence to complete a self-assessment
Teachers assess the lesson by highlighting the appropriate
rubric phrases
Teachers provide this assessment to the observer in
advance of the post teaching conference
The observer review the teacher’s evidence prior to the
post.
The observer highlights in a different color, on his/her
rubric, the components of agreement only prior to the
post.
The observer leaves blank the components of difference
prior to the post conference (the evaluator highlights the
component numbers for easy reference).
7/17/2015
pbevan
93
The Purpose of the
Post-Observation Conference

To discuss the components of difference
(not yet marked by observer)

To elicit any evidence that still remains to
be added about the lesson

To arrive at an assessment on the rubric
for components of difference.
7/17/2015
pbevan
94
Words NOT to Use in the
Post-Observation Conference
Defend
 Prove
 Argue
 Convince

Avoid language that suggests opposition of
that might bring about a defensive response
7/17/2015
pbevan
95
Language for the
Post-Observation Conference
Say more about. . .
 Comment on the evidence for. . .
 Let’s look at the rubric for. . .
 What is the best match for. . .
 What’s the backstory for. . .

7/17/2015
pbevan
96
Framing Suggestions as
Questions
Worksheet #11 page 26
Change the comment to a question that
will elicit the correct response from the
teacher.
Who is to do the Thinking?
 Who is to do the Growing?

7/17/2015
pbevan
97
5 “Rules” for Teacher
Evaluation
1.
2.
3.
4.
5.
Defensible definition of teaching
Differentiation of evaluative
processes
Evidence-driven process
Teacher learning integral
Transparency
7/17/2015
pbevan
98
Rule # 4:
Teacher Learning Integral
Conduct evaluations
in such a way that they produce
teacher learning.
7/17/2015
pbevan
99
Professional Learning
“Learning is done by the learner;
it is mental WORK.”
- Charlotte Danielson
Who does the mental work in your evaluation
process? (Overarching Question)
7/17/2015
pbevan
100
The Nature of Professional Learning:
Mental Work for Teachers
Reflection on practice
 Collaboration
 Self-assessment
 Self-directed inquiry (action
research)
 Feedback based upon evidence

7/17/2015
pbevan
101
“Narrative-Free” Evaluation
The rubric contains the narrative
 Select the language that matches
the evidence
 The teacher participates in language
selection
 The highlighter is the tool
 A summative domain statement is
optional

7/17/2015
pbevan
102
Supporting Teachers Correctly
Directive
Collaborative
Non-Directive
From evaluator Back and forth From teacher
to teacher
to evaluator
Immoral,
illegal,
dangerous,
clueless
Drowning
Both have
ideas to
contribute
The teacher
deserves to
take the lead
Swimming
Championship
Swimming
7/17/2015
pbevan
103
5 “Rules” for Teacher
Evaluation
1.
2.
3.
4.
5.
Defensible definition of teaching
Differentiation of evaluative
processes
Evidence-driven process
Teacher learning integral
Transparency
7/17/2015
pbevan
104
Rule # 5:
Transparency
Teachers must learn the rubrics and
the process.
How might this happen in your
setting?
7/17/2015
pbevan
105
Involving All Stakeholders
Many teacher evaluation systems fail
due to resistance that comes from
the perception that the evaluation
system resulted from the
secret efforts of an elite few.
7/17/2015
pbevan
106
Notification is NOT Communication
Communication is two-way
not one-way
7/17/2015
pbevan
107
Revisit your Agree or Disagree
Assessment
Do you wish to change any answers?
 Report out results

7/17/2015
pbevan
108
7/17/2015
pbevan
109
Questions?
[email protected]
[email protected]
7/17/2015
pbevan
110