Transcript Summative Rating and Impact on Student Learning Matrix

Washington State
Teacher and Principal
Evaluation Project
Combining Multiple Measures Into
a Summative Rating
1
Updated March 2014
Entry Task
As you enter, please take a moment to place a sticky note under the
appropriate place on the chart paper. Make sure to put your district name
on the note, along with which instructional framework you are using.
Four options include:
1. We know the types of evidence that we will use for criterion scoring and
how to derive a criterion score using that evidence.
2. We know the types of evidence that we will use for criterion scoring, but are
still working out how to derive a criterion score using that evidence.
3. We know some of the types of evidence that we will use for criterion
scoring.
4. We do not know the types of evidence that we will collect for
criterion scoring.
2
2
Welcome!
Introductions
Logistics
Agenda




Agenda






3
Connecting
Learning I
Learning II
Implementing
Reflecting
Wrap-Up
Modules
Introduction to Educator Evaluation in Washington
Using Instructional and Leadership Frameworks in
Educator Evaluation
Preparing and Applying Formative Multiple Measures of
Performance: An Introduction to Self-Assessment, Goal
Setting, and Criterion Scoring
Including Student Growth in Educator Evaluation
Conducting High-Quality Observations and Maximizing
Rater Agreement
Providing High-Quality Feedback for Continuous
Professional Growth and Development
Combining Multiple Measures Into a Summative Rating







4
Overview of Intended Outcomes for
Module
Examine the most appropriate types of evidence needed
to assign each criterion score.
Translate multiple forms of evidence to the eight criterion
scores.
Understand how the instructional framework rubric and
student growth rubrics are used within the summative
scoring methodology for the focused and comprehensive
evaluations.
Assign a summative score to EXAMPLE teachers or
principals using the instructional AND leadership
framework rubrics and student growth rubrics.




5
Guidance Icon Key
RCW 28A.405.100
6
A capital “G!” indicates that
the guidance represents
Washington state law.
RCW 28A.405.100
A lower-case “g” indicates that the
guidance represents research-based
best practice but is not mandated by law.
Session Norms

Pausing
Paraphrasing
Posing Questions
Putting Ideas on the Table
Providing Data
Paying Attention to Self and Others
Presuming Positive Intentions

What Else?






7
Connecting
Builds community, prepares the team for learning, and links to
prior knowledge, other modules, and current work
8
Where Are We?
Let’s look at the chart papers from the entry task
Where are most districts at in using multiple forms of
evidence to create criterion scores?




9
What is the distribution of responses?
What is this distribution telling us?
The Year-Long Evaluation Cycle
8 Criteria
Frameworks
+
Student
Growth
Rubrics
•
•
•
Observation
Student
Growth
Evidence
Step 1:
•
•
•
•
Criteria aligned to
instructional /leadership
and student growth
rubrics
Professional goals (
Instructional/leadership
goals (
Student growth goals
(
Step 2 & 3: Select
and collect evidence
•
2 observations (
•
Student growth (
•
Other evidence (
Districtdetermined
process
Statedetermined
process
Distinguished
Proficient
Basic
Unsatisfactory
Distinguished
Proficient
Basic
Unsatisfactory
Step 4:
Step 5:
Determine
8 Criteria
Scores (
10
Summative
Score (
From Plan to Action



Review the goal sheet for Tom Wilson
On your handout (Handout 3), write down two pieces of
evidence that could be collected to show progress
toward that goal.
Think about how the evidence aligns to the criteria.
11
Learning I:
Transforming Evidence
Review the type of evidence needed to assign multiple
criterion scores
Review how to translate multiple forms of evidence to all
eight criterion scores
12
OSPI’s Guiding Principles for
Criterion Scoring for Teacher Evaluation

The primary goal of any system of teacher evaluation is to promote teacher
and student learning.

Accurate teacher evaluation requires trained observers using a researchbased instructional framework. Trained observers make accurate
assessments of practice based on evidence.

The value of accurate assessments of practice is to shape the
conversations that lead to improved practice.

Embedded in each instructional framework is a system for growth in
teaching practice.

Reliability and validity of the instructional framework relies on
implementation of the full framework rather than individual
components/indicators.

It is imperative to remain in the formative mindset until the final summative
rating is determined.
13
Sources of Evidence
for Summative Scoring

RCW 28A.405.100
Three sources of information
1.
Observations based on your chosen instructional
framework
2.
Student growth data as measured by student growth rubrics
3.
Other evidence relevant to the frameworks
14
The Evidence Cycle –
Roles and Responsibilities
RCW 28A.405.100
The Evidence Cycle
3. Interpret
and clarify
1. Collect
2. Sort and
align
15
4. Draw
conclusions
Step of the
Evidence Cycle
Teacher Role
1. Collect
X
2. Sort and align
X
Evaluator Role
X
3. Interpret and clarify
X
4. Draw conclusions
X
Strategies for Collecting Pieces of
Evidence

Identify pieces of evidence that the majority of educators
will need to collect (e.g., lesson plan, parent
communications)

Staff share examples of high-quality pieces of evidence

Discuss how they provide evidence of a criterion score OR
how they can cut across multiple criteria

Think about how the pieces of evidence align to the OSPI
Guiding Principles

Remember, it’s about collecting quality and a variety of
evidence that you already use in your classroom
16
Artifacts From Tom Wilson

Included are five pieces of evidence

Evidence cover pages are missing


17
Alignment to criteria
Evidence statements
Set
Evidence
Location
A
A two-day lesson plan
Handout 3
B
Unit assessment data
Team meeting minutes
Handouts 4 and 5
C
Parent communication log
E-mail exchange
Handouts 6 and 7
Identifying Evidence


Within your school team, divide into pairs
Each pair will do the following:



Review one set of evidence
Complete the evidence cover page for the missing components
Consider these questions:



18
After reviewing these pieces of evidence, do you have enough
evidence to make an accurate assessment of practice?
Do these pieces of evidence provide sufficient evidence to help shape
a conversation that will improve practice?
How can these pieces of evidence be used for teacher growth in
practice?
Learning II:
Summative Scoring
Understand how the instructional framework rubric and student
growth rubrics are used within the summative scoring
methodology for the formative and comprehensive evaluations
Assign a summative score to teachers or principals using the
instructional framework rubrics and student growth rubrics for
the formative and comprehensive evaluations
19
Summative Performance Rating

Things to remember:



20
It is a process, not a final rating!
Balance between professional judgment and transparent
rating process
Uniformity and transparency in developing the summative
rating
Comprehensive Evaluation:
Teachers
RCW 28A.405.100

Assesses all eight evaluation criteria

All criteria contribute to the comprehensive summative
evaluation rating

Student growth rubrics embedded in criteria (3, 6, and 8)

All provisional classroom teachers and classroom
teachers not on level 3 or level 4 receive comprehensive
evaluation

All classroom teachers shall receive a comprehensive
summative evaluation at least once every four years
21
Comprehensive Evaluation:
Principals
RCW 28A.405.100

Assesses all eight evaluation criteria

All criteria contribute to the comprehensive summative
evaluation rating

Student growth rubrics embedded in criteria (3, 5, and 8)

“Due to the importance of instructional leadership and assuring
rater agreement among evaluators, particularly those evaluating
teacher performance, school districts are encouraged to conduct
comprehensive summative evaluations of principal performance
on an annual basis.”
— Section 1, (12 c(v))
22
Comprehensive Evaluation

Three Steps to Assign Comprehensive Evaluation System



23
Step 1: Assign Preliminary Summative Score
Step 2: Determine Impact on Student Learning
Step 3: Use the Summative Rating and Impact on Student Learning
Matrix to determine summative score
Step 1: Comprehensive Evaluation

Step 1: Assign Preliminary Summative Score




24
1a: Transfer criterion scores to summative scoring sheet
1b: Add the eight criterion scores to create a sum
1c: Compare the sum score to the scoring band
1d: Assign a preliminary summative score
The RAW Score Model:
Preliminary Summative Score
Teaching Criteria
* Indicates criterion embedded with student growth rubrics
Criterion 1: Centering instruction on high expectations for student achievement
Criterion 2: Demonstrating effective teaching practices
*Criterion 3: Recognizing individual student learning needs and developing
strategies to address those needs
Criterion 4: Providing clear and intentional focus on subject matter content and
curriculum
Criterion 5: Fostering and managing a safe, positive learning environment
*Criterion 6: Using multiple student data elements to modify instruction and
improve student learning
Overall
Criterion
Score
3
4
3
2
3
2
Criterion 7: Communicating and collaborating with parents and school community
3
*Criterion 8: Exhibiting collaborative and collegial practices focused on improving
instructional practice and student learning
2
Total Summative Score
Evaluators place
teachers into
preliminary summative
rating categories
based on score bands.
As shown, this
teacher would
receive a
preliminary overall
summative rating
of proficient.
22
OSPI-Approved Summative Scoring Band
8–14
1
Unsatisfactory
25
15–21
2
Basic
22–28
3
Proficient
29–32
4
Distinguished
25
Step 2: Comprehensive Evaluation

Step 2: Determine Impact on Student Learning




26
2a: Transfer student growth rubric scores to student growth
summative scoring sheet
2b: Add the five student growth rubric scores
2c: Compare the sum to the student growth scoring band
2d: Assign impact on student learning score
Student Growth Rubric and Rating
(Teachers Only)
Student
Growth
Criterion 3
Criterion 6
Criterion 8
Student Growth
Score
Goal-Setting
Score Based
on Rubric
Student
Growth* Score
Based on
Rubric
3
2
2
2**
2**
N/A
Overall
Student
Growth
Criterion
Score
5
4
2
7
4
11
Evaluators place
teachers into
summative rating
categories based on
score bands. As
shown here, this
teacher would
receive a low
student growth
rating.
OSPI-Approved Student Growth Impact Rating
Scoring Band
5-12
Low
13-17
Average
18-20
High
*Must include a minimum of two student growth measures (i.e., state-, district-, school-, and
classroom-based measures).
** A student growth score of “1” in any of the student growth rubrics will result in a
low growth rating.
27
27
Summative Rating & Impact on
Student Learning Matrix
Distinguished
Proficient
Rating
Distinguished Rating
Summative Rating
Student Growth Inquiry
Proficient
Proficient
Rating
Student Growth Inquiry
Basic
Basic Rating
Student Growth Inquiry
Consequences as a result of intersection
between summative rating and impact
on student learning rating
Basic Rating
Unsatisfactory Rating
Unsatisfactory
28
Proficient Rating
Plan of improvement
Low
Average
High
Impact on student learning
28
Step 3: Comprehensive Evaluation

RCW 28A.405.100
Step 3: Use the Summative Rating and Impact on Student Learning
Matrix to determine summative score

Educators with preliminary rating of distinguished with average
or high student growth rating: These educators will receive an
overall distinguished rating and will be formally recognized and/or
rewarded (per regulations).

Educators with preliminary rating of unsatisfactory and high
student growth rating: These evaluations will be reviewed by the
evaluator’s supervisor when an educator is rated unsatisfactory and
receives a high student growth rating. The supervisor will take these
discrepancies into account in the evaluator’s evaluation.

Educators who receive a score of 1 on the achievement of student
growth goals will automatically receive a low student growth rating.
29
Learning Activity:
Putting the Pieces Together

Step 1: Determine framework dimension/component score


Step 2: Determine criterion score


Use summative scoring sheet (Handout 8)
Step 4: Determine student growth impact rating


Use criterion scoring sheet (instructional framework-specific;
Handout 12, and Handout 13a, b, or c)
Step 3: Determine preliminary summative score


Use evidence collection alignment form (instructional frameworkspecific; Handout 14a, b or c)
Use student growth rubric and rating form (Handout 9)
Step 5: Determine comprehensive evaluation score

30
Use summative rating and impact on student learning matrix
(Handout 10)
Putting the Pieces Together Debrief


Find an individual from another district using the same
framework
Discuss three main questions

Did you come up with the same summative score?


How do/how will the five steps fit in with your district's
current process?


31
Why or why not?
Will you have to make big changes or adjust a few things?
How can/will the summative scoring process be used as a way
to promote professional growth?
Focused Evaluation
Certificated Classroom Teachers
RCW 28A.405.100

Includes an assessment of one of the eight criteria

Student growth rubrics from one of the three criteria
 If a teacher chooses criterion 3, 6, or 8, their accompanying
student growth rubrics will be used.
 If a teacher chooses criterion 1, 2, 4, 5, or 7, the accompanying
student growth rubrics from criterion 3 or 6 will be used.

Approved by the teacher’s evaluator

A focused evaluation must be performed in any year that a
comprehensive evaluation is not scheduled
32
Focused Evaluation
Principals and Assistant Principals
RCW 28A.405.100

Includes an assessment of one of the eight criteria

Student growth rubrics from one of the three criteria

The focused evaluation will include the student growth rubric
row selected by the principal or assistant principal.

Criterion and student growth rubric rows must be
approved by the principal’s evaluator

A focused evaluation must be performed in any year that
a comprehensive evaluation is not scheduled
33
Focused Evaluation: Selected Criterion
Includes Student Growth Score

Similar process to criterion scoring
Criterion 3: Marzano
Formative evaluation score based on two Marzano components aligned to criterion
3 and two student growth rubric scores aligned with criterion 3
Indicator
Unsatisfactory
1
Basic
2
3.1 Effective scaffolding of
information
Distinguished
Summative Score
4
X
3.2 Planning and preparing
for needs of all students
X
Criterion 3: Goal-setting
student rubric score
X
Criterion 3: Student growth
rubric score
X
34
Proficient
3
2
Focused Evaluation: Selected Criterion
Does Not Include Student Growth Score

To create formative score, the following are needed:


Component scores for the criterion (similar to criterion score)
Growth rubric scores from criterion 3 or 6 (from teachers); from criterion 3 or 5 (from
principals)
Criterion 4: Marzano
Formative evaluation score based on two Marzano components aligned to criterion 4
and two student growth rubric scores aligned with criterion 6
Indicator
Unsatisfactory
1
Basic
2
Proficient
3
4.1 Attention to est. content
standards
X
4.2 Use available resources
and technology
X
Criterion 6: Goal-setting
student rubric score
X
Criterion 6: Student
growth rubric score
35
Distinguished
Summative Score
4
?
X
Learning Activity IIB: Reteach

Split into pairs.You will reteach each other the process of
the focused evaluation process.

When you reteach, provide an example from the teacher or
principal framework your district uses.

Partner 1: Reteach the focused evaluation process when
educator selects a criterion that includes student growth.

Partner 2: Reteach the focused evaluation process when
educator selects a criterion that does NOT include
student growth.
36
Implementing
Supports teams in problem solving and planning next steps for
schools and districts
37
Identifying Tools and Processes for
Gathering and Organizing Evidence



Create a plan for implementing the data collection
process and how that will be used for summative
evaluation.
Use Handout 11 (Implementation Planning – Three Steps)
from your packet to structure your conversation with
your school team.
This packet is similar what you did at the end of the
Preparing and Applying Formative Multiple Measures of
Performance module

38
If you have completed part of it, go back through your
decisions and refine them based on this module, and any other
modules you have completed since then.
Implementing Activities Debrief

Each team shares one decision that was made today to
increase the clarity and feasibility of the teacher
evaluation process.
39
Reflecting
Engages participants in providing feedback, reflecting on learning,
and closing the session
40
Debrief: Hand Plant



Plant your hand on a piece
of blank paper and trace it.
On each finger, write the
five most important facts
to remember and teach
others about combining
multiple measures into a
summative rating.
Share at your tabletops
and be prepared to share
one with the large group.
41
What’s Next?

Homework Options


42
District: Continue to work on a district teacher evaluation
guidebook that includes all of the nuts and bolts of the teacher
evaluation process. Use the information you recorded on the
“Implementation Planning” handout as a starting place.
School or Teams: Identify the processes and procedures at your
school for how the evidence teachers collect will be organized
and stored for effective implementation of the summative
scoring process.
Thank you!
Presenter’s Name
XXX-XXX-XXXX
[email protected]
1234 Street Address
City, State 12345-1234
800-123-1234
43