Transcript Teacher Evaluation in Ohio - Ohio Education Association

Michele Winship, Ph.D.
[email protected]
 Presentation
slides
 Ohio Teacher Evaluation System
Framework
 Battelle for Kids Value-Added
Resources
 Student Growth Measures Resources
 Race to the Top Work Flow Chart
2




Specifies that the statutory requirements regarding
teacher evaluations prevail over collective bargaining
agreements entered into on or after September 29,
2011.
Allows for third-party evaluators, such as Educational
Service Centers, to be contracted by the board to
perform evaluations (requires that an evaluator must
hold a credential from ODE).
Specifies that the public school teachers who are
subject to the requirement in current law to undergo
evaluation by their employers are those teachers who
are employed under a teacher license and spend at
least 50% of their time employed providing student
instruction.
Requires the district to annually report teacher name
and evaluation rating to ODE.
3

A national push for teacher evaluation reform
from policy makers





Recognition through research that current teacher
evaluation practices are not effective in helping
teachers improve performance and identifying
underperforming teachers
A desire to identify levels of teacher performance to
reward high performers and remove low performers
RttT mandate to change evaluation practices
State-level policies that change evaluation
requirements
Student performance as a significant factor in
teacher (and principal) evaluation (adopted in 13
states currently)
4
5











Sec. 3319.112
(A) Not later than December 31, 2011, the state board of education shall develop a standardsbased state framework for the evaluation of teachers. The framework shall establish an evaluation
system that does the following:
(1) Provides for multiple evaluation factors, including student academic growth which shall
account for fifty per cent of each evaluation;
(2) Is aligned with the standards for teachers adopted under section 3319.61 of the
Revised Code;
(3) Requires observation of the teacher being evaluated, including at least two formal
observations by the evaluator of at least thirty minutes each and classroom walkthroughs;
(4) Assigns a rating on each evaluation in accordance with division (B) of this section;
(5) Requires each teacher to be provided with a written report of the results of the
teacher's evaluation;
(6) Identifies measures of student academic growth for grade levels and subjects for which
the value-added progress dimension prescribed by section 3302.021 of the Revised Code
does not apply;
(7) Implements a classroom-level, value-added program developed by a nonprofit
organization described in division (B) of section 3302.021 of the Revised Code;
(8) Provides for professional development to accelerate and continue teacher growth and
provide support to poorly performing teachers;
(9) Provides for the allocation of financial resources to support professional development.
(HB 153 as signed by the Governor)
6
7
Sec. 3319.111 [Effective 9/29/2011] Teacher evaluation
(A) Not later than July 1, 2013, the board of education that of each school district, in consultation with teachers
employed by the board, shall adopt a standards-based teacher evaluation policy that conforms with the framework
for evaluation of teachers developed under section 3319.112 of the Revised Code. The policy shall become operative
at the expiration of any collective bargaining agreement covering teachers employed by the board that is in effect
on the effective date of this section and shall be included in any renewal or extension of such an agreement.
(B) When using measures of student academic growth as a component of a teacher's evaluation, those measures shall
include the value-added progress dimension prescribed by section 3302.021 of the Revised Code. For teachers of
grade levels and subjects for which the value-added progress dimension is not applicable, the board shall administer
assessments on the list developed under division (B)(2) of section 3319.112 of the Revised Code.
(C)(1) The board shall conduct an evaluation of each teacher employed by the board at least once each school year,
except as provided in divisions (C)(2) and (3) of this section. The evaluation shall be completed by the first day of
April and the teacher shall receive a written report of the results of the evaluation by the tenth day of April.
(2) If the board has entered into a limited contract or extended limited contract with the teacher pursuant to
section 3319.11 of the Revised Code, the board shall evaluate the teacher at least twice in any school year in which
the board may wish to declare its intention not to re-employ the teacher pursuant to division (B), (C)(3), (D), or (E)
of that section.
One evaluation shall be conducted and completed not later than the fifteenth day of January and the teacher being
evaluated shall receive a written report of the results of this evaluation not later than the twenty-fifth day of
January. One evaluation shall be conducted and completed between the tenth day of February and the first day of
April and the teacher being evaluated shall receive a written report of the results of this evaluation not later than
the tenth day of April.
(3) The board may elect, by adoption of a resolution, to evaluate each teacher who received a rating of
accomplished on the teacher's most recent evaluation conducted under this section once every two school years. In
that case, the biennial evaluation shall be completed by the first day of April of the applicable school year, and the
teacher shall receive a written report of the results of the evaluation by the tenth day of April of that school year.
8

HB 153 creates an advantage for RttT
districts



RttT districts can use their funds to buy the time
and support to re-create their evaluation
systems, including the development of an
assessment system
RttT districts can use their funds to purchase
support for assessment systems (data
management, specific testing protocols, testing
materials and grading support)
However…these funds will go away. How will the
systems be supported financially in the future?
9
We must begin with the belief that the main
purpose of teacher evaluation is improved
teaching practice and student learning.
 Teacher performance is to be measured through
multiple sources of evidence, with observation
as one source.
 Student performance is required to be 50% of the
evaluation, BUT student performance is to be
measured through multiple sources of data, not
just a single standardized test score.
 The State Board of Education has adopted a
framework; districts must still develop their
evaluation system that includes processes,
procedures and forms.

10
11
Student academic growth will be measured
through multiple measures which must include
value-added scores on evaluations for teachers
where value-added scores are available.
 Value-added scores are ONLY available for tested
grades and subjects, math and reading in grades
4 – 8. Some extended reports are available in
locals who participate in Battelle for Kids
projects.
 Even if there are value-added scores, there must
be additional student growth measures for all
teachers to meet the multiple measures
requirement.

12
 Districts
will create a local student
growth measure worth 50% of the
evaluation from a combination of
the following:
 Value Added Data
 Vendor Assessment Student Growth
Data
 LEA-developed Student Growth
Measures
13
A. Teacher
Value-Added
10-50%
+
LEA Measures
0-40%
=
Student
Growth
(50%)
B. Approved
Vendor
Assessments
10-50%
+
LEA Measures
0-40%
=
Student
Growth
(50%)
C. No V-A or
Approved
Vendor
Assessments
+
LEA Measures
50%
=
Student
Growth
(50%)
14
 Value-Added
+ Vendor Assessments
 Value-Added + LEA Measures
 Value-Added + Vendor Assessments + LEA
Measures
 Vendor Assessments + LEA Measures
 Multiple LEA Measures
 Currently,
districts will be able to configure
their own combinations which may look
different for different teachers.
15
 Generally
three years’ of data are needed in
order to draw any conclusions.
 Until that data is available in all categories,
districts could choose how to weight
different data.
 Example: 6th grade math teacher with V-A



Year 1: 10% Teacher V-A + 40% LEA Measures
Year 2: 25% Teacher V-A + 25% LEA Measures
Year 3: 40% Teacher V-A + 10% LEA Measures
 There
is no research to determine the best
combination for accuracy and usefulness.
16
 Value-Added
Modeling (VAM) has become the
“gold” standard for measuring educator
effectiveness.
 One year’s growth in one year’s time is the
benchmark = effective.


Teachers who exceed this growth rate have a
positive value-added rating (+) = highly effective
Teachers who fail to meet this growth rate have
a negative value-added rating (-)
 HB
153 supplements districts rated excellent
at $17 per student (the performance index
bar is being raised; few schools will be be
“excellent.”
17



VAM can provide useful information for teachers:
 To look at trends in their classes over time;
 To look at performance of various sub-groups of
students;
 To interpret the impact of their curriculum,
instruction, programs and practices on student
achievement.
However, V-D data is only part of the picture.
“Student test scores alone are not sufficiently
reliable and valid indicators of teacher effectiveness
to be used in high-stakes personnel decisions, even
when the most sophisticated statistical applications
such as value-added modeling are employed.” (EPI
Briefing Paper--Problems with the Use of Student
Test Scores to Evaluate Teachers)
18

Given that students
are not randomly
assigned to classes,
VAM can’t distinguish
between teacher
effects and the
effects based on
students’ needs.

VAM do not provide
information to help
“struggling” teachers.
Lack of properly
scaled year-to-year
tests makes it
difficult to evaluate
gains along the
continuum.
 Mobility of students
(especially in high
needs schools) impact
the data
 VAM cannot
distinguish among
teachers in the
middle range of
performance. 19


About 69% of teachers
can’t be accurately
assessed with VAMs*

Teachers in subject
areas that are not
tested

Teachers in grade levels
(lower elementary)
where no prior test
scores are available

Special education & ELL

VAM estimates vary
with the tests used

If a teacher is in the bottom
quintile based on one test
there is a 43% chance she will
be in the bottom quintile on
a different test, but a 16%
chance she will be in the
top two quintiles.

If a teacher is in the top
quintile based on one test
there is a 43% chance she will
be in the top quintile on a
different test, but a 13%
chance she will be in the
bottom two quintiles.
20
Maintained by ODE
 LRC
SAS® Data
Processing
Accountability
Measures and
Reports



Valued-Added
AYP Growth Measure
ODE Reports-School
and District (LEA)
Measures
Maintained by SAS®
Single Limited Access
Password Protected Data:
 District/LEA and school
 Student information
 Analytic tools
 Teacher-level reports
 Limited Use Public Access
 Includes BFK SAS® EVAAS®
reporting
 Enhanced reporting
features
Developed & supported
by BFK

Diagnostic
Tools
EVAAS®
ODEBFK
Partnership
Regional System
 Trained VAL’s support
districts/LEAs through
DVALT training
 Support to teacherteams
 Focus on school
improvement
 Toolkits
 Online courses

Technical
Assistance
and
Support
21
o
o
o
o
Rollout Schedule
o
30% of LEAs Link in Year 1 RttT (reports received fall 2011)
o
60% of all RttT LEAs in Year 2
o
100% of all LEAs in Ohio in Years 3 & 4
Requirements—Accuracy of Reporting
o
Must conduct linkage
o
Minimum number of students and time enrolled
Access to Reporting
o
Online via EVAAS® accounts
o
Password protected
Grades/Subjects Available
o
ODE: grades 4-8, math & reading
o
BFK: grade 3, math & reading; grades 3-8, science & social
studies; high school—algebra I & II, geometry, pre-calculus,
biology, chemistry, English 9, 10 & 11

Issue—Public Records Requests
22
23
24
25
26
 ODE
has solicited information from test
vendors about the assessments they offer
which are appropriate for measuring student
growth.
 There are many tests that measure “student
achievement,” but fewer that are scaled and
administered in such a way to determine
student growth.
 From the information received, ODE has
published a list of “approved” assessments
from which districts can select to administer.
27
Assessment Name / Vendor Name
Grade(s) / Subject(s)
STAR Early Literacy
Renaissance
STAR Early Math
Renaissance
STAR Reading
Renaissance
Stanford 10
Pearson
PK-3 ELA
Stanford Aprenda 3
Pearson
K-12 Math Science, Social Studies,
Solution 2
Terra Nova 3
CTB
Iowa Assessments
Riverside
Riverside Interim Assessments
Riverside
K-12, 1-12 ELA, Math, Science, Social
Studies
K-12 ELA, Math, Science, Social Studies
Performance
Global Scholar
K-12 ELA, Math, Science
G1-3, 9-12 Math
G1-3, 9-12 Reading
K-12 Math Science, Social Studies,
Solution 1
G 2-11 ELA, Math
28
Assessment Name / Vendor Name
Grade(s) / Subject(s)
iReady Diagnostic
Curriculum Associates
Explore
ACT
Quality Core
ACT
K-8 ELA, Math
The ACT
ACT
PLAN
ACT
Compass
ACT
MAP
NWEA
MAP
NWEA
G 11-12 ELA, Math, Science
G 8-9 ELA, Math, Science
G 9-12 End of Course Exams: Algebra I,
II, Geometry, Pre-Calculus, Biology,
Chemistry, Physics, U S History
G 10 English, Reading, Math, Science
G 10-12 Writing, Reading, Math
G 9-10 Science
G 9-12 Math Reading, Language Usage
29
30
 If
local boards of education choose
to administer assessments chosen
from the Ohio Department of
Education’s assessment list for
teachers of subjects where valueadded scores are not available, they
will have to purchase these
assessments.
 If vendor assessments are used, they
must be part of the 50% SGM.
31
1.
2.
3.
Rigorous. Are measures “rigorous,” focused on
appropriate subject/grade standards?
Measuring students’ progress towards college
and career readiness?
Comparable. Are measures “comparable
across classrooms,” ensuring that students are
being measured with the same instruments and
processes?
Growth over time. Do the measures enable
student learning growth to be assessed
“between two points in time”?
32
4.
5.
Standards-based. Are the measures focused
on assessing growth on important high-quality
grade level and subject standards for students?
Improve teaching. Does evidence from using
the measures contribute to teachers’
understanding of their students’
needs/progress so that instruction can be
planned/adapted in a timely manner to ensure
success?
33
Student Learning Objectives (SLOs) are targets of
student growth that teachers set at the start of
the school year and strive to achieve by the end
of the semester or school year.
 These targets are based on a thorough review of
available data reflecting students' baseline skills
and are set and approved after collaboration and
consultation with colleagues and administrators.
 AISD uses SLOs as part of a performance
compensation model. Teachers set and assess
two student learning objectives and receive
financial incentives for meeting one or both of
these goals at the end of the school year.

34
1.
2.
3.
4.
5.
6.
Review baseline data
Create SLOs
Obtain SLO approval, per local process
Monitor progress toward attainment of SLO
growth targets
Revise SLOs, if necessary
Review evidence and evaluate progress
towards and attainment of SLO growth
targets
35
36
Score
Description
Exceptional
Attainment (5)
Evidence across all Student Learning Objectives indicates exceptional student mastery or progress. All objectives
are exceeded. This category is reserved for the educator who has surpassed expectations described in their
Student Learning Objectives and/or demonstrated and outstanding impact on student learning.
Full
Attainment (4)
Evidence across all Student Learning Objectives indicates superior student mastery or progress. This category
applies to the educator who overall has nearly met the majority of the expectations described in their Student
Learning Objectives and/or who has demonstrated a notable impact on student learning.
Considerable
Attainment (3)
Evidence across all Student Learning Objectives indicates significant student mastery or progress. If an objective
was not met, evidence indicates that is was nearly met. This category applies to the educator who overall has
nearly met the majority of the expectations described in their Student Learning Objectives and/or who has
demonstrated a considerable impact on student learning.
Evidence across all Student Learning Objectives indicates some student mastery or progress. Educator may have
met or exceeded some objectives and no met other objectives. Educator may have nearly met all objectives. This
category applies to the educator who has demonstrated an impact on student learning, but overall has not met
the expectations described in their Student Learning Objectives.
Partial
Attainment (2)
Minimal or No
Attainment (1)
Evidence across all Student Learning Objectives indicates little student mastery or progress. This category applies
to the educator who has not met the expectations described in their Student Learning Objectives and has not
demonstrated a sufficient impact on student learning. This category also applies when evidence of objectives is
missing, incomplete, or unreliable or when the educator has not engaged in their in the process of setting and
gathering evidence for Student Learning Objectives.
 To
be meaningful, student performance data
should be used by educators to



Identify achievement gaps,
Inform instructional practice, and
Direct professional development.
 To
effectively use the data, teams of
educators should



Be trained in the analysis and interpretation of
student performance data,
Have real-time access to the data, and
Meet regularly in teams to analyze the data and
plan intervention, instruction and professional
development.
38
How do we create the conditions
for educators to use student
performance data effectively?
39
 Requiring
student performance in teacher
evaluations means districts will need to:
1.
2.
3.
4.
5.
6.
Map current school-based and district-wide
assessments in all grades and subjects
Determine where assessment “gaps” exist
Create groups of educators to select/develop
appropriate assessments for “gaps”
Create an assessment timeline for all grades
and subjects
Collect, analyze and store student
performance data
Provide time and training for educators to work
together with student data to improve their
own instruction
40
 Use
the assessments you have first.
 Determine what assessments you need
to create a rigorous, comparable and
inclusive assessment system that is
designed to provide student
performance data to be used for
educator professional growth and also
for inclusion in an evaluation system.
 Chart a course of action with a timeline,
persons responsible and deliverables.
41
42
 Each
evaluation will consist of two formal
observations of the teacher at least thirty
minutes each in duration, as well as periodic
classroom walkthroughs.
 Teacher
performance metrics must also use
multiple and variable sources of data,
such as lesson plans, samples of student
work, classroom assessment results, and
portfolios, in addition to data from direct
observation in classrooms.
43
44
45
 Goal



Setting
Self Assessment against Ohio Standards
Analysis of student data
Identifying 2 professional growth goals
 Formative
Assessment of Teacher
Performance—Formal Observation



Pre-observation conference
Observation
Post-observation conference and reflection
 Evidence
Collaboration and Professionalism
(determined locally)
 Student Growth
46
 The
overall teacher performance rating
(50%) will be combined with the results of
student growth measures (50%) to produce a
summative evaluation rating as depicted in
the following matrix.
 Teachers will be rated in one of four
categories:




Accomplished
Proficient
Developing
Ineffective
47
Evaluation Matrix
Above
Expected
Below
Student Growth Measures
Teacher Performance
4
3
2
1
Accomplished
Accomplished
Proficient
Developing
Proficient
Proficient
Developing
Developing
Developing
Developing
Ineffective
Ineffective
48




Teachers with above expected levels of student
growth will develop a professional growth plan and
may choose their credentialed evaluator for the
evaluation cycle.
Teachers with expected levels of student growth will
develop a professional growth plan collaboratively
with the credentialed evaluator and will have input
on their credentialed evaluator for the evaluation
cycle.
Teachers with below expected levels of student
growth will develop an improvement plan with their
credentialed evaluator. The administration will assign
the credentialed evaluator for the evaluation cycle
and approve the improvement plan.
This is entirely unrealistic and does not reflect
what actually happens in schools.
49
 Anyone
who evaluates a teacher will have to
be trained and credentialed by state trainers
(similar to Resident Educator mentor
training)
 Training is 3 days of face-to-face work
viewing teaching videos and learning how to
collect evidence and score using the OTES
rubric
 Credentialing requires passing an online
assessment that will involve viewing a video
and scoring within an acceptable range.
50
At the local level, the board of education will
include in its evaluation policy, procedures for
using the evaluation results for retention and
promotion decisions and for removal of poorlyperforming teachers.
 Seniority will not be the basis for teacher
retention decisions, except when deciding
between teachers who have comparable
evaluations.
 The local board of education will also provide for
the allocation of financial resources to support
professional development.

51
With a July 1, 2013 deadline for system
completion, evaluation work will need to begin
ASAP and may not fit into current bargaining
cycle
 Effective evaluation reform will require
collaboration with administration at a very
different level in many locals
 Future evaluation language in CBAs will need to
include all processes, procedures and tools
 Stakes are high; we can’t afford to adopt
systems that aren’t designed to support teachers

52
 Composition
and selection of evaluation
team members
 Timeline for evaluation work
 Compensation for work outside of the school
day
 Mandatory training for evaluators for
observation protocols and ratings
 Training for staff about evaluation processes,
procedures and tools
 No-fault piloting provision to work out
problems
53
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Identify and engage district evaluation team, including
teachers from various levels/areas
Review and analyze teacher current evaluation polices
and rules
Conduct ODE Evaluation GAP Analysis
Review effective evaluation models including the OTES
Select/Develop a district evaluation system and tools
Map and develop student assessments that will provide
student performance data
Create training for evaluators and teachers
Construct a pilot timeline
Have volunteer teachers and evaluators pilot the system
Review and revise the system based on pilot data
Train all evaluators and teachers
Implement the new evaluation system
54
55
Teacher Evaluation Systems materials and
resources (login required)
http://www.ohea.org/teacher-evaluationsystems
 www.lauragoe.com Includes various state and
local systems and examples of multiple measures
for teacher performance and student growth
 Teacher Assessment and Evaluation: The NEA's
Framework
http://www.nea.org/home/41858.htm
 Getting Teacher Assessment Right: What
Policymakers Can Learn from Research -- the
source for Dr. Hinchey’s
presentation: http://nepc.colorado.edu/publica
tion/getting-teacher-assessment-right

56






Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating
teacher effectiveness: A research synthesis. Washington, DC: National
Comprehensive Center for Teacher Quality.
Goe, L., Holdheide, L., Miller, T. (2011) A practical guide to designing
comprehensive teacher evaluation systems. Washington, DC: National
Comprehensive Center for Teacher Quality.
Hinchy, P. (2010). Getting Teacher Assessment Right: What
Policymakers Can Learn From Research. Boulder, CO: National
Education Policy Center.
Mathers, C., Oliva, M., with Laine, S. W. M. (2008). Improving
instruction through effective teacher evaluation: Options for states
and districts. Research and Policy Brief. Washington, DC: National
Comprehensive Center for Teacher Quality.
National Education Association. (2009). Teacher evaluation systems:
The window for opportunity and reform. Washington, D.C.
Stronge, J. H, & Tucker, P. D. (2003). Handbook on teacher
evaluation: Assessing and improving performance. Larchmont, NY:
Eye on Education.
57
Laura Goe--Webinar for Oregon School Coaches,
April 20, 2011:
http://www.lauragoe.com/LauraGoe/OregonApril%202011.pptx
 EPI Briefing Paper--Problems with the Use of
Student Test Scores to Evaluate Teachers:
http://www.epi.org/publications/entry/bp278
 Rand Education—Evaluating Value-Added Models
for Teacher Accountability:
http://www.rand.org/pubs/monographs/2004/R
AND_MG158.pdf

58
Michele
Winship
614-227-3001
[email protected]
59