Transcript The Reformed Teaching Observation Protocol (RTOP) in

The Reformed Teaching
Observation Protocol (RTOP)
Kathleen Falconer
SUNY-Buffalo State College
University of Alberta
Fall 2013
Abstract
The Reformed Teaching Observation Protocol (RTOP) was
developed as an observation instrument to provide a
standardized means for detecting the degree to which K-20
classroom instruction in mathematics or science is reformed.
RTOP was based upon the previous thirty years of research
on curriculum and instruction for science and mathematics
education as well as a previous observation protocol.
Since RTOP is now over ten years old, I will look at current
research in mathematics education and classroom observation
protocols as it relates to the current validity of RTOP. Also, I
will discuss the use of RTOP as a measure of quality
instruction.
University of Alberta Fall 2013
RTOP
Developed as
• a classroom observation instrument
• a standardized measure for evaluation
• reformed K-20 classroom instruction in
mathematics or science
• NSF ACEPT
University of Alberta Fall 2013
RTOP Instrument
25 Items with 0-4 point scale
 Lesson Design and Implementation.
 Content (Propositional Knowledge)
 Content (Procedural Knowledge)
 Classroom Culture (Communicative
Interactions)
 Classroom Culture (Student-Teacher
Relationships)
University of Alberta Fall 2013
Twenty-five RTOP items
(scored 0-4):
0
the behavior never occurred
1
the behavior occurred at least once
2
occurred more than once; very loosely
describes the lesson
3
a frequent behavior or fairly descriptive
of the lesson
4
pervasive or extremely descriptive of the
lesson
University of Alberta Fall 2013
12) Students made predictions, estimations and / or hypotheses and
devised means for testing them.
This item does not distinguish among predictions, hypotheses and
estimations. All three terms are used so that the RTOP can be descriptive of
both mathematical thinking and scientific r easoning. Another word that
might be used in this cont ext is “conjectures”. The idea is that students
explicitly state what they think i s going to happen before collecting data.
Sample ratings for a lesson on 1-D non-uni form motion with students
looking at position and velocity vs. time graphs.
Score Teacher Action
0
1
2
3
4
Teacher gives the students the formulas for non-uniform motion and
the students solve problems based on the graphs and problem sheets.
Teacher gives the students the theory and formulas for non-uniform
motion, has the students collect, analyze and interpret data using a
step-by-step process.
Teacher sets up an appropriate experiment, has the students make
predictions about what was going to happen, the students do the
experiment and analyze the data following the set procedure.
Teacher sets up an appropriate experiment, has the students make
predictions about what was going to happen, the students do the
experiment with some procedural help and then analyze the data to try
and understand what relationship th ey see.
Teacher asks the students to look at the graphs and asks them to
predict what type of motion gives what graph. Teacher asks students
to design and carry out experiments, which would develop the
relationships for non-linear motion. The t eacher facilitates the
experiment as required.
University of Alberta Fall 2013
RTOP draws on five major sources





Horizon Research 1997-98 Local Systemic
Change Revised Classroom Observation Protocol
”Standards" in science and mathematics education
Principles of reform underlying the ACEPT
project
Work of ACEPT Co-Principle Investigators,
particularly that of Tony Lawson and the ASU
Mathematics Education group led by Marilyn
Carlson
Members of Evaluation Facilitation Group (EFG)
University of Alberta Fall 2013
RTOP





RTOP is a highly reliable instrument
Strong predictive validity
Over 400 K-20 science and mathematics
classrooms
RTOP both operationally defines and assesses
reformed teaching in the classroom.
RTOP scores were found to strongly correlate
with student conceptual gains (Figure 1) showing
that reformed teaching is also effective teaching.
University of Alberta Fall 2013
University of Alberta Fall 2013
So…What does reformed
teaching look like?




Hands on & Minds on
Student-centered
Student dialog and
discourse
Reflection
http://www.youtube.com/watch?v=LM
EAK3CtQIw
University of Alberta Fall 2013
Typical scores are:








traditional university lecture (passive)
university lecture with demonstrations (some student participation)
traditional high school physics lecture (with student questions)
partial HS reform (some group work; most discourse still with teacher)
medium sized (100 > n > 50) university lectures with Mazur-like groupwork (ConcepTests) and a student Personal Response System
the author’s modified (whiteboards etc) large (170 > n > 75) lectures
modeling curriculum, cognitively guided instruction
University of Alberta Fall 2013
< 20
< 30
< 45
< 55
65-75
70-75
65-99
RTOP Website:
http://PhysicsEd.BuffaloState.edu/RTOP
Presentation:
http://PhysicsEd.BuffaloState.edu/pubs/AAPT
mtgs/EdmontonDec2013
University of Alberta Fall 2013
References
Adamson, A.E., Banks, D., Burtch, M., Cox III, F., Judson, E., Turley, J.B., Benford, R. & Lawson, A.E.
(2003). Reformed Undergraduate Instruction and Its Subsequent Impact on Secondary School
Teaching Practice and Student Achievement. Journal of Research in Science Teaching 40(10), 939958.
Cochran-Smith, M., Jong, C., Pedulla, J. J., Reagan, E. M., & Salomon-Fernandez, Y. (2010). Exploring
the link between reformed teaching practices and pupil learning in elementary school mathematics.
School Science and Mathematics, 110(6), 309+.
Flick, L. B., Morrell, P. D., Sadri, P., Schepige, A., & Wainwright, C. (2009). A cross discipline study of
reformed teaching by university science and mathematics faculty. School Science and Mathematics,
109(4), 197+.
MacIsaac, D.L. & Falconer, K.A. (2002, November). Reforming physics education via RTOP. The Physics
Teacher 40(8), 479-485. Available from
< http://physicsed.buffalostate.edu/pubs/TPT/TPTNov02RTOP/ >, describes physics-specific RTOP
use.
Piburn, M., Sawada, D., Falconer, K., Turley, J. Benford, R., Bloom, I. (2000). Reformed Teaching
Observation Protocol (RTOP). ACEPT IN-003. The RTOP rubric form, training manual and
reference manual containing statistical analyses, are all available from
< http://PhysicsEd.BuffaloState.Edu/AZTEC/rtop/RTOP_full/PDF />. IN-001 contains the RTOP
rubric alone, IN-002 constains rubric and trainingmanual, IN-003 adds the statistical analyses.
Sawada, D., Piburn, M., Judson, E., Turley, J., Falconer, K., Benford, R. & Bloom, I. (2002). Measuring
reform practices in science and mathematics classrooms: The Reformed Teaching Observation
Protocol. School Science and Mathematics, 102(6), 245-253. M
.
University of Alberta Fall 2013