Transcript The Reformed Teaching Observation Protocol (RTOP) in

The Reformed Teaching
Observation Protocol (RTOP)
in Physics Classes
Daniel MacIsaac
&
Kathleen Falconer
SUNY-Buffalo State College
NYSS AAPT/APS Spring 2004
Symposium
RTOP
Developed as
• a classroom observation instrument
• a standardized measure for evaluation
• reformed K-20 classroom instruction in
mathematics or science
• NSF ACEPT
NYSS AAPT/APS Spring 2004
Symposium
So…
What does a reformed look like?




Hands on
Student-centered
Student dialog and
discourse
Reflection
Quic kT ime™ and a MPEG-4 Video dec ompress or are needed to see this pic ture.
NYSS AAPT/APS Spring 2004
Symposium
RTOP Instrument
25 Items with 0-4 point scale
 Lesson Design and Implementation.
 Content (Propositional Knowledge
 Content (Procedural Knowledge
 Classroom Culture (Communicative
Interactions
 Classroom Culture (Student-Teacher
Relationships)
NYSS AAPT/APS Spring 2004
Symposium
12) Students made predictions, estimation s and / or hypoth eses and
devised means for t esting them.
This item does not di stingui sh among prediction s, hypo theses and
estimations . All th ree terms are used so that th e RTOP can b e descriptive of
both mathe matical thin king and scientif ic reasoning. Anoth er word that
might be used in th is cont ext is “conjectures”. The idea is that student s
explicitly state what they think is going to happ en befo re collecting da ta.
Sample ratings for a lesson on 1 -D non-uni form motion with students
looking at position and velocity vs. time graphs.
Score Teacher Action
0
1
2
3
4
Teacher giv es the students the formul as for non -uniform motion and
the students solve problems based on the graphs and problem sheets.
Teacher giv es the students the theory and formul as for non -uniform
motion , has the students collect, analyze and interpret data using a
step-by-step process.
Teacher sets up an appropriate experiment, has the students make
predictions about what was going to happen, the students do the
experiment and analyze the data follo wing the set procedure.
Teacher sets up an appropriate experiment, has the students make
predictions about what was going to happen, the students do the
experiment with some procedural help and then analyze the data to try
and und erstand what relationship th ey see.
Teacher asks the students to look at the graphs and asks them to
predict what type of motion giv es what graph. Teacher asks students
to design and carry out experiments, which would d evelop the
relationships for non-l inear motion . The t eacher facilit ates the
experiment as requi red.
NYSS AAPT/APS Spring 2004
Symposium
RTOP draws on five major sources





Horizon Research 1997-98 Local Systemic
Change Revised Classroom Observation Protocol
”Standards" in science and mathematics education
Principles of reform underlying the ACEPT
project
Work of ACEPT Co-Principle Investigators,
particularly that of Tony Lawson and the ASU
Mathematics Education group led by Marilyn
Carlson
Members of Evaluation Facilitation Group (EFG)
NYSS AAPT/APS Spring 2004
Symposium
RTOP





RTOP is a highly reliable instrument
Strong predictive validity
Over 400 K-20 science and mathematics
classrooms
RTOP both operationally defines and assesses
reformed teaching in the classroom.
RTOP scores were found to strongly correlate with
student conceptual gains (Figure 1) showing that
reformed teaching is also effective teaching.
NYSS AAPT/APS Spring 2004
Symposium
Normalized Gain vs. Avg RTOP on PCS PHS 110 Fall 1999
1.00
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
Experiment al 1
Experiment al 2
Experiment al3
Experiment al 4
Control 1
The correlation coefficient between Normalized Gain and RTOP is 0.88.
NYSS AAPT/APS Spring 2004
Symposium
Control 2
Normalized Gain
RT OP
Typical scores are:








traditional university lecture (passive)
university lecture with demonstrations (some student participation)
traditional high school physics lecture (with student questions)
partial HS reform (some group work; most discourse still with teacher)
medium sized (100 > n > 50) university lectures with Mazur-like groupwork (ConcepTests) and a student Personal Response System
the author’s modified (whiteboards etc) large (170 > n > 75) lectures
modeling curriculum (varies with amount and quality of discourse)
NYSS AAPT/APS Spring 2004
Symposium
< 20
< 30
< 45
< 55
65-75
70-75
65-99
BSC Summer 2003
Two week long Summer Courses
NYSS AAPT/APS Spring 2004
Symposium
NYSS AAPT/APS Spring 2004
Symposium
NYSS AAPT/APS Spring 2004
Symposium
Table One
Means and Standard Deviations on DIRECT, CSEM and FCI for Summer 2003 Teacher Participants
DIRECT
M
SD
M
Pre
18
5
17
6
22
7
Post
22
4
20
6
25
5
Max. Score29
CSEM
30
FCI
SD
M
30
NYSS AAPT/APS Spring 2004
Symposium
SD
Table Two
Hake Gain and Effect Size for DIRECT, CSEM and FCI for Summer 2003 Teacher Participants
DIRECT
CSEM
FCI
Hake Gain
0.4
0.3
0.4
Effect Size
1
0.6
0.5
NYSS AAPT/APS Spring 2004
Symposium
http://physicsed.buffalostate.edu/rtop
NYSS AAPT/APS Spring 2004
Symposium
References
Adamson, A.E., Banks, D., Burtch, M., Cox III, F., Judson, E., Turley, J.B., Benford, R. & Lawson, A.E.
(2003). Reformed Undergraduate Instruction and Its Subsequent Impact on Secondary School
Teaching Practice and Student Achievement. Journal of Research in Science Teaching 40(10), 939958.
Piburn, M., Sawada, D., Falconer, K., Turley, J. Benford, R., Bloom, I. (2000). Reformed Teaching
Observation Protocol (RTOP). ACEPT IN-003. The RTOP rubric form, training manual and
reference manual containing statistical analyses, are all available from
< http://PhysicsEd.BuffaloState.Edu/AZTEC/rtop/RTOP_full/PDF />. IN-001 contains the RTOP
rubric alone, IN-002 constains rubric and trainingmanual, IN-003 adds the statistical analyses.
Sawada, D., Piburn, M., Judson, E., Turley, J., Falconer, K., Benford, R. & Bloom, I. (2002). Measuring
reform practices in science and mathematics classrooms: The Reformed Teaching Observation
Protocol. School Science and Mathematics, 102(6), 245-253. M
MacIsaac, D.L. & Falconer, K.A. (2002, November). Reforming physics education via RTOP. The Physics
Teacher 40(8), 479-485. Available from
< http://physicsed.buffalostate.edu/pubs/TPT/TPTNov02RTOP/ >, describes physics-specific RTOP
use.
MacIsaac, D.L. & FalconerK.A. (2001-3). Wxx: RTOP Workshops for Physics Teachers. AAPT
Announcer. One day (7 hour) workshops held yearly at the AAPT National Summer Meetings. See <
http://www.aapt.org/Events/ >.
NYSS AAPT/APS Spring 2004
Symposium