Transcript Dissertation Defense - Ohio State University

Observing patterns of classroom
connectivity: Protocol development
and implementation
Stephen J. Pape, University of Florida,
Sukru Kaya, Karen E. Irving, Vehbi Sanalan, Douglas T.
Owens, The Ohio State University
The research reported here was supported by the Institute of Education Sciences, U.S. Department of
Education, through Grant R305K050045 to The Ohio State University. The opinions expressed are those
of the authors and do not represent views of the U.S. Department of Education.
18-Jul-15
1
Additional Authors
L. Abrahamson, Better Education Foundation
S. Granade, Wilkes Community College, &
Sedat Ucar, Cukurova University, Adana, Turkey
Additional Research Team
Frank Demana, The Ohio State University;
Christie Boscardin, Joan Herman, Hye Sook Shin UCLA,
CRESST;
Mike Kositzke, Project Administrative Assistant, OSU
Ugur Baslanti, University of Florida
TI Navigator slide adapted from a presentation by Eileen Shihadeh, Texas Instruments
18-Jul-15
2
Changing roles for teachers
include
 Thinking beyond skills-based conceptions
 Setting norms for discourse
 Fostering mathematical and scientific reasoning
 Using problem solving and inquiry to support
knowledge construction
 Developing mathematical and scientific
competence more broadly defined
 Using formative as well as summative assessment
 Differentiating instruction
18-Jul-15
3
Context -- Classroom Connectivity in Promoting
Mathematics and Science Achievement
 Interdisciplinary research project
 National Sample of Algebra I and Physical
Science teachers
 Classroom connectivity technology
 Professional Development—Summer Institute &
T3 conference follow-up
 Randomized assignment, cross-over design with
control group provided treatment during year 2
 Mixed methodology
18-Jul-15
4
18-Jul-15
™
The TI-Navigator Connected Classroom
The TI-Navigator
System allows the
teacher to:
 Create a collaborative
learning environment
 Engage in formative
assessment by way of
immediate feedback
 Enhance classroom
management of TI
graphing technology
 Quick Poll provides
teacher understanding
by receiving impromptu
feedback
18-Jul-15
5
5
Prior Research on Connected
Classrooms (Roschelle, Penuel, & Abrahamson, 2004)
 Students:
 Increased student engagement; student
understanding; interactivity
 Improved classroom discourse
 Knowledge of classmates’ learning
 Teachers:
 Improved pre- and post- assessment of student
learning
 Increased awareness of student difficulties
 Improved questioning
18-Jul-15
6
Learning Environments in Connected Classrooms
1. Learner Centered
(transfer)
Questions, tasks, and activities to:
• show existing conceptions that
students bring to setting
• extend and make connections
with previous knowledge
3. Assessment Centered
Formative assessment naturally gives:
• feedback to students provides opportunities to
reverse and improve quality of thinking and learning
• feedback to teacher gives cognizance of class
positions and window in student conceptions
4. Knowledge Centered
Focus on:
• conceptual understanding
• reveal, diagnose, and
remedy misconceptions
2. Learner Centered
(Active Engagement)
Appropriate amount of
pressure on students to:
• think through the issues
• establish positions
• commit to positions
5. Sense of Community
Class discussion
Peer interaction
Reasons for actions taken
Knowledge of class positions
Same side as teacher
Lack of embarrassment
Pride in class achievement
Know others have same difficulties
Cheering and enthusiasm
Non-confrontational competition
Owens , Demana , Abrahamson, Meagher, & Herman (2004)
18-Jul-15
7
The Potential of the Connected
Classroom Includes
 Multiple interconnected representations
 Conceptual development supported through
activity-based learning experiences
 Immediate, anonymous formative
assessment
 Public displays of class knowledge
 Teacher identified critical junctures
18-Jul-15
8
The Potential of the Connected
Classroom Includes
 Classroom discourse
 Explanations and justifications
 Focus on process
 Strategic behavior as object of discourse
 Changing classroom atmosphere making
possible
 Increased motivation/engagement
 Positive dispositions toward mathematics and
science
18-Jul-15
9
Theoretical Perspectives for Observation
Protocol Include
 Goal orientation of motivation & social
cognitive view of learning (e.g., Ames, 1992; Ames &
Archer, 1988; Blumenfeld, 1992; Meece, 1991)
 Classroom context that promote involvement
(Turner et al., 1998, 2002)
 Classroom discourse practices
(Cobb, Boufi,
McClain, & Whitenack, 1997; Meyer & Turner, 2002; Nathan & Knuth,
2003; Turner, et al., 1998; Wood, 1999)
 Contexts that support mathematical
understanding (Fennema, Sowder, & Carpenter, 1999)
18-Jul-15
10
Theoretical Perspectives for
Observation Protocol Include
 Contexts that support SRL (DeCorte, Verschaffel, &
Eynde, 2000; Pape, 2005; Perry & VandeKampe, 2000)
 Formative Assessment (Bell & Cowie, 2001; Black &
Wiliam, 1998)
 Analytic & Social Scaffolding
(Nathan, 2003)
 Environmental scaffolding of strategic
behavior (Pape, Bell, & Yetkin, 2003)
18-Jul-15
11
Purpose of Presentation
1. To describe the process of
developing and piloting the CCMS:
Classroom Observation ProtocolAlgebra I
2. To discuss future implementation
strategies
18-Jul-15
12
Observation Protocols Examined
 Professional Development Observation
Protocol (Horizon Research, 2004)
 OPAL: Observing Patterns of Adaptive
learning (Patrick, et al., 1997)
 Oregon Mathematics Leadership Institute
Classroom Observation Protocol (RMC Research
Corporation, 2005)
 Reformed Teaching Observation Protocol
(RTOP) (Arizona Collaborative for Excellence in the Preparation of
Teachers, 2000)
18-Jul-15
13
OPAL – Method of Protocol
Development
 Goal: “to investigate associations between
teacher behaviors and practices and their
students’ perceptions of the motivational
climate of those classrooms” (p. 1)
 Format – narrative running record
 9 categories: Task, Authority, Recognition,
Grouping, Evaluation, Time, Social, HelpSeeking, & Messages
18-Jul-15
14
OPAL – Training
 Project team familiar with literature
 Observation protocol based on literature
 Phase 1
 View videotapes & record running records
 Team discussion of observations
 Consensus building on important points of observation
 Third person review & comment
 Phase 2
 Pair observations of summer school & write running
records
 Compare field notes
 Third person review & comment
18-Jul-15
15
CCMS Protocol Development
 Extensive review of literature including:





Classroom discourse processes
Classroom contexts that promote self-regulated learning
Classroom contexts that increase student motivation
Formative assessment practices
Classroom observation protocols
 Focus—list of instructional practices that support
 Mathematical understanding
 Self-regulation
 Positive dispositions
 Generate list of instructional strategies with
descriptions
18-Jul-15
16
Three-day Training for Research Team—
Day One
 Reviewed constructs to establish
understanding
 Viewed videotape & scripted lesson
 Pairs compared lesson scripts identifying
similarities and differences
 Initial ratings & discussion of ratings
 Generated positive and negative evidence
for constructs
 Revised protocol based on discussion
18-Jul-15
17
Three-day Training for Research Team—
Day Two & Three
 Day Two – Classroom visit
 Day Three – Viewed videotaped class
 Groups observed, scripted, & rated lesson
 Independently indicated ratings
 Groups discuss ratings, discrepancies, and
evidence for ratings
 Revise observation protocol
18-Jul-15
18
Training (Phase 2) & Inter-rater Reliability
 Initial focus – calibration and construct
clarification
 3-4 observers visit class, script, & rate
observation
 Record individual ratings without discussion
 Discussion toward consensus with rationale
for ratings
 Final revisions – deleted/clarified unreliable
categories
18-Jul-15
19
Research Team Considerations and Data
Collection Decisions
 Given focus on communication – camera
focused on teacher in relation to students
 Evidence for ratings from observed behavior
vs. inferences from self-report data
 Three column format: time, behavior,
analytical notes
 Observers – consistency of ratings;
reliability; number of observers
18-Jul-15
20
Inter-rater Reliability
 Inconsistent constructs, in terms of large variability
around the consensus score, included:







Multiple Answers
S-to-T & S-to-S discourse
Initiate-Respond-Evaluate (IRE)
Multiple Representations
Processes
Autonomy
Assessment, Knowledge, & Community Centeredness
 In addition, one rater consistently higher than
others; one rater consistently lower than others
18-Jul-15
21
Changes to Protocol during Development
 Feedback during training incorporated to
elaborate and clarify constructs
 Low reliability constructs revised –
clarification, elaboration, and modification
 Constructs Deleted
 Difficulty of providing evidence – e.g., modifying
tasks to control challenge, strategy application,
teacher knowledge
 Relevance to research – e.g., conversational
turns, wait time
18-Jul-15
22
CCMS Observation Protocol –
Classroom Context Form
I.
Background Information: Date of observation;
Location (City, State); Observer; Observation duration
II. Classroom artifacts: Classroom rules and slogans;
Classroom displays (including graded work); Instructional
materials
III. Classroom layout: Provide a labeled map of the
classroom. Physical arrangement of context “in terms of
specific tasks.”
IV. Scripting the Observation: Specific, objective
descriptions; three column format: (1) time; (2) running log
stated objectively; (3) Reflections, theoretical connections,
observer notes.
18-Jul-15
23
CCMS Observation Protocol –
Observation Rating Form
I.
Classroom context summary
II. Tasks: manner in which teachers structure
tasks and learning activities
18-Jul-15
i.
Structural: Content, procedures (explicit & implicit),
materials, participation structure, and products of task
ii.
Psychological: Student &/or teacher affect assoc with
the task, level of challenge and press, instructional
support
24
CCMS Classroom Observation Protocol
(con’t)
III. Teacher Designed Questions
i.
Lower-Order Questions – require recalling and
stating known facts, carrying out a simple
algorithm or procedure
 Procedural, knowledge, comprehension questions
ii.
Higher-Order Questions – require manipulation
of information, transform information,
synthesize, generalize, explain, hypothesize
 Justification, classification, comparison,
exploration/extension, application, analysis,
synthesis, evaluation questions
18-Jul-15
25
CCMS Classroom Observation Protocol
(con’t)
IV. Sociomathematical norms – ways in which teachers and
students interact within classroom
Teacher press for student involvement – ways teacher ensures &
increases involvement
ii. Teacher press for elaboration, explanation, & justification – ways
teacher pushes students to elaborate to make reasoning explicit;
includes uptake of correct & incorrect responses
iii. Soliciting multiple answers or solution methods
iv. Scaffolding – teacher support such as modeling, questioning;
analytic & social scaffolding
v. Learning strategies – explicit discussion of techniques for learning
vi. Performance orientation – focus on the outcome/products &
showing competence
vii. Mastery orientation – focus on understanding, learning skills with
meaning
i.
18-Jul-15
26
CCMS Classroom Observation Protocol
(con’t)
V. Classroom Discourse -- Relative proportion or
focus on
i.
ii.
iii.
iv.
v.
vi.
vii.
viii.
18-Jul-15
Teacher-to-student – teacher-initiated interaction
Student-to-teacher – student-initiated interaction
Student-to-student interaction
Initiation-Response-Evaluation (IRE)
Classroom management procedures
Use of multiple representations
Collective argumentation – two or more students
contribute to establish a claim
Student discussion of understanding
27
CCMS Classroom Observation Protocol
(con’t)
VI. Formative Assessment – teacher behaviors
targeted toward gathering data from students
and making instructional decision accordingly
VII. Educational goals associated with instructional
decisions
i.
ii.
iii.
iv.
v.
18-Jul-15
Rules and facts
Communication
Application
Evaluation
Mathematical processes
28
CCMS Classroom Observation Protocol
(con’t)
VIII. Knowledge Construction
i.
ii.
iii.
IX.
18-Jul-15
Individually vs. group constructed
Holder of knowledge – degree to which teacher or
students are authority of knowledge construction
Depth of knowledge – superficial vs. central
Technology Use – overall use; quick poll; screen
capture; learn check; student inquiry/data
aggregation; activity center; TI presenter;
graphing calculator; TI viewscreen; Overhead
projector; Computer; Other
29
CCMS Classroom Observation Protocol
(con’t)
X.
Instructional Practices
Traditional/Teacher-centered
ii. NCTM Standards-based, cognitively active
environment, student-centered
iii. Inquiry, problem-based learning, discovery learning
i.
XI. How People Learn
i.
Learner-centeredness
ii. Assessment-centeredness
iii. Knowledge-centeredness
iv. Community-centeredness
18-Jul-15
30
Update/Implementation/Future Work
 To date, 2 years of observations have been conducted
(approx. 15 per Algebra I cohort & 5 per Physical Science cohort)
 Observations will be used to triangulate self-report data
 Teacher Practices and Beliefs Survey
 Student Perceptions of Instruction Survey
 Implementation rating from Telephone Interviews
 Planned Analyses
 Differences between Algebra I and Physical Science teaching
 Changes to protocol to analyze Physical Science instruction
 Purposefully chosen cases examined and elaborated
 Examine classroom instruction in relation to development of student
dispositions and SRL
 Conduct in-depth ethnographies of instruction with classroom
connectivity technology during years 3 & 4
18-Jul-15
31