Transcript Physics Concept Surveys

Physics Concept Surveys
TDSB physics team
May 28, 2004
Problem

Students do not learn physics concepts very
well under traditional classroom instruction
Problem


Students do not learn physics concepts very
well under traditional classroom instruction
They memorize “plug and chug” techniques,
get 80% on a physics exam, but only score
50% on a multiple choice concept survey
Problem



Students do not learn physics concepts very
well under traditional classroom instruction
They memorize “plug and chug” techniques,
get 80% on a physics exam, but only score
50% on a multiple choice concept survey
In fact, when concept surveys are applied
pre-post, gains are only .286 (30% to 50%)
Purpose

2003-2004: develop concept surveys for
Ontario SPH 3U teachers to use as pre-post
diagnostics to measure conceptual gains
Purpose


2003-2004: develop concept surveys for
Ontario SPH 3U teachers to use as pre-post
diagnostics to measure conceptual gains
2004-2005: encourage teachers to try new
interactive teaching strategies to see if they
lead to increased conceptual gains
History of Project I

August, 2003 – team of physics teachers
developed a plan: Shawn Bullock, Peter
Carlow, Doug Hayhoe, Jason Law
History of Project I


August, 2003 – team of physics teachers
developed a plan: Shawn Bullock, Peter
Carlow, Doug Hayhoe, Jason Law
Sept. 2003 – we studied existing concept
surveys, such as Force Concept Inventory,
as well as the Ontario curriculum
History of Project I



August, 2003 – team of physics teachers
developed a plan: Shawn Bullock, Peter
Carlow, Doug Hayhoe, Jason Law
Sept. 2003 – we studied existing concept
surveys, such as Force Concept Inventory,
as well as the Ontario curriculum
Oct. to Dec. 2003 – drafted and revised five
surveys of approx. 20 MC questions each
(Each item is referenced to the curriculum)
Force and Motion: Unit 1 Survey
12. A large truck collides head on with a small compact car.
During the collision …
A. The truck exerts a greater amount of force on the car than
the car exerts on the truck
B. The car exerts a greater amount of force on the truck than
the truck exerts on the car
C. Neither exerts a force on each other. The car gets
smashed because it is in the way
D. The truck exerts a force on the car but the car does not
exert a force on the truck
E. The truck exerts the same amount of force on the car as
the car exerts on the truck
Curriculum References for Unit 1, #12
Specific Curricular Expectation

state Newton’s laws, and apply them to
explain the motion of objects in a variety of
contexts
Curriculum References for Unit 1, #12
Specific Curricular Expectation

state Newton’s laws, and apply them to
explain the motion of objects in a variety of
contexts
Achievement Chart Criteria

demonstrate understanding of concepts,
laws, and theories
Light and Geometric Optics: Unit 4
17. In the following diagram, two different lenses with different
focal lengths are placed in sequence in an optical device. An
object is located where the arrow is in the diagram.
Left
Right
The second image, after light from the object has passed
through both lenses, would appear to be:
A. to the left of the converging lens and right side up
B. between the two lenses and right side up
C. between the two lenses and upside down
D. to the right of the diverging lens and right side up
E. to the right of the diverging lens and upside down
Curriculum References for Unit 4, #17
Specific Curricular Expectation

describe and explain, with the aid of lightray diagrams, the characteristics and
positions of the images formed by lenses
Curriculum References for Unit 4, #17
Specific Curricular Expectation

describe and explain, with the aid of lightray diagrams, the characteristics and
positions of the images formed by lenses
Achievement Chart Criteria

understand relationships among concepts
History of Project II

January to May, 2004 – Field-tested concept
surveys in 20 schools across the TDSB
History of Project II


January to May, 2004 – Field-tested concept
surveys in 20 schools across the TDSB
April to June, 2004 – conducted item
analysis of questions on concept surveys
History of Project II



January to May, 2004 – Field-tested concept
surveys in 20 schools across the TDSB
April to June, 2004 – conducted item
analysis of questions on concept surveys
May 7, 2004 – physics team met to examine
each question on surveys and recommend
for inclusion, revision or deletion (including
Dianne Ness and Tasha Richardson)
Field-test Results Overall

Unit 1: 9 schools, 306 students, 47% average
Field-test Results Overall

Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Field-test Results Overall


Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Unit 2: 9 schools, 364 students, 52% average
Field-test Results Overall


Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Unit 2: 9 schools, 364 students, 52% average
(3 schools, 69 students, pre-post 41%-52%)
Field-test Results Overall



Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Unit 2: 9 schools, 364 students, 52% average
(3 schools, 69 students, pre-post 41%-52%)
Unit 3: 7 schools, 262 students, 49% average
Field-test Results Overall



Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Unit 2: 9 schools, 364 students, 52% average
(3 schools, 69 students, pre-post 41%-52%)
Unit 3: 7 schools, 262 students, 49% average
(5 schools, 170 students, pre-post 33%-49%)
Field-test Results Overall




Unit 1: 9 schools, 306 students, 47% average
(4 schools, 127 students, pre-post 35%-52%)
Unit 2: 9 schools, 364 students, 52% average
(3 schools, 69 students, pre-post 41%-52%)
Unit 3: 7 schools, 262 students, 49% average
(5 schools, 170 students, pre-post 33%-49%)
Units 4 & 5: results are still being collected
Force and Motion: Unit 1 Survey
12. A large truck collides head on with a small compact car.
During the collision …
A. The truck exerts a greater amount of force on the car than
the car exerts on the truck
B. The car exerts a greater amount of force on the truck than
the truck exerts on the car
C. Neither exerts a force on each other. The car gets
smashed because it is in the way
D. The truck exerts a force on the car but the car does not
exert a force on the truck
E. The truck exerts the same amount of force on the car as
the car exerts on the truck
Item Assessment Criteria

Difficulty level – What % of students
answered the question correctly?
Item Assessment Criteria


Difficulty level – What % of students
answered the question correctly?
Discrimination coefficient – Did a higher %
of better students answer correctly?
Item Assessment Criteria



Difficulty level – What % of students
answered the question correctly?
Discrimination coefficient – Did a higher %
of better students answer correctly?
Pre-post gain – Did more students answer
correctly after studying the topic?
Difficulty Level – 51% Correct
12. A large truck collides head on with a small compact car.
During the collision …
Students writing post-test
306
A. The truck exerts a greater amount of force on the car than
the car exerts on the truck
135
B. The car exerts a greater amount of force on the truck than
the truck exerts on the car
6
C. Neither exerts a force on each other. The car gets
smashed because it is in the way
1
D. The truck exerts a force on the car but the car does not
exert a force on the truck
8
E. The truck exerts the same amount of force on the car as
the car exerts on the truck
156
Discrimination Coefficient = 0.31
12. A large truck collides head on with a small compact car.
During the collision …
306 students divided into higher 153 & lower 153
A. The truck exerts a greater amount of force on the car than
the car exerts on the truck
45
90
B. The car exerts a greater amount of force on the truck than
the truck exerts on the car
3
3
C. Neither exerts a force on each other. The car gets
smashed because it is in the way
0
1
D. The truck exerts a force on the car but the car does not
exert a force on the truck
3
5
E. The truck exerts the same amount of force on the car as
the car exerts on the truck
102
54
Pre-post Gain = 0.54 (21% to 64%)
12. A large truck collides head on with a small compact car.
During the collision …
Students writing both pre and post 127 Pre Post
A. The truck exerts a greater amount of force on the car than
the car exerts on the truck
79
41
B. The car exerts a greater amount of force on the truck than
the truck exerts on the car
5
1
C. Neither exerts a force on each other. The car gets
smashed because it is in the way
3
0
D. The truck exerts a force on the car but the car does not
exert a force on the truck
8
4
E. The truck exerts the same amount of force on the car as
the car exerts on the truck
27
81
Field-test Results for Unit 3, #14
14. Consider the following wave properties:
1. frequency
2. wavelength
3. Amplitude
Which of these properties listed above never changes as a
wave travels along one rope?
A.
B.
C.
D.
E.
1 only
2 only
3 only
1 and 2 only
1 and 3 only
Difficulty Level – 21% Correct
14. Consider the following wave properties:
1. frequency
2. wavelength
3. Amplitude
Students writing post-test
258
Which of these properties listed above never changes as a
wave travels along one rope?
A.
B.
C.
D.
E.
1 only
2 only
3 only
1 and 2 only
1 and 3 only
91
47
48
55
17
Discrimination Coefficient = 0.19
14. Consider the following wave properties:
1. frequency
2. wavelength
3. Amplitude
258 students divided into higher 129 & lower 129
Which of these properties listed above never changes as a
wave travels along one rope?
A.
B.
C.
D.
E.
1 only
2 only
3 only
1 and 2 only
1 and 3 only
57
14
14
40
5
34
33
34
15
12
Pre-post Gain = 0 (21.7% to 21.1%)
14. Consider the following wave properties:
1. frequency
2. wavelength
3. amplitude
Students writing both pre and post 166 pre post
Which of these properties listed above never changes as a
wave travels along one rope?
A.
B.
C.
D.
E.
1 only
2 only
3 only
1 and 2 only
1 and 3 only
40
35
45
36
12
58
29
34
35
10
Future of Project I

June to August, 2004 – preparation of “final”
surveys for pre-post use in 2004-2005
Future of Project I


June to August, 2004 – preparation of “final”
surveys for pre-post use in 2004-2005
They will have this format
Part 1. Units 1-2, Mechanics, 35 items
Part 2. Units 3-5, Waves, Light, EM, 35 items
Future of Project II

Sept-Jan, 2005: collection of baseline data
(volunteer teachers will use revised concept surveys
to collect pre-post course data without changing their
teaching strategies)
Future of Project II

Sept-Jan, 2005: collection of baseline data
(volunteer teachers will use revised concept surveys
to collect pre-post course data without changing their
teaching strategies)

Feb-June 2005: collection of experimental
data (the same teachers will collect pre-post course
data on a second course, in which they use
substantially different teaching strategies)