Transcript One Measure of Concept Learning: The Force Concept

A Standard Survey for
Undergraduate
Non-Science Majors:
The Astronomy Diagnostic Test
Beth Hufnagel
University of Maryland, USA
[email protected]
7 August 1999 AAPT Summer Meeting
San Antonio, Texas
1
• Faculty
E. F. (Joe) Redish
David Hammer
John Layman (emeritus)
• Grad Students
Aziza Baccouche
Lei Bao
Hadley Lawler
Rebecca Lippmann
Mel Sabella
• Postdocs
§
Richard Steinberg
Michael Wittmann
Andy Elby*
Apriel Hodari*
Beth Hufnagel*
(§ NAE Spencer Fellow
*PFSMETE Fellow)
2
Overview
Astronomy Education Research
Students knowledge and
beliefs about …
The Universe
Knowing
“Conceptions” “Epistemology”
Themselves
“Affect”
Development and assessment of
curricula and teaching techniques
3
Measuring Concept Learning:
Why A Standardized Survey?
• Understand student preparedness.
• Guide selection of curriculum/teaching approach.
• Measure the effectiveness of different teaching
approaches on learning.
• Document the effectiveness of teaching.
4
Collaboration for Astronomy
Education Research (CAER)
Montana State
Univ. of Maryland
Jeff Adams
Christine Brick
Tim Slater
Grace Deming
Beth Hufnagel
Univ. of Nebraska
Univ. of New Mexico
Rebecca Lindell Adrian
Michael Zeilik
5
What is the
Astronomy Diagnostic Test?
• 33 multiple-choice questions
– 21 concept/knowledge questions
– 12 student background questions
• Validated and Reliable
– A wrong answer means the student doesn’t understand.
– A right answer usually means the student understands.
• Diverse Comparison Database
– 22 classes ranging from 6-201 students
– 17 institutions of 5 types
6
Pre-Course Results Spring 1999
Sample ADT Question
8. Where does the Sun’s energy come from?
A. The combining of light elements
into heavier elements.
B. The breaking apart of heavy elements
into lighter ones.
C. The glow from molten rocks.
D. Heat left over from the Big Bang.
Did not attempt to answer.
Total
No.
420
%
27
870 56
142
9
122
8
3
0
1557 100
1998/1999 Development of
ADT version 2.0
• The choice of ADT topics was based on two prior
diagnostics by Phil Sadler and Mike Zeilik.
• The ADT version 1.0 was re-written using
standard psychometric rules.
• 30 student written responses to open-ended questions.
• 2500+ students in 26 institutions of 6 types
– Statistics, e.g., Spring 1999
Discrimination range 0.32–0.67 (out of –1.00 to +1.00).
• Validation based on ~60 interviews,
– open-ended in Fall 1998,
– with distractors in Spring 1999.
8
Research Questions
Do the results depend on …
• the type of institution community college, liberal arts,
state university, or technical?
Yes
• the class size?
No
• gender?
Yes – Why?
9
Results from Spring 1999
Pre-Course Scores by Gender
400
350
300
250
200
Females
Males
150
100
50
0
7% 21% 35% 50% 64% 78% 82%
Female Male
N
825 683
Mean
28% 38%
Std. Error 0.4% 0.6%
Gender
matters.
10
Summary
• The ADT is now available in PDF format from
http://solar.physics.montana.edu/aae/adt/.
• Encourage conceptual understanding as a
goal for astronomy courses.
• More quantitative measurement of the
effectiveness of curricula/learning tools.
• CAER will develop a web-based, ADT
database with independent concept clusters.
11
Favorite Quote from
Student Interview
“I seem to know, or think I know,
a lot of things. I just don’t know
… the insides of them.”
- Debbie
(Undergraduate astronomy student during
the last week of the course.)
12
Thanks to
these generous
participants:
13
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Phil Appleton, Iowa State University
Chris Depree, Agnes Scott College
Van Dixon, Vista Community College
Andrea Dobson, Whitman College
Jack Dostal, Hawkeye Community College
Doug Hamilton, University of Maryland at College Park
Mary Kay Hemenway, University of Texas at Austin
Thomas J. Kassebaum, Denison University
Janet Landato, W.R. Harper College
Bernie McNamara, New Mexico State University
Heather Morrison, Case Western Reserve University
Jeff Saul, North Carolina State University
Steve Shawl, University of Kansas
Harry Shipman, University of Delaware
Patrick Shopbell, Prince George’s Community College
Greg Stacy, Southern University
14
Bill Warren and Rolfe Chandler, Lord Fairfax Community College
Results from Spring 1999
Pre-Course Scores by Class Size
>200
101-149
81-90
61-70
41-50
21-30
Class Average %
40
35
30
25
20
15
10
5
0
0-10
ADT Score by Class Size
Class Size
The average pre-course score
does not depend on class size.
15
Results Spring 1999 Pre-course
Scores by Institution/Gender
#Classes #Students Women% Men%
State Universities 11
Liberal Arts
5
Community Coll. 4
Technical
1
Women’s
1
635/499
110/124
52/43
6/17
22/0
28(1.0)
26(1.1)
28(1.7)
29(14)
30(8)
37(1.5)
38(5.1)
39(2.6)
59(21)
n/a
(standard error in parentheses)
In all 22 classes, the average
pre-course score for men is higher
than the average score for women.
16
Implications for Instruction:
Explicitly Evoke Useful Ideas
Students come to an introductory astronomy class with a
great deal of knowledge. A professor can help her students use
their existing correct and relevant ideas to construct theories
closer to those held by astronomers.
For example, all of the students knew that the Sun is hot,
and that fluids (like coffee) will cool if no heat source is
present. Use a demonstration to encourage them to apply this
knowledge to stars so that they will conclude stars need an
internal heat source.
17
San Antonio AAPT Summer Meeting
18