Transcript Moving Beyond Testing to Assess Learning

Research on Alternative Conceptions in
Student Understanding: the Carbon Cycle
Diane Ebert-May
Department of Plant Biology
Michigan State University
www.first2.org
Question 1
Please respond on a scale of 0 - 100 in increments of 10:
How important is it to use data to assess
student learning?
How important is it to use data to assess
student learning?
40
35
30
%
25
20
15
10
5
0
10
20
30
not important
40
50
60
70
80
90
100
important
Question 2
Please respond on a scale of 0 - 100 in increments of 10:
How often do I use data to make instructional
decisions?
How often do you use data to make
instructional decisions?
30
25
%
20
15
10
5
0
0
10
not often
20
30
40
50
60
70
80
90
often
100
True or False?

Assessing student learning in science
actually parallels what scientists do as
researchers.
Assessment in Teaching
Parallels Assessment in Research







We ask questions and develop hypotheses to solve problems about
learning.
Questions are based on current knowledge and theories and
creative, original and relevant to the investigator.
Research designs and methods used to collect data are logical
arguments to answer questions.
Instruments/techniques we use are valid, repeatable measures of
learning.
Assessment (results) help us understand student thinking.
Results drive our next questions or decisions about a course.
Our ideas are peer reviewed - informally or formally
What is assessment?
Data collection with the purpose of answering
questions about…
» students’ understanding
» students’ attitudes
» students’ skills
» instructional design and implementation
Graduate Education

Often excellent at preparing
individuals to design and carry out
disciplinary research.
Graduate Education


Often inadequate and haphazard in
preparing future faculty/professionals to
take on the increasingly complex demands
of the professoriate.
Teaching is not mentored, peer reviewed,
or based on accumulated knowledge.
Solution: a model

Intergenerational teams in cooperative
academic environments
» Who: senior faculty, junior faculty,
postdoctoral and graduate students.
» What: Scholarship of science teaching and
learning is fully integrated into the
professional culture along with disciplinebased activities.
Collaborators








Doug Luckie - Physiology
Janet Batzli - Plant Biology (University of Wisconsin)
Scott Harrison - Microbiology
Tammy Long - Plant Biology
Jim Smith - Zoology
Deb Linton - Plant Biology, postdoc
Heejun Lim - Chemistry Education
Duncan Sibley - Geology
Recognizing and Rewarding
Evaluating and Improving Undergraduate
Teaching in Science, Technology,
Engineering, and Mathematics (2003)
»National Research Council
»www.nap.edu/catalog/10024.html
What are central questions about
learning?
1. What do we want our students to know
and be able to do?
2. What knowledge or misconceptions do our
students bring to the course?
3. What evidence will we accept that
students know and can do?
4. How does our teaching help learning?
What Type of Learning?
Bloom (1956)
 Major categories in the Cognitive
Domain of Educational Objectives
Convergent Thinking
Knowledge - remember material
 Comprehension - grasp the


meaning of material
Application - use learned material
in new concrete situations
– Adapted from Grolund (1970)
Divergent Thinking



Analysis - break down material to
understand organizational structure
Synthesis - put parts together to form
a new whole
Evaluation - judge value of material for
a purpose
– Adapted from Grolund (1970)
Research Question
How can diagnostic assessment
questions help us understand
students’ thinking about the
carbon cycle?
Background


Conceptual change theory
Force Concept Inventory
(David Hestenes, Physics Dept., ASU)
Carbon Cycle = Rich Problem
Why?




Integrates many biological concepts at multiple
scales.
Instruction can return to elements intrinsic in
the carbon cycle - bioenergetics, metabolism.
Several documented student misconceptions
associated with the carbon cycle.
Real-world applied consequences if students
continue to misunderstand.
Some Common Misconceptions about
Photosynthesis & Respiration
Concept 1: Matter disappears during decomposition of organisms in the soil.
Concept 2: Photosynthesis as Energy: Photosynthesis provides energy for uptake of
nutrients through roots which builds biomass. No biomass built through
photosynthesis alone.
Concept 3: Thin Air: CO2 and O2 are gases therefore, do not have mass and
therefore, can not add or take away mass from an organism.
Concept 4: Plant Altruism: CO2 is converted to O2 in plant leaves so that all
organisms can ‘breathe’.
Concept 5: All Green: Plants have chloroplasts instead of mitochondria so they can
not respire.
Instructional Design

Active, inquiry-based learning
» Cooperative learning
» Questions, group processing, large lecture
sections, small discussion sections, multi-week
laboratory investigations
» Homework problems including web-based
modules
Students - Introductory Biology (majors)
Two courses:
» Bio 1 - organismal/population biolgy (semester)
» Bio 2 - cell and molecular biolgy (semester)
Three cohorts:
» Cohort 1
Bio 1
» Cohort 2
Bio1/Bio2
» Cohort 3
XXX/Bio2
Assessment Design



Multiple iterations/versions of the carbon
cycle problem
Pretest, midterm, final with additional
formative assessments during class
Administered during instruction
» Semester 1 - pretest, midterm, final exam
» Semester 2 - final exam
Multiple choice question (pre-post)
The majority of actual weight (dry biomass) gained by plants
as they progress from seed to adult plant comes from
which one of the following substances?
a. Particle substances in soil that are take up by plant roots.
(15%).
b. Molecules in the air that enter through holes in the plant
leaves (4%).
c. Substances dissolved in water taken up directly by plant
roots. (28%).
d. Energy from the sun (29%).
N=138
Radish Problem (formative)



Experimental Setup:
Weighed out 3 batches of radish seeds each weighing 1.5 g.
Experimental treatments:
» 1. Seeds placed on moistened paper towels in LIGHT
» 2. Seeds placed on moistened paper
towels in DARK
» 3. Seeds not moistened (left DRY) placed in light
Radish problem (2)


After 1 week, all plant material was dried in an
oven overnight (no water left) and plant biomass
was measured in grams.
Predict the biomass of the plant material in the
various treatments.
» Water, light
» Water, dark
» No water, light
Results: Weight of Radish Seeds
1.46 g
1.63 g
1.20 g
Write an explanation about the results.
Evaluation - depends on purpose




Reports from groups
Peer evaluation
Individual evaluation by instructor
Score - 5 points
Whale Problem (midterm F01)
Two fundamental concepts in ecology are
“energy flows” and “matter cycles”. In an
Antarctic ecosystem with the food web
given above, how could a carbon atom in the
blubber of the Minke whale become part of
a crabeater seal? Note: crabeater seals
do not eat Minke whales. In your response
include a drawing with arrows showing the
movement of the C atom.
In addition to your drawing, provide a written description of the steps the
carbon atom must take through each component of the ecosystem
Describe which biological processes are involved in the carbon cycle.
Grandma Johnson Problem
(final, Fall 01)
Hypothetical scenario: Grandma Johnson had very
sentimental feelings toward Johnson Canyon, Utah, where
she and her late husband had honeymooned long ago. Her
feelings toward this spot were such that upon her death
she requested to be buried under a creosote bush
overlooking the canyon. Trace the path of a carbon atom
from Grandma Johnson’s remains to where it could become
part of a coyote. NOTE: the coyote will not dig up
Grandma Johnson and consume any of her remains.
Jaguar Problem (final, Spr 02)
Deep within a remote forest of Guatemala, the remains of a spider
monkey have been buried under an enormous mahogany tree.
Although rare, jaguars have been spotted in this forest by local
farmers. Use coherently written sentences and clearly labeled
drawings to explain how a carbon atom in glucose contained within
muscle cells of the spider monkey might become part of a cell within
the stomach lining of a jaguar. (Note:The jaguar does not dig up the
monkey and eat the remains!) Include in your answer descriptions of
the key features (not complete biochemical pathways!) of the
organismal and cellular processes that explain how the carbon atom of
the monkey’s corpse could become a part of the jaguar’s body.
Analysis of Responses
Used same scoring rubric for all three problems - calibrated
by adding additional criteria when necessary, rescoring:
Examined two major concepts:
Concept 1: Decomposers respire CO2
Concept 2: Plants uptake of CO2
Explanations categorized into two groups:
Organisms (trophic levels)
Processes (metabolic)
Trace Carbon from Whale to Seal
(Bio1 students, n=141)
100
Organism
Process
80
%
60
40
20
Concept 1
Concept 2
Decomposers respire CO2
Plants uptake CO2
Photosynthesis
Glucose
Through Root
Through Air
Primary produces
Release CO2
Respiration
Decomposers
0
Cellular Respiration by Decomposers
(Bio1/Bio2 students, n=63)
100
80
%
60
40
20
0
Q1 Whale
Q2 Grandma J
Q3 Jaguar
Concept 1: Decomposers respire CO2
2(2) = 20.16, p < 0.01
Pathway of Carbon into Primary Producer
(Bio1/Bio2 students, n=63)
100
Air
Root
80
60
%
40
20
0
Q1 Whale
Q2 Grandma J
Q3 Jaguar
Concept 2: Plants uptake CO2
2(2) = 4.778, p = .092
Trace Carbon from Spider Monkey to Jaguar
100
Respiration
NA
80
60
%
40
20
0
Bio1/Bio2 (n=63)
0ther + Bio2 (n=40)
Concept 1: Decomposers respire CO2
2(1) = 14.59, p < .01
Pathway of Carbon into Primary Producer
100
Air
Root
NA
80
60
%
40
20
0
Bio1/Bio2 (n=63)
0ther + Bio2 (n=40)
Concept 2: Plants uptake CO2
2(1) = 8.89, p < 0.05
So What?
Problem sets about major concepts:
» Diagnostic re: what students
understand/misconceptions
» Methods; parallel to process in disciplinary research
» Learn what prior knowledge students brought to
course
» Learned what knowledge students’ gained
» Unveil new misconceptions
» Influenced what we taught, how we taught it
So What? (2)
Curricular changes:
» Bacteria/Archaea metabolism - often omitted
» Primary production - models in lab
» Source/Sink and carbon flux
» ‘Spiral’ major concepts - over/over/over
» Use of CTOOLS (concept mapping java applet
ctools.msu.edu)
So Who?

Scientists in the disciplines -- use the
process they know best to gather data
about student learning to guide the
direction of biology education.