Transcript Diane Ebert-May Department of Plant Biology Michigan State University Engaging Students: Techniques and

Engaging Students: Techniques and
Strategies to Enhance Student Learning
Diane Ebert-May
Department of Plant Biology
Michigan State University
www.first2.org
HHMI
Goals for Workshop
As a result of your participation, you will...

Think about large courses as environments where you teach less and
students learn more.

Experience a learning cycle model of instruction

Analyze strategies and instructional designs for learner-centered,
large courses

Articulate significant learning goals for your course/curriculum

Determine the ‘why’ and practical use of multiple forms of
assessment
 Use assessment data to drive instructional decisions
Learning Cycle: Models for Instruction

Karplus et al:
BSCS
» Exploration
» Concept Introduction
» Concept Application
»
Engage
Explore
Explain
Elaborate
Engage

Questions are designed to:
» Find out more about students’ thinking
» Level the playing field (everyone involved)
» Remind students they HAVE a role in this
course
» Unveil alternative/misconceptions
“Consensogram” Directions
1. Take one color-coded post-it for each question,
write the question # in the corner.
2. Write a number between 0-100 on each
post-it in increments of 10.
3. Please do not share responses.
“Consensogram” Questions
Please respond on a scale of 0 -100 in increments of 10:
1.
2.
3.
4.
Teaching is a human endeavor that does not and cannot
improve over time. (100 agree - 0 disagree)
Human beings are fantastic learners.
(100 agree - 0 disagree)
Humans don’t learn well in a teaching-centered
classroom. (100 agree - 0 disagree)
What percent of your students appear to understand
concepts in your course “very well” during class time, but
perform disappointingly “less well” on the exams?
Consesogram Qs
(2)
5. To what degree do the assessments you use in
your courses provide convincing data about student
learning?
6. How important is it to use multiple kinds of data to
assess your students?
7. How often do you use data to make instructional
decisions?
8. In my department, teaching is as important as
research and is rewarded accordingly
(100 agree - 0 disagree)
Cooperative Groups
A type of formal structure for in class activities.
 4 students per group
 Person A, B, C, D in each group (assign/select)
 First - read problem/think about task individually
 Discuss: A with B

C with D
 Form group consensus
Explore



Introduce concepts, ideas
Ask more questions
Find out more about what students
know, misunderstand
Assessment in Teaching
Parallels Assessment in Research







We ask questions and develop hypotheses and to solve problems and
make predictions about student learning.
Our questions are based on current knowledge and theories, are
creative, original and relevant to the investigator.
Research designs and methods we use to collect data are logical
arguments to answer questions.
Instruments/techniques we use are valid, repeatable measures of
learning.
Assessment (results) help us understand student thinking.
Results drive our next questions or decisions about a course.
Our ideas are peer reviewed - informally or formally
What is assessment?
Data collection with the purpose of answering
questions about…
» student understanding
» students’ attitudes
» students’ skills
» instructional design and implementation
Graduate Education

Often excellent at preparing
individuals to design and carry out
disciplinary research.
Graduate Education


Often inadequate and haphazard in
preparing future faculty/professionals to
take on the increasingly complex demands
of the professoriate.
Teaching is not mentored, peer reviewed,
or based on accumulated knowledge.
Solution: IRD model


Intergenerational research teams (IRDs) in
cooperative academic environments
» Who: senior faculty, junior faculty, postdoctoral
and graduate students.
» What: scholarship of science teaching and learning
is fully integrated into the professional culture
along with discipline-based activities.
Assessment is critical to both practices.
Collaborators








Janet Batzli - Plant Biology (University of Wisconsin)
Doug Luckie - Physiology
Scott Harrison - Microbiology (graduate student)
Tammy Long - Plant Biology
Jim Smith - Zoology
Deb Linton - Plant Biology (postdoc)
Heejun Lim - Chemistry Education (postdoc)
Duncan Sibley - Geology
Recognizing and Rewarding
Evaluating and Improving Undergraduate
Teaching in Science, Technology,
Engineering, and Mathematics (2003)
»National Research Council
»www.nap.edu/catalog/10024.html
What are central questions about
learning?
1. What do we want our students to know
and be able to do?
2. What knowledge or misconceptions do our
students bring to the course?
3. What evidence will we accept that
students know and can do?
4. How does our instruction help learning?
Cognitive Theory


“Learners are not simply passive
recipients of information; they
actively construct their own
understanding.”
Svinicki 1991
What Type of Learning?
Bloom (1956)
 6 major categories in the
Cognitive Domain of Educational
Objectives
Cognitive Levels



Knowledge - remember
Understanding and Application - grasp
meaning, use, interpret
Critical Analysis - original thinking,
open-ended answers, whole to parts,
parts to whole, evaluation
What is assessment?

Data collection with a purpose
»gather data about students’ learning.
»use tools like Bloom’s taxonomy to
‘calibrate’ data
What type of data do we gather?



Depends on the evidence we will accept that
students have learned what we want them to
learn.
Data must be aligned with the course goals.
Measures of knowledge, attitudes, and skills.
» tests, extended responses, concept maps,
» research papers, teamwork, communication
Assessment -> Inseparable from Instruction




What kind of data do you want from the
assessment? (non-trivial?)
How is data collection embedded in context of
learning over time?
Is assessment of student learning direct, rather
than indirect?
How will the data influence your instructional
design?
False Hopes of Grading (Evaluation)



Total objectivity
Total agreement
Hope for one-dimensional student
motivation for learning.

From Walvoord and Anderson (1998)
Managing Grading
1. Use to enhance learning (socially
constructed/context dependent process).
2. Substitute judgment for objectivity.
3. Distribute time effectively.
4. Be open to change - grades/grading
systems.

From Walvoord and Anderson (1998)
Managing Grades (2)
5. Listen and observe student.
6. Communicate and collaborate with students.
7. Integrate grading with other key processes planning, teaching, interacting.
8. Seize teachable moment - emotional process.
9. Make student learning primary goal - involve
them with high expectations, assessment,
feedback.
Managing Grades (3)
10. Be a teacher first, gatekeeper
last.
11. Encourage learning-centered
motivation.
12. Emphasize student involvement.
Explain
Now:
 Let us work through a detailed
example of assessment of student
understanding…
Learning Goal



Students will be able to demonstrate
their understanding of
photosynthesis and cellular
respiration.
Tools: multiple forms of assessment
Feedback loop to instructional design
Some Common Misconceptions about
Photosynthesis & Respiration
Concept 1: Matter disappears during decomposition of organisms in the soil.
Concept 2: Photosynthesis as Energy: Photosynthesis provides energy for uptake of
nutrients through roots which builds biomass. No biomass built through
photosynthesis alone.
Concept 3: Thin Air: CO2 and O2 are gases therefore, do not have mass and
therefore, can not add or take away mass from an organism.
Concept 4: Plant Altruism: CO2 is converted to O2 in plant leaves so that all
organisms can ‘breathe’.
Concept 5: All Green: Plants have chloroplasts instead of mitochondria so they can
not respire.
Instructional Design



Two class meetings on carbon cycle (160 minutes)
Active, inquiry-based learning
» Cooperative groups
» Questions, group processing, large lecture sections,
small discussion sections, multi-week laboratory
investigation
» Homework problems including web-based modules
Different faculty for each course
» One graduate/8-10 undergraduate TAs per course
Experimental Design
Two introductory courses for majors:
» Bio 1 - organismal/population biology (faculty A)
» Bio 2 - cell and molecular biology (faculty B)
Three cohorts:
» Cohort 1
Bio 1
» Cohort 2
Bio1/Bio2
» Cohort 3
Other/Bio2
Assessment Design



Multiple iterations/versions of the carbon
cycle problem
Pretest, midterm, final with additional
formative assessments during class
Administered during instruction
»Semester 1 - pretest, midterm, final
exam
»Semester 2 - final exam
Multiple choice question (pre-post)
The majority of actual weight (dry biomass) gained by plants
as they progress from seed to adult plant comes from
which one of the following substances?
a. Particle substances in soil that are take up by plant roots.
(15%).
b. Molecules in the air that enter through holes in the plant
leaves (4%).
c. Substances dissolved in water taken up directly by plant
roots. (28%).
d. Energy from the sun (29%).
N=138
Radish Problem (formative)



Experimental Setup:
Weighed out 3 batches of radish seeds each weighing 1.5 g.
Experimental treatments:
» 1. Seeds placed on moistened paper towels in LIGHT
» 2. Seeds placed on moistened paper
towels in DARK
» 3. Seeds not moistened (left DRY) placed in light
Radish problem (2)


After 1 week, all plant material was dried in an
oven overnight (no water left) and plant biomass
was measured in grams.
Predict the biomass of the plant material in the
various treatments.
» Water, light
» Water, dark
» No water, light
Results: Weight of Radish Plants
1.46 g
1.63 g
1.20 g
Write an explanation about the results.
Assessment - depends on purpose




Reports from groups, formative
Peer evaluation
Individual evaluation by instructor
Score - 5 points
Whale Problem (midterm Bio 1)
Two fundamental concepts in ecology are
“energy flows” and “matter cycles”. In an
Antarctic ecosystem with the food web
given above, how could a carbon atom in the
blubber of the Minke whale become part of
a crabeater seal? Note: crabeater seals
do not eat Minke whales. In your response
include a drawing with arrows showing the
movement of the C atom.
In addition to your drawing, provide a written description of the steps the
carbon atom must take through each component of the ecosystem
Describe which biological processes are involved in the carbon cycle.
Grandma Johnson Problem
(final, Bio 1)
Hypothetical scenario: Grandma Johnson had very
sentimental feelings toward Johnson Canyon, Utah, where
she and her late husband had honeymooned long ago. Her
feelings toward this spot were such that upon her death
she requested to be buried under a creosote bush
overlooking the canyon. Trace the path of a carbon atom
from Grandma Johnson’s remains to where it could become
part of a coyote. NOTE: the coyote will not dig up
Grandma Johnson and consume any of her remains.
Jaguar Problem (final, Bio 2)
Deep within a remote forest of Guatemala, the remains of a spider
monkey have been buried under an enormous mahogany tree.
Although rare, jaguars have been spotted in this forest by local
farmers. Use coherently written sentences and clearly labeled
drawings to explain how a carbon atom in glucose contained within
muscle cells of the spider monkey might become part of a cell within
the stomach lining of a jaguar. (Note:The jaguar does not dig up the
monkey and eat the remains!) Include in your answer descriptions of
the key features (not complete biochemical pathways!) of the
organismal and cellular processes that explain how the carbon atom
of the monkey’s corpse could become a part of the jaguar’s body.
Analysis of Responses
Used same scoring rubric for all three problems - calibrated by
adding additional criteria when necessary, rescoring:
Examined two major concepts:
Concept 1: Decomposers respire CO2
Concept 2: Plants uptake of CO2
Explanations categorized into two groups:
Organisms (trophic levels)
Processes (metabolic)
Trace Carbon from Whale to Seal
(Bio1 students, n=141)
100
Organism
Process
80
%
60
40
20
Concept 1
Concept 2
Decomposers respire CO2
Plants uptake CO2
Photosynthesis
Glucose
Through Root
Through Air
Primary produces
Release CO2
Respiration
Decomposers
0
Cellular Respiration by Decomposers
(Bio1/Bio2 students, n=63)
100
80
%
60
40
20
0
Q1 Whale
Q2 Grandma J
Q3 Jaguar
Concept 1: Decomposers respire CO2
2(2) = 20.16, p < 0.01
Pathway of Carbon into Primary Producer
(Bio1/Bio2 students, n=63)
100
Air
Root
80
60
%
40
20
0
Q1 Whale
Q2 Grandma J
Q3 Jaguar
Concept 2: Plants uptake CO2
2(2) = 4.778, p = .092
Trace Carbon from Spider Monkey to Jaguar
100
Respiration
NA
80
60
%
40
20
0
Bio1/Bio2 (n=63)
0ther + Bio2 (n=40)
Concept 1: Decomposers respire CO2
2(1) = 14.59, p < .01
Pathway of Carbon into Primary Producer
100
Air
Root
NA
80
60
%
40
20
0
Bio1/Bio2 (n=63)
0ther + Bio2 (n=40)
Concept 2: Plants uptake CO2
2(1) = 8.89, p < 0.05
So What?
Problem sets about major concepts:
» Diagnostic re: what students
understand/misconceptions
» Methods; parallel to process in disciplinary research
» Predict what prior knowledge students brought to
course
» Learned what knowledge students’ gained
» Unveil new misconceptions
» Influenced what we taught, how we taught it
So What? (2)
Curricular changes:
» Bacteria/Archaea metabolism - often omitted
» Primary production - models in lab
» Source/Sink and carbon flux
» ‘Spiral’ major concepts - over/over/over
» Use of CTOOLS (concept mapping java applet
ctools.msu.edu)
So Who?

Faculty in the disciplines -- use the
research approach they know best to
gather data about student learning to
guide the direction of courses and
curriculum.
Gene-DNA-Chromosome


Students could explain transcription
& translation but not the relation...
“Gene-DNA-Chromosome.”
Concept mapping forces students to
“Think different” and confront their
(mis) understanding.
Concept Maps
Concept Maps
are
Visual Diagrams
or Models
Concept Maps
are
Visual Diagrams
or Models
Used for
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
Visual Diagrams
or Models
display
Concepts
connected
with
Used for
Linking Words
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
Visual Diagrams
or Models
Knowledge or
Understanding
has
Structure
Used for
display
represent
Concepts
connected
with
has
Hierarchy
Linking Words
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
display
represent
has
Hierarchy
is constructed
with
Assessment
Organization
promotes
Reflection &
Learning
connected
with
has
Structure
Used for
Concepts
Knowledge or
Understanding
Visual Diagrams
Or Models
Prior
Knowledge
Linking Words
Context
New
Information
Assessment Gradient
low
Potential for Assessment of Learning
high
Multiple Choice … … Concept Maps … … Essay … … Interview
high
Ease of Assessment
Theoretical Framework
•Ausubel 1968; meaningful learning
•Novak 1998; visual representations
•King and Kitchner 1994; reflective judgement
•National Research Council 1999; theoretical
frameworks for assessment
low
Make a draft of a concept map
Take those three chapters, concepts from your
class, and some post-it notes and make a concept
map.
Steps to making a concept map
1. List the concepts: brain, genome, dog, plant
2. Arrange them - rank-order in terms of the top,
most general, to at the bottom, most specific.
3. Add linking lines that connect the subordinate
concepts under the broader ones.
4. Add linking words that indicate the relationship
between two linked concepts e.g., connect to, are
found in, build proteins inside.
Concepts







Photosynthesis
Respiration
Carbon cycle
Decomposers
Primary producers
Consumers
Carbon dioxide
Glucose
Energy
Water
Oxygen
Make a draft of a concept map
Now form a group of three people, merge your postits and make a concept map of all the science
concepts.
Open-ended questions


Align with learning goals
What thinking skills do you wish to assess,
choose one questioning format
» interpret data?
» write conclusions from previous work?
» describe?
» solve a problem?
Writing Open-ended Questions



Write a description of the situation.
Write the directions for writing.
Develop a simple rubric
» Conceptual understanding
» Content knowledge
» Critical-thinking processes
» Communication skills
C-TOOLS
–Concept ConnectorMSU
http://hobbes.lite.msu.edu/concept/review
– username: guest
– password: concept
–[email protected]
Goals for Today
1. Participate in another learning cycle
of instruction.
2. Practice develop a scoring rubric
3. Develop action plans for courses
and/or curricula.
Goal: explain evolution by natural selection
Individual Problem

Explain the phenotypic changes in the
tree and the animal. Use your
understanding of evolution by natural
selection.
How do we develop rubrics?





Describe the goals for the activity, problem, task
Select the assessment tasks aligned with goals
Develop performance standards
Differentiate levels of responses based on clearly
described criteria
Rate (assign value) the categories
Scoring Rubric for Quizzes and Homework
Level of Achieve ment
Exe mplary
(5 pts)
General Approach
• Addresses the
question.
• States a relevant,
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (no
errors).
Comp rehe nsion
• Demonstrates an accurate and
complete understanding of the
question.
• Backs conclusions with data
and warrants.
• Uses 2 or more ideas,
examples and/or arguments that
support the answer.
Ade quate
(3 pts)
• Does not address the
question explicitly,
although does so
tangentially.
• States a relevant and
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (one
error).
• Demonstrates accurate but only
adequate understanding of
question because does not back
conclusions with w arrants and
data.
• Uses only one idea to support
the answer.
• Less thorough than above.
Needs Improveme nt
(1 pt)
• Does not address the
question.
• States no relevant
answers
• indicates
misconceptions.
• Is n ot clearly or
logically organized.
• Fails to use acceptable
style and grammar (two
or more errors).
• Does not demonstrate accurate
understanding of the question.
• Does not provide evidence to
support their answer to the
question.
No Answer (0 pts)
Advantages of Scoring Rubrics





Improve the reliability of scoring written assignments
and oral presentations
Convey goals and performance expectations of
students in an unambiguous way
Convey “grading standards” or “point values” and relate
them to performance goals
Engage students in critical evaluation of their own
performance
Save time but spend it well
Limitations of Scoring Rubrics

Problem of criteria
Problem of practice and regular use

Scoring Rubric website:

» http://www.wcer.wisc.edu/nise/cl1/flag/

Sample Rubrics for Organismal Biology

http://www.msu.edu/course/lbs/144/f01
Learning Cycle: Models for Instruction

Karplus et al:
BSCS
» Exploration
» Concept Introduction
» Concept Application
»
Engage
Explore
Explain
Elaborate
Engage



Form groups according to discipline or related interests.
1 min: think about a difficult topic you teach well.
3 min: describe to your peers
» Learning goal

» your topic
» how you teach it
» how you assess it
Collectively, make a defining features matrix’ on postits
» Vertical - teaching strategy characteristics
»
- assessment strategy characteristics
» Horizontal - a column for each person to place (+) or (-)
Defining Features Matrix


Requires students to categorize
concepts according to the presence (+)
or absence (-) of important defining
features.
Provides data on analytic reading and
thinking skills.
DFM - Procedure







Focus on 2-3 concepts that are similar enough to confuse
your student
Determine features of concepts most critical for the
students to recognize
List features each concept clearly does or does not
possess
Matrix - features on left side; concepts on top
Check - if each cell can be (+) or (-)
Give copies, or use postits with students
Explain purpose of matrix and directions for filling in.
Data Analysis


Scan/compare/tally
Ask:
» Are students paying more attention to certain
features than others?
» Are students failing to notice defining
differences of specific kinds that would be
obvious to an expert?
Explore
Adaptations of this classroom assessment:
 Students develop their own defining features
matrix on an important topic
 Design matrices to use more than binary
response (e.g., always present, often present,
rarely present, never present)
 Students write statements explaining
configuration of data
Scoring Rubric for Quizzes and Homework
Level of Achieve ment
Exe mplary
(5 pts)
General Approach
• Addresses the
question.
• States a relevant,
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (no
errors).
Comp rehe nsion
• Demonstrates an accurate and
complete understanding of the
question.
• Backs conclusions with data
and warrants.
• Uses 2 or more ideas,
examples and/or arguments that
support the answer.
Ade quate
(3 pts)
• Does not address the
question explicitly,
although does so
tangentially.
• States a relevant and
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (one
error).
• Demonstrates accurate but only
adequate understanding of
question because does not back
conclusions with w arrants and
data.
• Uses only one idea to support
the answer.
• Less thorough than above.
Needs Improveme nt
(1 pt)
• Does not address the
question.
• States no relevant
answers
• indicates
misconceptions.
• Is n ot clearly or
logically organized.
• Fails to use acceptable
style and grammar (two
or more errors).
• Does not demonstrate accurate
understanding of the question.
• Does not provide evidence to
support their answer to the
question.
No Answer (0 pts)
Advantages of Scoring Rubrics





Improve the reliability of scoring written assignments and
oral presentations
Convey goals and performance expectations of students in
an unambiguous way
Convey “grading standards” or “point values” and relate
them to performance goals
Engage students in critical evaluation of their own
performance
Save time but spend it well
Limitations of Scoring Rubrics
1. Problem of criteria
2. Problem of practice and regular use



Scoring Rubric website:
» http://www.wcer.wisc.edu/nise/cl1/flag/
Sample Rubrics for Organismal Biology
http://www.msu.edu/course/lbs/144/f01