Transcript Assessment - What Evidence do You Accept?

Assessment - What Evidence do You Accept?
Diane Ebert-May
Lyman Briggs School
Department of Plant Biology
Michigan State University
HHMI
Our Team at MSU







Doug Luckie - Physiology
Janet Batzli - Plant Biology
Scott Harrison - Microbiology
Tammy Long - Ecology
Heejun Lim - Chemistry Education
Duncan Sibley - Geology
Joyce Parker - Biochemistry
“Consensogram” Directions
1. Take one color-coded post-it for each question,
write the question # in the corner.
2. Write a number between 0-100 on each
post-it in increments of 10.
3. Do not share responses
“Consensogram” Questions
Please respond on a scale of 0 -100 in increments of 10:
1.
2.
3.
4.
5.
6.
To what degree is your course based on active, inquiry-based
learning?
To what degree are your course learning objectives,
instructional design and assessment aligned?
How important is it to use multiple kinds of data to assess your
students?
How often do I use data to make instructional decisions?
In my department, teaching is as important as research for
graduate students (100 agree - 0 disagree).
In my department, effective teaching is rewarded.
(100 agree - 0 disagree)
Goals for Today
As a result of your participation in this workshop, you will...
 Participate in analysis of learning: constructive, inquiry,
discovery, active, problem-based, cooperative, outcomes
based, project-based.
 Examine course goals and predicted learning outcomes.
 Use data to identify student understanding and
misconceptions.
 Use assessment techniques
True or False?

Faculty really are very interested in
assessing their students’ learning better,
but just don’t know how to?
True or False?

Lack of meaningful assessment in
undergraduate education occurs
because faculty are satisfied to be
less accountable in their teaching than
they are in their research.
True or False?

Assessing student learning in science
is more closely related to what
scientists actually do as research than
they realize.
Assessment in ‘Teaching’
Parallels Assessment in ‘Research’








We collect data with a purpose.
Data we collect are aligned with a question about a
problem
Questions we ask are meaningful, interesting, fundable.
Research methods and designs appropriate for question.
Instruments/techniques we use are calibrated.
We explain results in the context of our questions.
Results drive our next questions.
Our ideas are peer reviewed for publication/funding.
What are 3 central questions about
learning?
1. What do we want our students to
know and be able to do?
1.5. What evidence will we accept
that students know and can do?
2. How will we help students get
there?
Cognitive Theory


“Learners are not simply passive
recipients of information; they
actively construct their own
understanding.”
Svinicki 1991
Ultimate goal of teaching:
1. Improved student learning.
2. Improved student learning.
3. Improved student learning.
What Type of Learning?
Bloom (1956) described major
categories in
Cognitive
Domain of Educational Objectives
Convergent Thinking
Knowledge - remember material
 Comprehension - grasp the


meaning of material
Application - use learned material
in new concrete situations
– Adapted from Grolund (1970)
Divergent Thinking



Analysis - break down material to
understand organizational structure
Synthesis - put parts together to form
a new whole
Evaluation - judge value of material for
a purpose
– Adapted from Grolund (1970)
Classroom Environment


Teacher inspires students to struggle
with the discipline - both within and
outside the classroom.
Teacher needs evidence from students
about their progress in learning
What is assessment?


Data collection with a purpose
Courses: gather data about our
students’ learning.
What type of data do we gather?



Depends on the evidence we will accept that
students have learned what we want them to
learn.
Data must be aligned with the course goals.
Measures of knowledge, attitudes, and skills.
» tests, extended responses, concept maps,
» research papers, teamwork, communication
Basic Objectives
Biology Department: Hope College
…students to
 Be active learners - that is to learn
biology by doing biology.
 Learn the basic set of principles and
factual knowledge about each of the
major areas of biology.
Write a Learning Goal
Individually, write a learning goal for
one of your courses (in the context of
the departmental objectives)
 e.g.,

»
»
students will demonstrate…
students will be able to …
Next
Share your goal with a partner in your
group
 Write both goals on large post-its
 Beneath, write possible performance
expectations

Functions of Assessment Data




Formative: diagnostic feedback to
students/instructor
Summative: description of students’ level of
attainment
Evaluative: curricular feedback to instructor
» (e.g., effectiveness of field trip, lab investigation)
Educative: students engaged in interesting, challenging
experiences to develop further insight and
understanding
(Hodson 1992)
In effect...

Assessment IS a form of learning.
Goal => Assessment


Students will be able to demonstrate
their understanding of photosynthesis
and respiration in a variety of
problems.
Tools: multiple forms of assessment
Common Misconceptions: Photosynthesis & Respiration
•
Photosynthesis as Energy: Photosynthesis provides energy for uptake
of nutrients through roots which builds biomass. No biomass built
through photosynthesis alone.
• Plant Altruism: CO2 is converted to O2 in plant leaves so that all
organisms can ‘breathe’.
• All Green: Plants have chloroplasts instead of mitochondria so they
can not respire.
•Thin Air: CO2 and O2 are gases therefore, do not have mass and
therefore, can not add or take away mass from an organism.
Multiple choice question (pre-post)

Plants gain a tremendous amount of weight (dry biomass)
as they grow from seed to adult. Which of the following
substances contributes most to that weight gain
a. compounds dissolved in soil water that are take up by plant
roots
b. water
c. molecules in the air that enter through holes in the plant
leaves
d. organic material in the soil taken up directly by plant roots
e. solar radiation
Carbon Cycle Problem (post)

Two fundamental concepts in ecology are “energy flows” and
“matter cycles”. In an Antarctic ecosystem with the food
web given above, how could a carbon atom in the blubber of
the Minke whale become part of a crabeater seal? Note:
crabeater seals do not eat Minke whales. In your response
include a drawing with arrows showing the movement of the
C atom. In addition to your drawing, provide a written
description of the steps the carbon atom must take through
each component of the ecosystem Describe which biological
processes are involved in the carbon cycle.
Antarctic Food Web
Radish Problem



Experimental Setup:
Weighed out 3 batches of radish seeds each weighing
1.5 g.
Experimental treatments:
» 1. Seeds not moistened (dry) placed in LIGHT
» 2. Seeds placed on moistened paper towels in LIGHT
» 3. Seeds placed on moistened paper
towels in
DARK
Problem (cont)


After 1 week, all plant material was dried in
an oven overnight (no water left) and plant
biomass was measured in grams.
Predict the biomass of the plant material in
the various treatments (use think-pair-share).
» Light, No Water
» Light, Water
» Dark, Water
Results: Weight of Radish Plants
1.46 g
1.63 g
1.20 g
Write an explanation about the results.
(Remember all treatments started as 1.5g).
Misconceptions => Assessment => Instruction



What data do you want from the
assessment?
What do you do when you identify student
misconceptions?
How will the data influence your
instructional design?
Gene-DNA-Chromosome


Students could explain
transcription & translation but
not the relation...
“Gene-DNA-Chromosome.”
Concept mapping forces
students to “Think different”
and confront their (mis)
understanding.
Concept Maps
Concept Maps
are
Visual Diagrams
or Models
Concept Maps
are
Visual Diagrams
or Models
Used for
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
Visual Diagrams
or Models
display
Concepts
connected
with
Used for
Linking Words
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
Visual Diagrams
or Models
Knowledge or
Understanding
has
Structure
Used for
display
represent
Concepts
connected
with
has
Hierarchy
Linking Words
Assessment
Organization
promotes
Reflection &
Learning
Concept Maps
are
display
represent
Visual Diagrams
or Models
has
Hierarchy
is constructed
with
Assessment
Organization
promotes
Reflection &
Learning
connected
with
has
Structure
Used for
Concepts
Knowledge or
Understanding
Prior
Knowledge
Linking Words
Context
New
Information
Assessment Gradient
low
Potential for Assessment of Learning
high
Multiple Choice … … Concept Maps … … Essay … … Interview
high
Ease of Assessment
Theoretical Framework
•Ausubel 1968; meaningful learning
•Novak 1998; visual representations
•King and Kitchner 1994; reflective judgement
•National Research Council 1999; theoretical
frameworks for assessment
low
Assessment and Research


Assessment answers the “what” questions
about student learning.
Research provides explanations about the
“why” and “how” of student understanding.
Open-ended questions


Align with learning goals
What thinking skills do you wish to assess,
choose one questioning format
» interpret data?
» write conclusions from previous work?
» describe?
» solve a problem?
Writing Open-ended Questions



Write a description of the situation.
Write the directions for writing.
Develop a simple rubric
» Conceptual understanding
» Content knowledge
» Critical-thinking processes
» Communication skills
Goal: explain evolution by natural selection
Individual Problem

Explain the phenotypic changes in the
tree and the animal. Use your
understanding of evolution by natural
selection.
How do we develop rubrics?





Describe the goals for the activity, problem, task
Select the assessment tasks aligned with goals
Develop performance standards
Differentiate levels of responses based on clearly
described criteria
Rate (assign value) the categories
Scoring Rubric for Quizzes and Homework
Level of Achievement
Exe mplary
(5 pts)
General Approach
• Addresses the
questi on.
• Sta tes a rel evant,
justi fiable answer.
• Presents argu ments in
a log ical orde r.
• Uses a cceptable style
and gramm ar (no
errors).
Comprehe ns ion
• De monstrates a n accurate an d
complete u nderstan di ng of the
questi on.
• Backs conclusi ons w i th d ata
and warran ts.
• Uses 2 or m ore ideas,
examples and/or argu ments that
support the a nswer.
Ade quate
(3 pts)
• Does n ot address the
questi on expl ici tly,
al though does so
tang enti all y.
• Sta tes a rel evant and
justi fiable answer.
• Presents argu ments in
a log ical orde r.
• Uses a cceptable style
and gramm ar (one
error).
• De monstrates a ccurate b ut only
adequ ate understanding o f
questi on becau se d oes n o t back
conclusions with warrants and
da ta.
• Uses o nly one idea to supp ort
the answer.
• Less thorou gh than above.
Needs Improvement
(1 pt)
• Does n ot address the
questi on.
• Sta tes no re leva nt
answers
• i ndicates
misconceptio ns.
• Is n ot clearl y o r
logical ly orga ni zed.
• Fai ls to use acceptab le
style a nd gram ma r (two
or m ore errors).
• Does n ot dem onstrate accu rate
underst a nd in g of the qu est ion.
• Does n ot provide evidence to
support the ir answer to the
questi on.
No Answer (0 pts)
Advantages of Scoring Rubrics





Improve the reliability of scoring written assignments
and oral presentations
Convey goals and performance expectations of
students in an unambiguous way
Convey “grading standards” or “point values” and relate
them to performance goals
Engage students in critical evaluation of their own
performance
Save time but spend it well
Limitations of Scoring Rubrics

Problem of criteria
Problem of practice and regular use

Scoring Rubric website:

» http://www.wcer.wisc.edu/nise/cl1/flag/

Sample Rubrics for Organismal Biology

http://www.msu.edu/course/lbs/144/f01
Proposal Assessment Plan - Essentials






Agree on goals and objectives for learning
Design and implement a thoughtful approach to
planning
Involve individuals from on/off campus
Select/design data collection approaches
Examine, share, act on assessment findings
Regularly examine assessment process