Transcript Science Faculty with Education Specialties: One Scholar’s

Identifying Core Concepts and
Constructing Concept Inventories
Jenny L. McFarland, Ph.D.
Biology Department
Edmonds Community College
Collaborators: Joel Michael (Rush Medical College), Mary Pat Wenderoth
(Univ of Washington), Bill Cliff (Niagara University), Ann Wright (Canisius
College), Harold Modell (Physiology Education Research Consortium, PERC
APS Intersociety Meeting:
Comparative Approaches to Grand Challenges in Physiology
7 October 2014, San Diego, CA
Supported by NSF grant DUE-1043443
Backwards Design & Core Concepts
1. What is “Backwards Design”?
2.



What are the core concepts for
undergraduate biology?
physiology education?
comparative physiology education?
3. What is a concept inventory?
4. Development of a homeostasis conceptual
assessment for undergraduate physiology.
Backwards Design & Core Concepts
1. Identify desired goals or outcomes
 Core Concepts
2. Determine acceptable evidence (design
conceptual assessment)
 Concept Inventories
3. Design learning experiences/ instruction
 Student-centered active learning
4. Evaluate alignment of instruction, outcomes
and assessment
 instructor metacognition: explicit assessment
of teaching & learning
Wiggins and McTighe 1998; Dirks, Wenderoth and Withers, 2014
Backwards Design in this workshop
1. Identify desired goals or outcomes
 Cynthia Bauerle, Jenny McFarland
2. Determine acceptable evidence (design
conceptual assessment)
 Douglas Luckie, Jenny McFarland
3. Design learning experiences/ instruction
 Barbara Goodman, Douglas Luckie
4. Evaluate alignment of instruction, outcomes
and assessment
 Miranda Byse – finding resources for doing this!
What are the Core Concepts?
Step 1 in Backwards Design: Identify desired
goals or outcomes for undergraduates
 What are the Core Concepts in Biology?
 What are the Core Concepts in Physiology?
 What are the Core Concepts in Comparative
Physiology? – contact Kerry Hull ([email protected])
Biology Core Concepts: Vision & Change
The Vision & Change report
identified 5 core concepts
for undergraduate biology
Evolution
Structure and Function
Energy and matter
Information flow
Systems
AAAS 2011
Physiology Core Concepts
Physiology faculty identified 15 core concepts
Some physiology core concepts were explicitly identified
in the Vision and Change Report, others were not.








Homeostasis (systems)
Cell Membrane
Cell-Cell Communication
Interdependence
Flow Down Gradients
Energy
Structure/Function
Scientific Reasoning (V&C







core competency)
Cell Theory
Physics/Chemistry
Genes to Proteins (info)
Levels of Organization
Mass Balance
Causality
Evolution
AAAS 2011; Michael and McFarland 2011
Comparative Physiology Core Concepts
A group of physiology faculty at the APS Institute on Teaching & Learning (APSITL, June 2014) have begun to identify Core Concepts in Comparative
Physiology.
This group has asked me to share their core concepts & competencies with you
and request your help. The next five slides will not be discussed in this talk
(see handout), but are included here to encourage comparative physiologists
here to participate in this effort.
Comparative Physiology Core Concepts
AP1. Animals use diverse physiological
mechanisms/strategies to solve similar
environmental challenges.
AP2. Animals inhabit diverse habitats and
some possess unique physiology (adaptations)
allowing for survival in so-called extreme
environments.
AP6. Physiological phenotype is a product of
genotype and environment.
AP7. Life cycles and life history traits influence
physiological processes.
AP8. Comparative physiology informs the
medical physiology of humans.
AP3. The evolutionary and developmental
origin and history of an animal constrains
anatomical structures and physiological
processes.
AP9. Comparative physiology brings to light
how body systems interact to meet
environmental challenges and promotes an
understanding of integrative physiology
AP4. The capabilities of interacting
physiological systems represent dynamic
trade-offs in function and efficiency.
AP10. Physiological phenomena can be
explained in multiple, compatible ways that
involve different levels of functional
complexity and different scales of time
(molecular, cellular, developmental,
organismal, environmental/ecological,
evolutionary)
AP5. Body size influences the behavior of
physiological systems.
Concept of “Trade-Offs” Unpacked
AP4. The capabilities of interacting physiological systems represent dynamic trade-offs in function
and efficiency.
1. Optimal operation of a body system may adversely impact the homeostasis achieved by other
body systems and/or the optimal operations of these systems.
2. Body systems may work below optimal capacity in order to reduce or avoid disturbances in
homeostasis and/or adverse changes in other body systems. The activities of individual body
systems are adjusted to achieve a trade-off in the combined actions of different body systems.
This trade-off favors overall homeostasis and results in the most efficient operation of the body
under the existing system constraints.
3. Trade-offs in body system activities are prioritized according to the importance of each activity
to the immediate survival of the animal. Less commonly, the animal will compromise immediate
survival in order to increase reproductive success.
4. A number of factors interact to determine the balance point for physiological tradeoffs. These
include:
Body size of the animal
Life history of the animal
Phenotypic plasticity of the animal
Environmental conditions
Comparative Physiology Core Competencies
AP1. Demonstrate knowledge that animals use
diverse physiological mechanisms/strategies to
solve common environmental challenges.
AP6. Demonstrate understanding that
physiological phenotype is a product of
genotype and environment.
AP2. Demonstrate understanding that animals
inhabit diverse habitats and some possess
unique physiology (adaptations) allowing for
survival in so-called extreme environments.
AP7. Explain how life cycles and life history
traits influence physiological processes.
AP3. Recognize that the evolutionary and
developmental origin and history of an animal
constrains anatomical structures and
physiological processes.
AP4. Analyze the dynamic trade-offs in
function and efficiency seen in interacting
physiological systems.
AP5. Demonstrate understanding of how body
size influences the behavior of physiological
systems.
AP8. Demonstrate knowledge of how
comparative physiology informs the medical
physiology of humans.
AP9. Recognize how comparative physiology
brings to light how body systems interact to
meet environmental challenges and promotes
an understanding of integrative physiology
AP10. Explain physiological phenomena in
multiple, compatible ways that involve
different levels of functional complexity and
different scales of time (molecular, cellular,
developmental, organismal,
environmental/ecological, evolutionary)
“Trade-Offs” Competency with Learning Objectives
AP4. Analyze the dynamic trade-offs in function and efficiency seen in interacting
physiological systems.
 Explain the checks and balances in resource allocation between somatic growth
and reproduction.
 Explain, using examples, the reasons that physiological systems may be locally,
but not 100 %, optimized.
 Explain the reason that you would not expect a gill breathing animal to be
completely (or fully) homeothermic.
 Explain how the competing needs to exchange gases and retain water are met in
terrestrial animals.
This group would appreciate your feedback!
1. Have we missed major concepts/competencies that faculty
expect from their comparative physiology students?
2. How can we use these concepts/competencies to promote
the teaching and learning of comparative physiology?
Please send comments/suggestions to :
Kerry Hull ([email protected])
APS ITL - Animal Physiology Group
Beth Beason-Abmayr (Rice University)
Patricia Halpin (Univ. New Hampshire)
Jason Blank (Cal Poly San Luis Obispo)
Kerry Hull (Bishop’s University)
Sydella Blatch (Stevenson University)
Patricia Schulte (Univ. British Columbia)
Bill Cliff (Niagara University)
Alice Villalobos (Texas A&M)
How can we assess Core Concepts?
Step 2 in Backwards Design: Determine
acceptable evidence (design conceptual
assessment)
 What are Concept Inventories?
 How are Concept Inventories used?
 How are Core Inventories developed?
 How are Core Inventories validated and
assessed?
What are Concept Inventories?
 A set of questions (inventory) designed “to probe student
understanding” of fundamental concepts.
 Reliable and validated through statistical analysis.
Concept inventories can be used to
 assess student understanding and application of concepts.
 reveal common, persistent misconceptions that interfere
with progression to expert-level understanding.
 provide formative assessment during teaching & learning.
 assess conceptual learning gains in a course when used as a
pre-test and a post-test.
Bailey, EB 2011; Smith and Tanner 2010
Concept Inventory Development
1. Identify core concepts. (faculty)
2. Develop a conceptual framework for each core concept.
(faculty)
3. Understand student thinking & identify “misconceptions”.
(students & faculty)
4. Create open-ended questions. (student responses)
5. Create “multiple-choice” questions to assess student
thinking. (faculty)
6. Validate questions. (student interviews & faculty surveys)
7. Administer to classes (students) and do statistics (classical
test theory and item response theory).
Adams & Weiman, 2011
Conceptual Assessment for Undergraduate Physiology:
a Homeostasis Concept Inventory
1. Developed a conceptual framework for homeostasis
(“unpack”) and validate this framework with faculty. (paper in
progress)
2. Identified common student “misconceptions”. (paper in
progress)
3. Created 20 multiple-choice questions (MCQs)
 Created multiple-choice questions (MCQs),
 based on the conceptual framework
 used common misconceptions as distractors
4. Validated questions (MCQs)
5. Administer to classes to and analyze data to validate the
concept inventory as concept inventory. (2014)
Michael, McFarland et al. EB abstracts, 2012, 2013, 2014
Developing a Homeostasis Concept Inventory
1. conceptual framework: importance & difficulty
 Faculty agree that some elements in the framework are more
important for students to understand.
 Some elements in the framework are more difficult for students to
understand.
2. student “misconceptions”: some are “sticky”
 Some misconceptions are easy to “correct” or displace with expertlevel understanding.
 Some misconceptions “stickier”, they are more likely to persist after
instruction exposure to expert-level understanding.
Homeostasis: Conceptual Framework
I.
The organism maintains a relatively stable internal
environment in the face of fluctuating external environment.
II.
A substantial change to a regulated variable (a perturbation)
will result in a physiological response to restore it toward to
its normal range.
III. Homeostatic processes require a sensor inside the body
(“what can’t be measured can’t be regulated”)
IV. Homeostatic processes require a control center (which
includes an integrator).
V. Homeostatic processes require target organs or tissues, i.e.
“effectors”.
Conceptual Framework: sensors
III. Homeostatic processes require a sensor inside the body
(“what can’t be measured can’t be regulated”)
i.
Sensors detect the regulated variable and respond by transducing that
stimulus into a different signal.
ii.
Sensors respond within a limited range of stimulus values.
iii.
Sensors generate an output whose value is proportional to the
magnitude of the input to the sensor (i.e. the stimulus).
iv.
Sensors are constantly active (not just active when the regulated variable
is not at the set point value).
v.
An organ system may employ a variety of types of sensors (e.g.
chemoreceptors, baroreceptors, mechanoreceptors, etc) to regulate
variables associated with that organ system.
Framework: Difficulty & Importance
 Physiology faculty assessed the difficulty and importance of
elements of the framework.
 How should we focus our instructional activities?
Faculty Perception of Difficulty vs.
Importance
Difficulty
Easier 3.5
II.A.
The regulated variable is
held stable by a negative
feedback system
3.0
Homeostatic processes
require a sensor inside the
body
III.
2.5
Harder
3.5
Less Important
4.0
4.5
Importance
5.0
Essential
Negative Feedback Question
“The regulated variable is held stable by a negative feedback system.”
This idea was ranked essential (5/5) and
easier to understand (3.29/5).
The following question addresses this important component of the framework
(IIA) and includes student misconceptions as distractors:
In organisms, like humans, negative feedback mechanisms results in
A. an unfavorable, or damaging effect on the body.
B. a constant decrease in the regulated variable.
C. equilibrium amongst body cells and fluids.
D. maintaining an internal variable within a ‘normal’ range of values.
Are these misconceptions “sticky”?
In organisms, like humans, negative feedback mechanisms results in
Sensor Question
Homeostatic processes require a sensor inside the body …”
This idea was ranked important (4.67/5) and
difficult to understand (2.86/5).
The body has a sensor that measures blood pressure, but does not have a
sensor that can measure heart rate. Which of the following are held more
or less constant even when the internal or external environment changes?
A. heart rate
B. blood pressure
C. Both
D. Neither
Is there a “sticky” misconception?
The body has a sensor that measures blood pressure, but does not have a
sensor that can measure heart rate. Which of the following are held more or
less constant even when the internal or external environment changes?
General Model vs. Application
Our Homeostasis questions are either
“general model” questions with no specific
physiological system mentioned or
“application” questions, situated in a specific
physiological system.
Predict which type students will answer more
accurately.
Think – Pair - Share
General Model vs. Application
4. A homeostatic control mechanism functions to maintain the
concentration of X at a relatively constant level. This mechanism is
functioning
A.
B.
C.
D.
when the concentration of X gets too high. [0.8%]
when the concentration of X gets too low. [0.8%]
when the concentration of X gets too high or too low. [46.5%]
at all concentrations of X. [51%]
9. Baroreceptors detect blood pressure. Blood pressure is maintained
relatively constant even when the internal or external environment
changes. Under what conditions do the baroreceptors send signals to
the brain?
A.
B.
C.
D.
when blood pressure is not at its normal value. [34.2%]
when blood pressure is increasing. [5.3%]
when blood pressure is constant. [1.2%]
at all levels of blood pressure. [58%]
General Model vs. Application
Students performed better on the general model questions (74.3% correct)
than on the application questions (58% correct).
100
90
80
Question # 1-20
(x-axis) and % correct 70
(y-axis) from 244
60
undergraduate
50
student responses.
40
30
20
10
0
General Model and Application Questions
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Homeostasis Concept Inventory
for Undergraduate Physiology
1. We are writing up a homeostasis conceptual framework for
homeostasis paper now. (paper in progress, to be submitted this
fall)
2. We are in the first drafts of a homeostasis “misconceptions”
paper. (paper in progress)
3. We are doing item analysis of the data from students from ~
6 pilot institutions to validate the questions.
4. In 2014-2015 we are administering the current draft to
students in broad range of courses to validate the concept
inventory as a whole. .
Michael, McFarland et al. EB abstracts, 2012, 2013, 2014
Thank you!
• Supported by NSF grant DUE-1043443
• Collaborators
 Joel Michael, Rush Medical College, Chicago IL
 Mary Pat Wenderoth, University of Washington, Seattle WA
 Bill (William) Cliff, Niagara University, Niagara NY
 Harold Modell, Physiology Education Research Consortium (PERC),
Seattle WA
(http://physiologyeducation.org)
 Ann Wright, Canisius College, Buffalo NY
 more than 200 physiology faculty who have responded to surveys
&/or responded to our data-gathering questions at workshops.
References
•
Adams, W.K. and Wieman, C.E. 2011. Development and validation of instruments to measure
learning of expert-like thinking. International Journal of Science Education 33:1289-1312.
•
American Association for the Advancement of Science (AAAS). 2011. Vision and Change in
Undergraduate Biology Education: A Call to Action, Washington, DC: American Association for the
Advancement of Science.
•
Association of American Medical Colleges (AAMC). 2009. Scientific Foundations for Future
Physicians. Washington, DC: AAMC. http://services.aamc.org/publications/
•
Bailey,C. 2011. Department of Biochemistry, University of Nebraska, Lincoln, Concept Inventories,
presentation at EB session "Promoting Concept Driven Teaching Strategies in BMB through Concept
Assessments"
•
Brownell, S.E., Freeman, S., Wenderoth, M.P., and Crowe, A.J. 2013. BioCore Guide: A tool for
interpreting the core concepts of vision and change for biology majors. CBE–Life Sciences
Education. 13:200-211.
•
D’Avanzo, C. 2008. Biology concept inventories: overview, status, and next steps. Bioscience
58:1079-1085.
•
Dirks, C., Wenderoth, M.P. and Withers, M. 2014. Assessment in the College Science Classroom.
New York NY: WH Freeman
References – continued
•
•
Fisher, K.M and Williams, K.S. Concept Inventories/Conceptual Assessments in Biology (CABs): An
annotated list. 2012
http://www.sci.sdsu.edu/CRMSE/files/Concept_Inventories_in_Biology_20110325.pdf
Hestenes, D., M. Wells, and G. Swackhamer. 1992. Force concept inventory. Physics Teacher
30(3):141–158.
•
Michael, J. (2007). Conceptual Assessment in the Biological Sciences: a National Science
Foundation-sponsored workshop. Advances in Physiology Education, 31: 389-391
•
Michael, J. and McFarland, J. (2011. The core principles (“big ideas”) of physiology: results of
faculty surveys. Advances in Physiology Education. 25:336-341.
•
Michael, J., Modell, H., McFarland, J., and Cliff, W. (2009). The “core principles” of physiology: what
should students understand? Advances in Physiology Education, 33: 10-
•
Smith A.C., 2008, Department of Cell Biology and Molecular Genetics University of Maryland, ASMCUE. HPI Concept Inventory
•
Smith, J.I. and Tanner, K. 2010. The problem of revealing how students think: concept inventories
and beyond. CBE–Life Sciences Education. 9:1-5.
•
Wiggins, G. and McTighe, J. 1998. Understanding by Design. Alexandria, VA: Association for
Supervision and Curriculum Development.
Disclosure: I am a PULSE leadership fellow
& PULSE is dedicated to the implementation of V&C