Transcript Research on Undergraduate Learning in STEM Disciplines Karl A. Smith

Research on Undergraduate
Learning in STEM Disciplines
Karl A. Smith
Civil Engineering
University of Minnesota
[email protected]
www.ce.umn.edu/~smith
National Research Council
National Science Resources Center
Math/Science Partnerships Workshop
December 5-7, 2004
Backdrop
National Research Council Reports:
1. How People Learn: Brain, Mind, Experience,
and School (1999).
2. How People Learn: Bridging Research and
Practice (2000).
3. Knowing What Students Know: The Science
and Design of Educational Assessment
(2001).
4. The Knowledge Economy and
Postsecondary Education (2002). Chapter 6
– Creating High-Quality Learning
Environments: Guidelines from Research on
How People Learn
Session Highlights
• Provide overview of some findings from reports
related to teaching & learning.
• Do a activities with you to illustrate some of the
points covered in the reports.
• Discuss implications for designing learning
environments that are learner centered,
knowledge centered, assessment centered, and
community centered.
Designing Learning
Environments Based on HPL
Learner-Centered Learning
Environments
• Learners use their current knowledge
to construct new knowledge. Effective
instruction must take into account
what learners bring to the classroom.
Active engagement in learning
supports the construction of
knowledge.
Learner-Centered Learning
Environments
• Learners should be assisted in developing
metacognitive strategies.
"Metacognition refers to people's abilities to
predict their performances on various tasks ...
and to monitor their current levels of mastery
and understanding" (HPL, p. 12)
Transfer can be improved by helping students
become more aware of themselves as
learners who actively monitor their learning
and performance strategies.
Learner-Centered Learning
Environments
• Learners learn more efficiently and
effectively when they are provided
with feedback to help them monitor
progress. Students need to be given
opportunities to practice skilled
problem solving and provided with
both, feedback to monitor progress,
and support to ensure progress.
Knowledge-Centered Learning
Environments
• Students are not blank slates, so instruction should
begin with students' current knowledge and skills.
• Instruction should help students organize
knowledge in ways that are efficient for recall and
for application in solving problems.
• Instruction should focus on helping students gain
deep understanding of the major concepts and
principles, rather than the acquisition of
disconnected facts and skills.
Assessment-Centered Learning
Environments
• Formative assessment (assessment done during
the course of instruction to monitor students'
progress and to help shape instruction) is pivotal
for providing feedback to students so that they
can revise and improve the quality of their
thinking, and should be done continuously as a
part of instruction.
• Formative assessment strategies should be
developed that make students' thinking visible to
the instructor, to the learner, and to other
classmates.
Assessment-Centered Learning
Environments
• Summative assessments (assessment done at
the end of instruction for such purposes as
assigning grades or evaluating competence)
should reflect the knowledge, concepts,
principles, and problem solving & lab skills of the
discipline that are considered crucial by experts.
• Students should learn how to assess their own
work and that of peers.
Community-Centered Learning
Environments
• Learners are embedded in social contexts. To make
effective use of their “prior knowledge,” they need to
relate the origins of their learning to school-based
concepts.
• It is important to help students see the relevance of
their school-based learning to non-school contexts and
problem solving. (Students' "awake" time: 14% in
school, 53% out of school.
• Communities of practice need to be encouraged.
How? Internships, class participation, dorm floors
arranged by major, etc.
Summary Points
• There is an emerging science of learning
• It has major implications for all aspects of
schooling -- curriculum, instruction, assessment,
plus preservice and inservice teacher education
• It provides a basis for knowing when, how and
why to use various instructional strategies
• It can guide the intelligent design and use of new
curricular materials as well as information
technologies
Lila M. Smith
Pedago-pathologies B Lee Shulman
Amnesia
Fantasia
Inertia
Shulman, Lee S. 1999. Taking learning seriously.
Change, 31 (4), 11-17.
What do we do about these pathologies?
– Lee Shulman
Activity
Reflection
Collaboration
Passion
Combined with generative content and
the creation of powerful learning
communities
Shulman, Lee S. 1999. Taking learning seriously.
Change, 31 (4), 11-17.
Lila M. Smith
Tracking Change - Seymour
"The greatest single challenge to SMET
pedagogical reform remains the problem
of whether and how large classes can be
infused with more active and interactive
learning methods."
Seymour, Elaine. 2001. Tracking the processes of change in US
undergraduate education in science, mathematics, engineering, and
technology. Science Education, 86, 79-105.
Formulate-Share-Listen-Create
(Think-Pair-Share)
• Individually read the quote “To teach is to
engage students in learning. . .”
• Underline/Highlight words and/or phrase
that stand out for you
• Turn to the person next to you, introduce
yourself
• Share words and/or phrases that stood out
and discuss
To teach is to engage students in learning; thus
teaching consists of getting students involved in the
active construction of knowledge. . .The aim of
teaching is not only to transmit information, but also to
transform students from passive recipients of other
people's knowledge into active constructors of their
own and others' knowledge. . .Teaching is
fundamentally about creating the pedagogical, social,
and ethical conditions under which students agree to
take charge of their own learning, individually and
collectively
Education for judgment: The artistry of discussion leadership. Edited by C. Roland Christensen,
David A. Garvin, and Ann Sweet. Cambridge, MA: Harvard Business School, 1991.
Strategies for
Energizing Large
Classes: From Small
Groups to
Learning Communities:
Jean MacGregor,
James Cooper,
Karl Smith,
Pamela Robinson
New Directions for
Teaching and Learning,
No. 81, 2000.
Jossey- Bass
Book Ends on a Class Session
Informal CL (Book Ends on a Class Session) with Concept Tests
Physics
Peer Instruction
Eric Mazur - Harvard B http://galileo.harvard.edu
Peer Instruction – www.prenhall.com
Richard Hake – http://www.physics.indiana.edu/~hake/
Chemistry
Chemistry ConcepTests - UW Madison B
www.chem.wisc.edu/~concept
Video: Making Lectures Interactive with ConcepTests
ModularChem Consortium B http://mc2.cchem.berkeley.edu/
STEMTEC
Video: How Change Happens: Breaking the ATeach as You Were Taught@
Cycle B Films for the Humanities & Sciences B www.films.com
Thinking Together video: Derek Bok Center B
www.fas.harvard.edu/~bok_cen/
Richard Hake (Interactive engagement vs traditional methods)
http://www.physics.indiana.edu/~hake/
Traditional
(lecture)
Interactive
(active/cooperative)
<g> = Concept Inventory Gain/Total
The “Hake” Plot of FCI
35.00
SDI
30.00
X
ALS
UMn-CL+PS
WP
25.00
20.00
UMn Cooperative Groups
15.00
X
PI(HU)
UMn Traditional
ASU(nc)
10.00
WP*
ASU(c)
HU
5.00
0.00
20.00
30.00
40.00
50.00
Pretest (Percent)
60.00
70.00
80.00
Physics (Mechanics) Concepts:
The Force Concept Inventory (FCI)
• A 30 item multiple choice test to probe student's
understanding of basic concepts in mechanics.
• The choice of topics is based on careful thought
about what the fundamental issues and
concepts are in Newtonian dynamics.
• Uses common speech rather than cueing
specific physics principles.
• The distractors (wrong answers) are
based on students' common inferences.
FCI Question
17
An elevator is being lifted up an elevator shaft at a
constant speed by a steel cable, as shown in the
figure. All frictional effects are negligible. In this
situation, forces on the elevator are such that:
(A) the upward force by the cable is greater than
the downward force of gravity.
(B) the upward force by the cable is equal to
the downward force of gravity.
(C) the upward force by the cable is smaller than
the down ward force of gravity.
(D) the upward force by the cable is greater than
the sum of the downward force of gravity and a
downward force due to the air.
(E) None of the above. (The elevator goes up because the
cable is shortened, not because an upward force is
exerted on the elevator by the cable).
Pre
64
Post
36
18
60
2
0
11
2
5
1
Problem Based Cooperative Learning Format
TASK: Solve the problem(s) or Complete the project.
INDIVIDUAL: Estimate answer. Note strategy.
COOPERATIVE: One set of answers from the group, strive for agreement,
make sure everyone is able to explain the strategies used to solve each
problem.
EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain
the strategies used to solve each problem.
EVALUATION: Best answer within available resources or constraints.
INDIVIDUAL ACCOUNTABILITY: One member from your group may be
randomly chosen to explain (a) the answer and (b) how to solve each
problem.
EXPECTED BEHAVIORS: Active participating, checking, encouraging, and
elaborating by all members.
INTERGROUP COOPERATION: Whenever it is helpful, check procedures,
answers, and strategies with another group.
Technical Estimation Exercise
TASK:
INDIVIDUAL: Quick Estimate (10 seconds). Note strategy.
COOPERATIVE: Improved Estimate (~5 minutes). One set of answers from
the group, strive for agreement, make sure everyone is able to explain the
strategies used to arrive at the improved estimate.
EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain
the strategies used to arrive at your improved estimate.
EVALUATION: Best answer within available resources or constraints.
INDIVIDUAL ACCOUNTABILITY: One member from your group may be
randomly chosen to explain (a) your estimate and (b) how you arrived at it.
EXPECTED BEHAVIORS: Active participating, checking, encouraging, and
elaborating by all members.
INTERGROUP COOPERATION: Whenever it is helpful, check procedures,
answers, and strategies with another group.
Model 1 (lower bound)
let L be the length of the room,
let W be its width,
let H be its height,
and let D be the diameter of a ping pong ball.
Then the volume of the room is
Vroom = L * W * H,
and the volume of a ball (treating it as a cube) is
Vball = D3,
so number of balls = (Vroom) / (Vball) = (L * W * H) / (D3).
Model 2 (upper bound)
let L be the length of the room,
let W be its width,
let H be its height,
and let D be the diameter of a ping pong ball.
Then the volume of the room is
Vroom = L * W * H,
and the volume of a ball (treating it as a sphere) is
Vball = 4/3 Br3,
so number of balls = (Vroom) / (Vball) = (L * W * H) / (4/3 Br3).
Model 1 (Vroom / D3ball) B Lower Bound
Model 2 (Vroom / (4/3 Br3ball)) B Upper Bound
Upper Bound/Lower Bound = 6/B . 2
How does this ratio compare with
1.The estimation of the diameter of the ball?
2.The estimation of the dimensions of the
room?
Model World
Real World
Model
Vr/Vb
Calc
Problem-Based Learning
START
Apply it
Problem posed
Normative Professional
Curriculum:
Learn it
Identify what we
need to know
1. Teach the relevant basic
science,
Subject-Based Learning
START
Given problem to
illustrate how to use
it
Learn it
Told what we
need to know
2. Teach the relevant
applied science, and
3. Allow for a practicum to
connect the science to
actual practice.
Problem-Based Learning (PBL)
Problem-based learning is the learning that results from the process
of working toward the understanding or resolution of a problem. The
problem is encountered first in the learning process B Barrows and
Tamlyn, 1980
Core Features of PBL
•Learning is student-centered
•Learning occurs in small student groups
•Teachers are facilitators or guides
•Problems are the organizing focus and stimulus for learning
•Problems are the vehicle for the development of clinical problemsolving skills
•New information is acquired through self-directed learning
Group Processing
B Plus/Delta Format B
Plus
Things That Group Did Well
Delta
Things Group Could Improve
Cooperative Learning is instruction that involves
people working in teams to accomplish a common
goal, under conditions that involve both positive
interdependence (all members must cooperate to
complete the task) and individual and group
accountability (each member is accountable for the
complete final outcome).
Key Concepts
Positive Interdependence
Individual and Group Accountability
Face-to-Face Promotive Interaction
Teamwork Skills
Group Processing
Cooperative Learning Research Support
Johnson, D.W., Johnson, R.T., & Smith, K.A. 1998. Cooperative learning returns to
college: What evidence is there that it works? Change, 30 (4), 26-35.
• Over 300 Experimental Studies
• First study conducted in 1924
• High Generalizability
• Multiple Outcomes
Outcomes
1. Achievement and retention
2. Critical thinking and higher-level
reasoning
3. Differentiated views of others
4. Accurate understanding of others'
perspectives
5. Liking for classmates and teacher
6. Liking for subject areas
7. Teamwork skills
Small-Group Learning: Meta-analysis
Springer, L., Stanne, M. E., & Donovan, S. 1999. Effects of small-group learning
on undergraduates in science, mathematics, engineering, and technology: A metaanalysis. Review of Educational Research, 69(1), 21-52.
Small-group (predominantly cooperative) learning in
postsecondary science, mathematics, engineering, and
technology (SMET). 383 reports from 1980 or later, 39 of
which met the rigorous inclusion criteria for meta-analysis.
The main effect of small-group learning on achievement,
persistence, and attitudes among undergraduates in
SMET was significant and positive. Mean effect sizes for
achievement, persistence, and attitudes were 0.51, 0.46,
and 0.55, respectively.
Creating High-Quality Learning Environments:
Guidelines from Research on How People Learn
Understanding by Design
Wiggins & McTighe
Backward Design
Stage 1.Identify Desired Results
Stage 2.Determine Acceptable Evidence
Stage 3.Plan Learning Experiences and Instruction
Wiggins, G. & McTighe, J. 1998. Understanding by design. ASCD.
Backward Design
Stage 1. Identify Desired Results
Filter 1. To what extent does the idea, topic, or
process represent a big idea or having
enduring value beyond the classroom?
Filter 2. To what extent does the idea, topic, or
process reside at the heart of the discipline?
Filter 3. To what extent does the idea, topic, or
process require uncoverage?
Filter 4. To what extent does the idea, topic, or
process offer potential for engaging
students?
Backward Design
Stage 2. Determine Acceptable Evidence
Types of Assessment
Quiz and Test Items:
Simple, content-focused test items
Academic Prompts:
Open-ended questions or problems that
require the student to think critically
Performance Tasks or Projects:
Complex challenges that mirror the issues or
problems faced by graduates, they are authentic
Backward Design
Stage 3. Plan Learning Experiences & Instruction
• What enabling knowledge (facts, concepts, and
principles) and skills (procedures) will students need to
perform effectively and achieve desired results?
• What activities will equip students with the needed
knowledge and skills?
• What will need to be taught and coached, and how
should it be taught, in light of performance goals?
• What materials and resources are best suited to
accomplish these goals?
• Is the overall design coherent and effective?
It could well be that faculty members of the
twenty-first century college or university will
find it necessary to set aside their roles as
teachers and instead become designers of
learning experiences, processes, and
environments James Duderstadt, 1999
We never educate directly, but indirectly by
means of the environment. Whether we permit
chance environments to do the work, or
whether we design environments for the
purpose makes a great difference.
John Dewey, 1906
CAEE Vision for Engineering
Education
Center for the Advancement
of Engineering Education
Cindy Atman, Director
CAEE Team
University of Washington
Colorado School of Mines
Howard University
Stanford University
University of Minnesota
CAEE Affiliate Organizations
City College of New York (CCNY), Edmonds Community College, Highline
Community College (HCC), National Action Council for Minorities in
Engineering (NACME), North Carolina A&T (NCA&T), San Jose State
University (SJSU), University of Texas, El Paso (UTEP), Women in
Engineering Programs & Advocates Network (WEPAN) and Xavier
University
CAEE - Elements for Success
• Scholarship on Learning Engineering
Learn about the engineering student experience
• Scholarship on Engineering Teaching
Help faculty improve student learning
• Scholarship on Engineering Education
Institutes
Cultivate future leaders in engineering
education
CAEE Approach
Theory
Research that makes a
difference . . . in theory
and practice
Research
Practice
Center for the Integration of
Research, Teaching, and Learning
(CIRTL)
NSF Center for Learning and Teaching
University of Wisconsin - Madison
Michigan State University
Pennsylvania State University
…develop a national STEM faculty ...
UNDERGRADS
FACULTY
Community College
Liberal Arts
HBCU
Masters University
Comprehensive Univ.
Research University
Community College
Liberal Arts
HBCU
Masters University
Comprehensive Univ.
Research University
Research Universities
100 RUs => 80% Ph.D’s
Teaching-as-Research
“The nation must develop STEM faculties who themselves
continuously inquire into their students’ learning.”
• Engagement in teaching as engagement in STEM research
• Hypothesize, experiment, observe, analyze, improve
• Aligns with skills and inclinations of graduatesthrough-faculty, and fosters engagement in
teaching reform
• Leads to self-sustained improvement of STEM education
NATIONAL ACADEMY OF ENGINEERING
OF THE NATIONAL ACADEMIES
Center for the Advancement of Scholarship on Engineering Education
A Work-in-Progress:
NAE Center for the Advancement
of Scholarship on
Engineering Education
Norman L. Fortenberry, Sc.D.
Director, CASEE
http://www.nae.edu/CASEE
[email protected]
(202) 334-1926
November 8, 2003
56
NATIONAL ACADEMY OF ENGINEERING
OF THE NATIONAL ACADEMIES
Center for the Advancement of Scholarship on Engineering Education
CASEE Mission
Enable engineering education to meet, in a significantly better
way, the needs of employers, educators, students, and society
at large.
CASEE Objectives
Working collaboratively with key stakeholders, CASEE

Encourages rigorous research on all elements of the
engineering education system, and

Seeks broad dissemination, adoption, and use of research
findings.
57
NATIONAL ACADEMY OF ENGINEERING
OF THE NATIONAL ACADEMIES
Center for the Advancement of Scholarship on Engineering Education
Research Thrust Areas
1. Define the bodies-of-knowledge required for
engineering practice and use of engineering
study for other careers.
2. Develop strategies that value diversity in the
formulation and solution of engineering
problems.
3. Develop cost-effective and time-efficient
strategies and technologies for
•
•
Improving student learning, and
Enhancing the instructional effectiveness of current and
future faculty.
4. Develop assessments of student learning and
instructional effectiveness.
58
Conducting Rigorous Research in
Engineering Education: Creating a
Community of Practice
NSF-CCLI-ND
American Society for Engineering Education
Karl Smith & Ruth Streveler
University of Minnesota &
Colorado School of Mines
Rigorous Research Workshop
 Initial Event for year-long project
 Presenters and evaluators representing
– American Society for Engineering Education (ASEE)
– American Educational Research Association (AERA)
– Professional and Organizational Development Network in Higher
Education (POD)
 Faculty funded by two NSF projects:
– Conducting Rigorous Research in Engineering Education (NSF DUE0341127)
– Strengthening HBCU Engineering Education Research Capacity (NSF
HRDF-041194)
• Council of HBCU Engineering Deans
• Center for the Advancement of Scholarship in Engineering Education
(CASEE)
• National Academy of Engineering (NAE)