Transcript Thinking and Acting Like a Scientist

Thinking and Acting Like a Scientist:
Investigating the Outcomes of Introductory
Science and Math Courses
Kevin Eagan
Jessica Sharkness
Sylvia Hurtado
Higher Education Research Institute, UCLA
Association for Institutional Research 49th Annual Forum
May 30 - June 3, 2009
Atlanta, Georgia
Background

Relatively few students earn degrees in natural
science or engineering in the U.S.

15% of U.S. BA degrees are in science/engineering


Compared to 67% in Singapore, 50% in China, 47% in
France, 38% in South Korea
U.S. needs more undergraduate science majors
to maintain achievement and innovation in
science and engineering

Also need to diversify the scientific workforce and
increase representation of women and minorities
Background

To graduate more bachelor’s degrees in science,
U.S. needs students to choose science majors
and to maintain interest in science majors



National increases in proportion of freshmen indicating
interest in science, technology, engineering and math
(STEM) majors
However, low proportion of students who intend to major
in STEM actually graduate with STEM majors
One obstacle to STEM major completion:
Introductory “gatekeeper” courses


Mechanism for sorting students
What is rewarded?
Introductory “Gatekeeper” Courses


First course in a series of courses in which
knowledge is cumulative
In science – relatively high drop-out and
failure rates in gatekeeper courses




Large lectures
Un-engaging
Highly competitive
Grading on a curve
Classroom Environments &
Instructor Pedagogies

Classroom climates have an impact on
learning and performance


Competitive environments have negative
impact on learning, performance, retention,
self-confidence
Collaborative environments that
emphasize group work can mitigate
negative effects of large lectures and
competitive environments

Can also promote critical thinking about
scientific concepts and their applications
Supportive Learning Environments and
the Skills Needed for Scientific Success

Six necessary conditions for a supportive
learning environment:



Quality of instruction, Teacher’s interest, Social
relatedness, Support of competence, Support
of autonomy
Engender greater self-motivation, encourages
self-directed learning
Two primary pedagogical techniques in
science


Domain-specific learning = memorization of
facts and causal relationships
Domain-general learning = reasoning
strategies and critical thinking skills
Additional influences on student success
in STEM courses

Experiences external to the classroom
environment



Participation in research projects
Peer Tutoring
Prior academic achievement and
preparation

Most significant influence on outcome of
introductory courses?
Goals of Current Study


Untangle effect of prior preparation and
background factors from performance
assessment in introductory STEM courses
Identify how students develop in
introductory courses the critical thinking
dispositions necessary for science
careers, and whether these dispositions
are reflected in student grades
Conceptual model
High School Science
Achievement
Tutoring & Research
Participation
Demographic
variables
Course
Grade
Amount of student
effort expended on
course
Course Learning
Environment &
Pedagogy
Ability to act and
think like a scientist
(Pre-Test)
Critical thinking
dispositions
Ability to act
and think like a
scientist
(Post-test)
Data & Sample



Data collected via online survey from
students in 12 introductory science and
math courses at 5 institutions
Two surveys – one at beginning of course
(pre-survey) and one at end (post-survey);
final analytic sample = 255
Final longitudinal sample:




70% female
34% White, 43% Asian, 8% Black, 13% Latino
86% majoring in STEM field
64% first-year students, 27% second-year
Thinking and Acting like a Scientist
Factor
Items*
Thinking like a scientist pre-test
See connections between different areas of science & math
Understand scientific concepts
Identify what is known and not known in a problem
Ask relevant questions
Draw a picture to represent a problem or concept
Make predictions based on existing knowledge
Come up with solutions and explain them to others
Investigate alternative solutions to a problem
Understand/translate scientific terminology into non-scientific language
Acting like a scientist pre-test
Relate scientific concepts to real-world problems
Synthesize or comprehend several sources of information
Conduct an experiment
Look up scientific research articles and resources
Memorize large quantities of information
Loadings
Pre-test Post-test
0.78
0.77
0.72
0.72
0.66
0.75
0.71
0.71
0.74
0.73
0.80
0.76
0.72
0.58
0.82
0.81
0.73
0.72
0.77
0.85
0.64
0.60
0.61
0.80
0.78
0.71
0.62
0.53
Measurement model fit statistics: χ2=300.69 (305, N=255), NNFI = 0.98, CFI = 0.98, RMSEA = 0.03,
reliability = 0.82.
*All items were asked as part of a questions stem that read, “Rate your ability in the following areas as it pertains to your
academic learning in the sciences.” Response options were Major Strength (5), Above Average (4), Average (3), Below
Average (2), Major Weakness (1)
Variables

Independent Variables:







Demographic characteristics
Prior preparation
Course pedagogy
Classroom environment
College experiences
Critical thinking dispositions (CCTDI
subscales)
Dependent Variables


Final course grade
Dispositions toward science
Analysis Plan

Identification of latent constructs (factors),
representing acting like a scientist and
thinking like a scientist


Exploratory Factory Analysis  Confirmatory
Factor Analysis (measurement model in SEM)
Structural Equation Modeling (SEM) to
model how student experiences in
introductory courses affect three outcomes
Structural Modeling Procedure


Added hypothesized predictors and paths
to structural equation model
Used prior research, theory and Wald and
LaGrange multiplier tests to add and
remove paths that did not contribute to
model fit

When all paths were deleted from a variable,
variable was removed from the model
Results (non-significant paths not shown)
External Support/
Science Experiences
Student Effort
Consistently got support
needed
HPW Engage in
lab activities
HPW prof’s research
project (college)
Crammed for
exams
Course Learning
Environment
Felt competition in
course
Felt Overwhelmed
Course employed
group activities
Sought tutoring on
campus
Participated in HS
research program
Grade
in intro
course
Demographics
Income
Pretests
URM Student
Avg HS GPA in
STEM courses
Female
AP Chemistry
Score
Tutored
Student in HS
Course
Pedagogy
Format primarily
lecture
Think like a
scientist
Post-test
Act like a
scientist
Post-test
BBS
Major
Act like a
scientist
Pre-test
Think like a
scientist
Pre-test
OpenCritical Thinking
Mindedness
Confidence
Critical thinking dispositions
Analyticity
Discussion of findings: Thinking and
Acting Like a Scientist

CCTDI – Openmindedness negatively predicted both
dispositions


CCTDI – Critical thinking self-confidence positively
predicted both dispositions


Indicative of students’ development of domain-general
science skills
CCTDI – Analyticity positively predicted thinking like
a scientist


Relativism in conflict with objective, empirical perspective of
science
Indicative of students’ ability to think more carefully and
critically during problem-solving activities
Students who felt overwhelmed scored lower on both
dispositions

May be indicative of naïveté about what is required of
science majors
Discussion of findings: Final Course
Grade

Thinking and acting like a scientist and
CCTDI subscales unrelated to final grade


Suggests that introductory courses focus too
much on the acquisition of knowledge rather
than development of higher-order thinking
skills
Final grade in large part predicted by prior
preparation (high school grades, research)

Suggests that failure to earn top grades may
merely lack the preparation necessary for
success in these courses rather than sciencerelated skills
Discussion of findings: Final Course
Grade

Cramming for exams positively predicted course
grades


Group work positively predicted final course grade


Underscores that grades reward rote memorization
rather than development of higher-order thinking skills
Supports prior research that concluded that peer
learning provides learning reinforcement, which may
have longer-term benefits
Tutoring (receiving and providing) positively
predicted final course grade

Reinforces the benefits of peer-to-peer teaching and
learning
Implications and Conclusions



Can we afford to cram content into
courses at the expense of development of
scientific skills and thinking?
Adjust grading practices so that they more
accurately reflect learning rather than prior
preparation
Further examination of pedagogical
practices and interventions in large,
lecture-based gatekeeper courses is
needed
Contact Information
Faculty and Co-PIs:
Sylvia Hurtado
Mitchell Chang
Administrative Staff:
Aaron Pearl
Graduate Research Assistants:
Kevin Eagan
Jessica Sharkness
Lorelle Espinsoa
Minh Tran
Christopher Newman
Paolo Velasco
Papers and reports are available for download
from project website:
http://heri.ucla.edu/nih
Project e-mail: [email protected]
Acknowledgments: This study was made possible by the support of the National Institute of
General Medical Sciences, NIH Grant Numbers 1 R01 GMO71968-01 and R01
GMO71968-05 as well as the National Science Foundation, NSF Grant Number 0757076.
This independent research and the views expressed here do not indicate endorsement by
the sponsors.