Transcript Estimating Correlates of Growth Between Mathematics and

Estimating Correlates of Growth
Between Mathematics and Science
Achievement via a Multivariate
Multilevel Design With Latent
Variables
Lingling Ma and Xin Ma
University of Kentucky
Oct 15, 2005
1
Literature Review
 “Schools make no difference” (Coleman et
al., 1966 and Jencks et al., 1971)
 School effectiveness research
 Schools affect students’ development
 There are observable regularities in the
schools that “add value” .
2
School Effectiveness Research
What would affect student performance?
 Policies and practices of the school
 Background characteristics of students
entering the school
 Social and economical factors
 The overall ability and SES composition of
a school
3
 School Context Effects
– Enrollment
– School location
– Percent of students eligible for federal lunch
assistance
– Percent of minority students
– Teacher-student ratio
 School Climate Effects
– Parental involvement
– Staff cooperation
– Teacher autonomy
– Teacher commitment
4
School Effectiveness Research
(Cont.)
 Single school subject (e.g., mathematics) as
academic outcome (Regression Analysis)
 A single subject across multiple points in time
(longitudinal study)
 The relationship across different academic
outcomes at the same time point (multivariate
analysis)
 The consistency of growth among different
academic outcomes across multiple points in time
5
Research Questions
 Do students (schools) have consistent rates
of growth between mathematics and science
achievement? That is, if students (schools)
grow faster in one area, do they also grow
faster in the other area?
 How do student and school characteristics
affect the correlation in rates of growth
between mathematics and science
achievement?
6
Data
 Longitudinal Study of American Youth
(LSAY)
 Stratified national probability samples of 52
schools
 In each selected school, about 60 students in
the 7th grade were randomly selected
 Students were followed for six years from
Grade 7 to Grade 12
 Math and Science achievement tests and
student questionnaires annually
 Parent, teacher and principal questionnaires
7
Why not PISA or TIMMS?
 PISA: Programme for International Student
Assessment
 TIMMS: Third International Mathematics
and Science Study (Trends in International
Mathematics and Science Study)
 Neither is a same-cohort longitudinal study
 Can’t really study growth in mathematics or
science
8
LSAY Achievement Tests
 The LSAY achievement tests were in
multiple-choice format with items selected
from the National Assessment of
Educational Progress (NAEP).
 To ensure content validity in line with
common mathematics and science curricula
across grades, the LSAY staff carefully
selected items from the NAEP item pool
built according to the NAEP content and
process framework.
9
Measures
 Mathematics content domains
– Numbers and operations; measurement;
relations, functions, and algebraic
expressions; geometry; mathematical
methods; discrete mathematics; and data
organization and interpretation.
 Mathematics cognitive domains
– Recall of skills and knowledge, routine
applications, and problem solving and
understanding.
10
Measures (Cont.)
 Science content domains
– Solid earth, water, air, earth in space, matter
and its transformations, energy and its
transformations, motion, change and
evolution, cells and their functions,
organisms, and ecology
 Science cognitive domains
– Conceptual understanding, scientific
investigation, and practical reasoning
11
Student Characteristics
 Student own characteristics
– Gender
– Age
– Race-ethnicity
 Family characteristics
– Parental socioeconomic status [SES]
– Number of parents
– Number of siblings
12
School Characteristics
 School Context Variables
– Enrollment, school location, percent of students
eligible for federal lunch assistance, percent of
minority students, teaching experience of teachers,
education level of teachers, grade span, teacherstudent ratio, and computer-student ratio
 School Climate Variables
– Academic press, disciplinary climate, parental
involvement, principal leadership, staff cooperation,
teacher autonomy, teacher commitment, general
support for mathematics, and extracurricular
activities
13
Why Hierarchical Linear Model?
 Hierarchical linear model (HLM)
incorporate data from multiple levels in an
attempt to determine the impact of
individual and grouping factors upon some
individual outcome.
 HLM, or multilevel models, can incorporate
factors from multilevel levels since HLMs
take into account error structures at each
level.
14
Level 1
 Latent Approach
 Basic skills, algebra, geometry,
and quantitative literacy were
indicators of the latent true score
for mathematics achievement
with measurement errors
 Biology, physics, and
environmental science were
indicators of the latent true score
for science achievement with
measurement errors
15
Level 2
 Growth Model
 Within each subject, there are a series of
separate linear regression equations that
model students' achievement scores with
their grade levels.
1 jkl  10kl  11kl Grade jkl  u1 jkl
 2 jkl   20kl   21kl Grade jkl  u2 jkl
16
Level 3
 In these equations, student rate of
growth is represented as a school
average rate of growth, contributions of
student characteristics and an error
term unique to each student.
11kl   110l    11 pl X pkl  v11kl
p
 21kl   210l    21 pl X pkl  v21kl
p
17
Level 4
 The fourth level of this multivariate
multilevel model contains two sets of linear
regression equations modeling the average
rates of growth in mathematics and science
achievement with school characteristics.
 110l  1100  110 rWrl  f110l
r
 210l  2100  210 rWrl  f 210l
r
18
Estimation of Parameters
 Iterative Generalized Least Squares (IGLS) or
Restricted Iterative Generalized Least Squares
(RIGLS) are the basis of parameter estimation
methods for multilevel models.
 Usually, it starts with reasonable estimates of
the fixed coefficients from an initial Ordinary
Least Square (OLS) fit.
 Based on the initial estimate, then an improved
estimate would be obtained, and so on,
iteratively, until convergence is achieved.
19
Results
20
 The correlation between the rates of growth in
mathematics and science achievement among
students was not zero but not large, and that
student and school characteristics had little or
no influence on the correlation.
 The correlation between the average rates of
growth in mathematics and science
achievement was rather strong among schools
(indicating consistency between the average
rates of growth in mathematics and science
achievement among schools), and that this
consistency was influenced by student and
school characteristics.
21
Discussion
 Learning difficulties in mathematics and
science are indeed likely to occur
simultaneously.
 Improvement efforts of teachers in one subject
without knowing students’ learning problems
in the other subject may not work at all.
 For example, if students do not learn
mathematics well, they are unable to apply
mathematics to problem solving in science
(e.g., physics).
22
Discussion (Cont.)
Major policy implication:
– Educators should be encouraged to help students
achieve their full academic potential in both
mathematics and science
– A call for educational policies that promote and reward
regular interaction and collaboration between math and
science departments or teachers
– A less dramatic strategy is to group teachers according
to grade levels rather than academic departments so as
for teachers to coordinate their curricular and
instructional efforts and develop coordinated remedial
programs for students with learning difficulties in
mathematics and science.
23
Discussion (Cont.)
 We suggest that perhaps it is school curriculum
and instruction rather than school context and
climate that promote balanced learning in
mathematics and science.
 For example, inquiry-based curriculum and
instruction that emphasize hands-on experiments
and discoveries may intertwine many aspects of
mathematics and science education (e.g., scientific
experiment and data analysis) to promote learning
in both mathematics and science simultaneously.
24