Transcript Jeffeory G. Hynes - Arizona State University

STUDENT AIMS PERFORMANCE
IN A PREDOMINANTLY
HISPANIC DISTRICT
Lance Chebultz
Arizona State University
2012
Introduction
The last decade of K-12 education has evolved
to accommodate federal and state policies on
accountability through measuring student
achievement with standardized tests.
Performance standards have been
established for all students, with
stratification on certain variables, e.g.
language proficiency.
Literature Review
• Last decade has seen an increase in the
importance of accountability, both nationally
and at the state level
• No Child Left Behind (2002)
• AZ LEARNS (2001)
• Proposition 203 (2000)
• State level assessments instituted
• AIMS
• AZELLA
Literature Review
• Arizona is among outlying states for numbers
of ELL students served in education system
• In 2004
•
Arizona; 155,789
•
California; 1,591,525
•
Florida; 299,346
•
Illinois; 192,764
•
New York; 203,283
•
Texas; 684,007
•
Puerto Rico; 578,534
Literature Review
• Arizona is among outlying states for numbers
of ELL students served in education system
• In 2004 - Arizona; 155,789
• ELL population steadily growing, particularly
in states with previously low numbers.
• Reports of persistent gaps in ELL and nonELL performance
•
Genesee, F., Lindholm-Leary, K., Saunders, W., & Christian, D.
(2005)
Literature Review
• Two primary factors likely to influence ELL
assessment
• Curriculum/Instruction
• Could investigate these two individually
• Not investigated in the current study
• Wright (2005) – Discusses approaches of
institutions with ELL programs and/or ESL
accommodations.
• Assessments
Literature Review
• Research has focused on three major factors
likely to influence ELL assessment
• Curriculum/Instruction
• Assessments
• Huempfner (2004) – based on assumption
that same assessment is valid for bilingual
and English speaking students
• Valenzuela (2005) - High stakes testing not
appropriate for ELL students
Purpose of the Study
The purpose of this study was to
investigate student performance
on the AIMS assessment for ELL
and non-ELL students. The study
was designed to determine student
performance differences on the
AIMS Math and Reading
assessments between ELL and nonELL students across grade and
time.
Research Questions
1. Are there significant differences in AIMS
performance (Reading and Math) for students
across the levels of ELL status?
Research Questions
1. Are there significant differences in AIMS
performance (Reading and Math) for
students across the levels of ELL status?
2. Does longitudinal performance on the AIMS
exam differ significantly for students
classified as proficient compared to
students classified as ELL?
Sample
• From a large Arizona K-8 school district
– District population – approximately 5000
students/year
• Predominantly Hispanic (over 99% in current
sample)
• 96% Second language learners at admission
• 9 Schools; one pre, six K-6th, and two 7-8th
• All Title 1 – approximately 100% students on
free/reduced lunch
Sample
• From a large Arizona K-8 school district
• Sample consisted of:
–
–
–
–
3rd, 4th, 5th, and 6th graders
Data from years 2008, 2009, and 2010
Must have completed Math or Reading AIMS
All Hispanic (Less than 1 % of available sample was
non-Hispanic, therefore dropped from analyses)
– 90% on free-reduced lunch at the time of the study
– Gender approximately 50/50 male/female, no gender
comparisons done in this study
Research Design
• Ex Post Facto
– Uses existing data
– Pre-formed groups (e.g. ELL vs. non-ELL,
grades, etc…)
– Compare performance on assessments for
students across time and grade
– Used Analyses of Variance to answer each
research question.
• Multi-factor Between-Subject design (Q. 1)
• Mixed ANOVA, between by within-factor
design (Q.2)
Demographic Information
Number of students by year, grade, and ELL Status from District X
Demographic Information
2008
2009
2010
ELL
Non-ELL
ELL
Non-ELL
ELL
Non-ELL
3rd Grade
74.4%
25.6%
68.0%
32.0%
64.4%
35.6%
4th Grade
69.3%
30.7%
66.7%
33.3%
61.4%
38.6%
5th Grade
56.3%
43.7%
54.9%
45.1%
46.2%
53.8%
6th Grade
54.1%
45.9%
49.9%
50.1%
33.3%
66.7%
Percentage of students by year, grade, and ELL Status from District X
Demographic Information
• Sub-Sample for Research Question 2
– Assessment data needed to be complete
for all three years
– Group of 765 students provided
assessment data for all three years
• 375 3rd grades, 390 4th graders
• 71.8% classified as ELL in 2008
• 51.9% Female, 48.0% Male
Research Question 1
Are there significant differences in AIMS performance
(Reading and Math) for students across the levels of
ELL status?
• Used a series of Between-subject ANOVAs
–Between-subject factors: Grade and ELL status
–Separate analyses for Math and Reading
–Separate analyses for each year of data, 2008, 2009, and
2010
–Outcome is category of performance on assessment
•FFB
•A
•M
•E
Research Question 1
• Math Assessment
Year
2008
Perf
Grade 3
Grade 4
Grade 5
Grade 6
N
%
N
%
N
%
N
%
FFB
78
17.7
78
17.7
105
21.0
68
14.2
A
117
26.5
92
20.9
129
25.7
86
18.0
M
207
46.9
224
50.9
225
44.9
265
55.3
E
38
8.6
45
10.2
42
8.4
59
12.3
Research Question 1
• Reading Assessment
Year
2008
Perf
Grade 3
Grade 4
Grade 5
Grade 6
N
%
N
%
N
%
N
%
FFB
66
15.0
66
15.0
118
23.6
56
11.7
A
160
36.3
155
35.2
175
34.9
170
35.5
M
203
46.0
209
47.5
202
40.3
247
51.6
E
11
2.5
9
2.0
6
1.2
5
1.0
Comparison Across Subject
Year
2008
Subject
Math
Reading
P/F
Grade 3
Grade 4 Grade 5 Grade 6
%
%
%
%
F
44.5
38.9
46.7
32.4
P
55.5
61.1
53.3
67.6
F
51.5
50.5
58.5
47.6
P
48.5
49.5
41.5
52.6
Research Question 1
•
Findings were consistent across year of data,
therefore will present 2008 information as an
example
•
Math analyses:
–
No interactions between grade and ELL status
–
Significant main effect of ELL status and grade
–
Large effect of ELL status
•
Reading Analyses:
–
Significant interactions between grade and ELL status
–
Some very minor difference between year, overall result that
grade 5 performance different for ELL and non-ELL students
relative to other grades.
–
Large effect sizes and differences between ELL and non-ELL
students
•
Research Question 1
2008 – Math Assessment Data
Grade 3
Grade 4
Grade 5
Grade 6
ELL Status
Mean
SD
Non-ELL
3.07
.69
ELL
2.25
.85
Non-ELL
2.99
.73
ELL
2.33
.91
Non-ELL
2.94
.67
ELL
1.99
.86
Non-ELL
3.14
.59
ELL
2.24
.88
•
Research Question 1
2008 – Reading Assessment Data
Grade 3
Grade 4
Grade 5
Grade 6
ELL Status
Mean
SD
Non-ELL
2.92
.55
ELL
2.16
.74
Non-ELL
2.89
.54
ELL
2.13
.74
Non-ELL
2.75
.55
ELL
1.76
.70
Non-ELL
2.88
.39
ELL
2.02
.69
•
Research Question 1
Effect Size Data – Math and Reading
Effect Size - Math
Grade
Effect Size- Reading
ELL Status Grade ELL Status ELL Status X
Grade
2008
.012
.192
.027
.272
.005
2009
.005
.199
.007
.245
.004
2010
.020
.237
.020
.299
.004
Conclusion for Research
Question 1
• Overall better performance for non-ELL students
than ELL students
– Largest Effect Size for ELL status – 20-24% of variance
– Consistent across Math and Reading
– Consistent across year of data
• Performance on Math better than performance on
Reading
• Transitioning students may attenuate the non-ELL
student performance at higher grades, i.e. all
students are improving at a higher rate, but time
since transitioning to proficient may be better
index than grade
Research Question 2
Does longitudinal performance on the AIMS exam
differ significantly for students classified as
proficient compared to students classified as ELL?
Research Question 2
Does longitudinal performance on the AIMS exam
differ significantly for students classified as
proficient compared to students classified as ELL?
• Used a series of Between by Within-Factor ANOVAs
–Between-subject factors: ELL status
–Within-Subject Factor: Year
–Separate analyses for Math and Reading
–Outcome is category of performance on assessment
•FFB
•A
•M
•E
Research Question 2
•
Math analyses:
–
No interactions between ELL status and Year
–
Significant main effect of ELL status and grade
–
Large effect of ELL status (.185)
–
Small effect of Year (.023)
•
Reading Analyses:
–
Significant interactions between ELL status and Year (ES = .020)
–
Large Effect for ELL status (.227)
–
Small Effect of Year (.044)
–
No real change across time for non-ELL students, but changes
across time for ELL students
Research Question 2
Math Analyses
2008
2009
2010
Mean
SD
N
Non-ELL
3.07
.69
216
ELL
2.34
.86
549
Non-ELL
3.21
.67
216
ELL
2.44
.89
549
Non-ELL
3.17
.81
216
ELL
2.32
1.00
549
Research Question 2
Reading Analyses
2008
2009
2010
Mean
SD
N
Non-ELL
2.94
.51
216
ELL
2.19
.72
549
Non-ELL
2.94
.50
216
ELL
2.30
.73
549
Non-ELL
2.99
.43
216
ELL
2.45
.69
549
Conclusion for Research
Question 2
•Similar results for Math and Reading results for
differences in ELL and non-ELL student performance
gap
–Large effect of ELL status
–Smaller effect of year
•Not a large amount of growth across time in Math and
Reading
–Possible attenuation from transitioning students
•Clear growth in reading across year for ELL students,
not for non-ELL students
Future Research
1. Investigate individual level changes in
AIMS performance for ELL and nonELL students.
•
Allows for control of variables at an
individual level
•
•
•
•
•
•
•
Acculturation
Transition points for AZELLA
Socio-economic
Academic achievement (grades)
Mobility (moving between schools)
Ethnicity
Gender
Future Research
2. Compare ELL and non-ELL students
across multiple states and multiple ELL
populations.
•
Extend findings to other
languages/ethnicities
•
Investigate curriculum and ELL program
differences across multiple districts and
multiple states to improve understanding
of the role of curriculum, ELL programs,
and the assessments themselves.
Future Research
3. Investigate and include appropriate
measures of acculturation, as well as
language
•
Acculturation vs. Language
•
Determine the relationship between
changes in acculturation across time
•
If acculturation accounts for differences
between ELL and non-ELL students on
assessment, can help identify ethnic bias.
QUESTIONS ?