Presentation for Review and Discussion of AMAO 2 Criteria & Targets
Download
Report
Transcript Presentation for Review and Discussion of AMAO 2 Criteria & Targets
Review and Discussion
of AMAO 2 Criteria & Targets
NC Department of Public Instruction
With
WestEd & Wisconsin Center for Education Research
Statewide Web Conference
August 30, 2010
Purpose
Review and comment on recommendations
for changes to NC State Board of Education
policy GCS-A-012, Annual Measurable
Achievement Objectives for NCLB Title III
NCDPI Recommendations in August
Policy Revisions to SBE in September
NCDPI
AMAO 2 Meeting
2
Session Agenda
Introduction and Purpose (Ground Rules)
Historical Perspective and 1-year Recap
AMAO 2 Proficiency Criteria
(Comprehensive Objective Composite,COC)
Review and Comparison (2009 & 2010)
AMAO 2 Targets for Consideration:
2009-10 and Beyond
Next Steps & Meeting Wrap Up
NCDPI
AMAO 2 Meeting
3
Introductions
New NCDPI Staff
Scott Beaudry, Testing Policy & Operations
Special Guests
Robert Linquanti, WestEd
Gary Cook, Wisconsin Center for Education
Research
Shirley Carraway, Appalachia Regional
Comprehensive Center
NCDPI
AMAO 2 Meeting
4
Historical Perspective
In 2007-2008:
1. Determined revisions to AMAOs 1 & 2 needed
Criteria were too loose or too stringent
Targets set without federal guidance
Targets not based on empirical data
2. Determined new standards and assessments needed
ACCESS for ELLs would replace IPT
3. Determined that AMAO 1 criteria and targets would be
revised after two years of ACCESS for ELLs data was
gathered.
NCDPI
AMAO 2 Meeting
5
Historical Perspective: AMAO 2
In 2007-2008, cont’d.:
Decided to keep the criterion for
proficiency the same for 2007-08
2007-08 target was set at 17% to account
for differences in using Form A and Form B
of the IPT
Targets for 2008-09 and beyond removed
as they needed to be based on empirical
results from new ELP assessment
NCDPI
AMAO 2 Meeting
6
Last Year Recap: AMAO 2
In 2008-09:
Per USED Notice of Final Interpretations (2008):
Only one data point needed to calculate AMAO 2 for
each ELL
All ELLs (K-12) must be included in calculation
2008-09 ACCESS results used to define COC
and new, one-year target for 2008-09 (14.7%)
Decision made to set future targets after
examining another year of ACCESS results
NCDPI
AMAO 2 Meeting
7
Last Year Recap: AMAO 2
In 2008-09, cont’d.:
Stakeholders endorsed state-recommended
COC derived from analyses of student
performance on 2008-09 ACCESS and
state’s reading and math assessments
Overall 4.8, R & W each 4.0 minimum
NCDPI
AMAO 2 Meeting
8
AMAO 2 Proficiency Criteria (COC)
Review & Comparison (2009 & 2010)
Purpose: Replicate analysis performed in
2008-09 to validate COC criteria chosen
Apply same decision consistency method
to 2009-10 ACCESS and EOG/EOC reading
and math assessments
NCDPI
AMAO 2 Meeting
9
Decision Consistency Method
These analyses identify language
proficiency level that optimally
classifies students as true-positives or
true-negatives on both NC EOG/EOC
Reading & Math Assessments and
ACCESS
NCDPI
AMAO 2 Meeting
10
Decision Matrix
Criterion A
Criterion B
Below
Above
True
False
Below
Negative Positive
False
Above
Negative
True
Positive
True Positives True Negatives
Correct
Total number in sample
NCDPI
AMAO 2 Meeting
11
Decision Matrix
ELP Assessment
Below
Above
Content
Assessment
Below
43
14
Above
18
25
Correct = 68%
NCDPI
AMAO 2 Meeting
12
Decision Matrix
ELP Assessment
Below
Above
Content
Assessment
Below
51
5
Above
10
34
Correct = 85%
NCDPI
AMAO 2 Meeting
13
Reading to ACCESS: 2009
NC EOG Reading to ACCESS Decision Consistency
across Clusters
Percent Classified Correctly
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
WIDA Composite Proficiency Band
Cluster 3-5
Cluster 6-8
NCDPI
AMAO 2 Meeting
Cluster 9-12
14
Mathematics to ACCESS: 2009
NC EOG Mathematics to ACCESS Decision Consistency
across Clusters
Percent Classified Correctly
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
WIDA Composite Proficiency Band
Cluster 3-5
Cluster 6-8
NCDPI
AMAO 2 Meeting
Cluster 9-12
15
Comparing DC Analysis, 2009 to 2010:
Reading to ACCESS, Grades 3-5
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G35 Read 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G35 Read 2010
NCDPI
AMAO 2 Meeting
16
Comparing DC Analysis, 2009 to 2010:
Reading to ACCESS, Grades 6-8
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G68 Read 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G68 Read 2010
NCDPI
AMAO 2 Meeting
17
Comparing DC Analysis, 2009 to 2010:
Reading to ACCESS, Grades 9-12
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G912 Read 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G912 Read 2010
NCDPI
AMAO 2 Meeting
18
Comparing DC Analysis, 2009 to 2010:
Math to ACCESS, Grades 3-5
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G35 Math 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G35 Math 2010
NCDPI
AMAO 2 Meeting
19
Comparing DC Analysis, 2009 to 2010:
Math to ACCESS, Grades 6-8
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G68 Math 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G68 Math 2010
NCDPI
AMAO 2 Meeting
20
Comparing DC Analysis, 2009 to 2010:
Math to ACCESS, Grades 9-12
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
<2.0
2.0-2.4
2.5-2.9
3.0-3.4
G912 Math 2009
3.5-3.9
4.0-4.4
4.5-4.9
5.0-5.4
5.5-6.0
G912 Math 2010
NCDPI
AMAO 2 Meeting
21
AMAO 2 Criterion Confirmed
Comprehensive Objective Composite (COC)
Current English language proficiency definition
on the ACCESS test holds:
Composite score of at least 4.8 and at least 4.0 on
Reading subtest and 4.0 on Writing subtest.
Note: Students who attain the COC as defined
above exit LEP identification. Those who do
not remain identified LEP.
NCDPI
AMAO 2 Meeting
22
REMINDER:
2009-10 AMAO 2 Cohort Definition
Cohort definition required by federal law:
ALL LEP students (K-12) must be included in AMAO
2 calculation
Numerator = # of LEP students attaining COC
Denominator = #of LEP students required to test
NCDPI
AMAO 2 Meeting
23
Historical AMAO 2 Target Data
Year
Target
Met
Not Met
Missing data
Total
2003-04
2004-05
20.0%
25.0%
86
81
6
1
0
1
92
83
2005-06
2006-07
2007-08
30.0%
35.0%
17.0%
4
76
3
83
38
47
0
85
2008-09
2009-10*
14.7%
11.8%
25
68
63
20
0
0
88
88
*All 2009-10 results are unofficial
NC LEA and State AMAO 2 Performance Using
Current AMAO 2 Criterion:
AMAO 2 Criterion
Overall 4.8 (R&W GE 4.0)
2010 Analysis
% LEP meeting AMAO 2 Criterion by LEA Percentile Rank
STATE
P10
P15
P20
% LEP
9.5%
10.7%
11.5%
P25
P50
P75
P80
P90
11.8% 13.8% 16.8% 17.4% 19.7%
14.5%
Shows percentage of LEPs meeting COC performance
level for LEAs at that ranking and Statewide
NCDPI
AMAO 2 Meeting
25
2010-2019 Annual Targets
for LEAs & State
using 2010 as Base Year
Recommended Targets for Consideration
Percent of LEP attaining English
proficiency
25.0
23.0
21.0
19.0
17.0
15.0
13.0
11.0
9.0
7.0
5.0
25 %ile
2009 2010 2011 2012 2013 2014 2015 2016
-10 -11 -12 -13 -14 -15 -16 -17
AMAO 2
11.8 12.4 12.9 13.5 14.0 14.6 15.1 15.7
Target
NCDPI
AMAO 2 Meeting
AMAO 2
• Proposed target
for 2009-10 is
11.8% of LEP
students in an
LEA attaining
English
language
proficiency.
• Proposed end
point in 2018-19
75 %ile is 16.8% of LEP
students in an
LEA attaining
2017 2018
English
-18 -19
language
16.2 16.8
proficiency.
26
Target Recommendation: 2010-2019
Set the 2009-10 target at 11.8%
(25th %ile)
Set the 2018-19 target at 16.8%
(75th %ile)
Structure targets to increase by equal
increments each year
(0.55 percentage points)
NCDPI
AMAO 2 Meeting
2009-10 AMAO Analysis
AMAO 1 Criteria and targets are the
same
improve at least one proficiency level in
at least one of the subtests of reading,
writing, speaking, or listening
Target = 70%
2009-10 Analysis (continued)
Sanctions for AMAOs Not Met
In 2008-09, Title III status based on
whether or not the LEA failed to make
progress toward meeting the same
AMAO
Starting in 2009-10, Title III status
based on failure to meet the
AMAOs
GCS-A-012 Revisions
Show Draft Policy
Policy to SBE in September as Action
on First Read
Next Steps (for 2009-10 data)
Updated GCS-A-012 sent to USED for Title III
Workbook and Title III Plan submission
Preliminary AMAO report sent to districts for
review in September
AMAO report presented to SBE in November
NCDPI
AMAO 2 Meeting
31
Next Steps (2010 -11 and beyond)
Analysis of potential changes to AMAO 1
progress definitions and targets to occur
during Fall 2010
AMAO 1 analyses and draft
recommendations vetted with
stakeholders during 2010-11 school year
AMAO 1 policy approval in 2010-11
Updated policy sent to USED for
submission with Title III Workbook and
Title III Plan
Federal Title III Audit in Spring 2011