NCLB: Then and Now

Purpose of No Child Left Behind

“…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments”

• • • • • • •

Critical Components of NCLB Accountability

Annual testing in language/arts reading and mathematics for all public school students in grade 3-8 and at least once in high school.

Additional academic indicator in elementary and middle schools plus high school graduation rate.

Requirement for increasing percentages of students to demonstrate proficiency each year (Annual Measurable Objective = AMO).

Requirement to disaggregate and hold schools and districts accountable for the performance of students by racial/ethnic, and low-income, LEP, and SWD groups.

Requirement for 95% participation in State Assessments.

Requirement that disaggregated groups, schools, and districts demonstrate Adequate Yearly Progress (AYP) either through exceeding the Effective Annual Measurable Objective (EAMO) or by closing the gap in terms of students not proficient (Safe Harbor).

Schools and Districts that do not make AYP are identified as in Need of Improvement. Such schools and districts are subject to increasing levels of assistance, interventions, and consequences.

What Critics are Saying

• Goals of NCLB are inappropriate or unrealistic leading states to: – Game the system and define proficiency down – Overemphasize test preparation – Narrow the curriculum • NCLB’s accountability system: – Is both overly simplistic and overly complex – Unfairly disadvantageous certain schools and districts – Diverts resources from school improvement to ineffective choice and SES programs.

What Supporters Are Saying

– NCLB is pushing us to improve assessment and data collection.

– NCLB is forcing us to confront the gap in student performance.

– NCLB is fostering choice and competition in education.

– NCLB is providing additional resources to those most in need.

– NCLB is ensuring that there are consequences for performance.

New York’s NCLB Plan: A brief History

– Submitted December 2002 – Revised and Approved January 2003 – Revisions submitted: • October 2004 • April 2005 • June 2005 • October 2005 • March 2006 • July 2006

M.L. Mencken said: For every complex problem, there is a solution that is clear, simple, and wrong.

Determining AYP in Elementary- and Middle-Level ELA or Math for Schools with 30 or More Continuously Enrolled Students

Participation Rate for schools with 40 or more students enrolled on test day School did not test 95 percent of every group of 40 or more School tested 95 percent of every group of 40 or more Compute weighted average of 2002 03 and 2003-04 participation rate for groups below 95% Below 95%

NO AYP

Schools with 30 to 39 students Check the Performance Index for each group with 30 or more students Above 95% Every group’s PI is NOT equal to or greater than the Effective AMO Every group’s PI is equal to or greater than the Effective AMO Find safe harbor targets in ELA or math and determine if the group met the science qualification for safe harbor AYP A group whose PI is below its Effective AMO did NOT make safe harbor Each group whose PI is below its Effective AMO made safe harbor

NO AYP

Will Change do you Good?

• Assessment Program • Annual Measurable Objectives • Methodology for Determining Elementary- and Middle-level accountability • Methodology for Determining District-level Adequate Yearly Progress • Definitions of high school ELA and math accountability and graduate rate cohorts • Requirements for meeting participation rate • Determining Adequate Yearly Progress for English language learners • Determining Adequate Yearly Progress for Students with disabilities.

Grades and Levels

For 2002-2003, 2003-2004, and 2004-05 School Years, NY had three grade levels: –Elementary (Grade 4) –Middle (Grade 8) –High School Beginning in 2005-2006, NY has two grade levels: - Elementary-Middle (Grades 3-8) - High School

Calculating the Grade 3-8 Performance Index

Grade 3 4 5 Number of Students 60 80 60 Levels 1 14 6 12 2 14 12 20 3 20 40 20 4 12 22 8 TOTAL 200 32 46 80 42 Total % 16% 23% 40% 21% Index = (23+40+21+40+21)=145

Original Annual Measurable Objectives for 2002–03 to 2013–14

School Year

2002-03 2003-04 2004–05 2005–06 2006–07 2007–08 2008–09 2009–10 2010–11 2011–12 2012–13 2013–14 154 162 169 177 185 192 200

Elementary-Level ELA Math

123 123 131 138 146 136 136 142 149 155 162 168 174 181 187 194 200 144 154 163 172 181 191 200

Middle-Level ELA Math

107 107 116 126 135 81 81 93 105 117 129 141 152 164 176 188 200 165 171 177 183 188 194 200

Secondary-Level ELA Math

142 142 148 154 159 132 132 139 146 152 159 166 173 180 186 193 200

Revised Annual Measurable Objectives for 2005–06 to 2013–14

School Year

2005–06 2006–07 2007–08 2008–09 2009–10 2010–11 2011–12 2012–13 2013–14

Elementary & Middle-Level ELA Math

X X X X+Y X+2Y X+3Y X+4Y X+5Y 200 X X X X+Y X+2Y X+3Y X+4Y X+5Y 200

Secondary-Level ELA Math

154 159 165 171 177 183 188 194 200 146 152 159 166 173 180 186 193 200

Calculating AMOs

• In 2000-2001 and 2001-2002 combined, 20% of the NY’s public school students were enrolled in school’s with PI’s below: – Grade 4 ELA: 123 – Grade 8 ELA: 107 – Grade 4 Math: 136 – Grade 8 Math: 81 • In 2004-05, the percent of students below the 2004-2005 AMO’s were: – Grade 4 and 8 ELA combined: 13.3% – Grade 4 and 8 math combined: 4.2% • Estimated 2005-2006 Grade 3-8 ELA results: – AMO at 13.3% = 122 – AMO at 20.0% = 133 – Average of Original Grade 4 and 8 2005-06 AMO: 132

Sample Identification of School for Improvement Status

School A fails to make AYP in the following groups: –Grade 4 ELA White Students in 2004–05 –Grade 3-8 Math Economically Disadvantaged Students in 2005–06 School A is

not

identified for improvement because it has

not

failed to make AYP for two consecutive years in the same subject and level.

School B fails to make AYP in the following groups: –Grade 4 ELA Asian Students in 2004–05 –Grade 3-8 ELA LEP Students in 2005–06 School B

is

identified for improvement because it

has

failed to make AYP for two consecutive years in the same subject and grade (grade 4 ELA).

District-Level Accountability

• 2002-2003: One bite at the apple system • 2003-2004 and 2004-2005: Three bites at the apple system • 2005-2006: Two Bites at the apple system: District Level Accountability based on subject. For a district to fail to make AYP in a subject it must fail to make AYP at all levels in that subject. • For example, if a district fails to makes AYP in Grade 3-8 ELA and fails to make AYP in High School ELA in 2005-2006 and 2006-07, the district is identified as a DINI for ELA.

Guide to Accountability Cohorts

High schools are accountable for three areas: • English and mathematics performance; • English and mathematics participation; and • graduation rate.

A different cohort of students is measured in each of these areas. Further, the cohort used to measure English and mathematics performance has been redefined beginning with the 2002 cohort; the cohort used to measure graduation rate has been redefined beginning with the 2003 cohort.

2005-06 School Year

Purpose English & Cohort Used 2002 Accountability Cohort (one-year math performance continuous enrollment) English & math participation Graduation rate All students reported in 2006 STEP as enrolled in grade 12 in 2005-06 2001 Graduation Rate Cohort (two years continuous enrollment) plus transfers to GED

Accountability Cohort Definition

1999 – 2001 Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the third year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their nineteenth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

2002 and Beyond Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the fourth year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their twentieth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

Accountability Cohort Examples: 2005-2006

Old rules: A student who entered grade 9 in September 2001 in School A and dropped out of School A prior to October 2003 is not part of the ELA and math accountability cohort for School A. Revised rules: A student who entered grade 9 in September 2002 in School B and dropped out of School B prior to October 2005 is not part of the ELA and math accountability cohort for School B.

Revised rules exclude students who dropped out between October 2004 and October 2005.

Accountability Cohort Examples: 2005-2006

Old rules: A student who entered grade 9 in September 2001 in School A and transferred to School B prior to October 2003 is part of the ELA and math accountability cohort for School B. Revised rules: A student who entered grade 9 in September 2002 in School A and transferred to School B prior to October 2005 is part of the ELA and math accountability cohort for School B.

Revised rules include students who transferred into School B between October 2004 and October 2005 and remained enrolled as of October 2005.

Graduation Cohort Definition

1999 – 2002 Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school on BEDS day of the third year following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, reached their nineteenth birthday during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

2003 and Beyond Cohorts: Accountability cohort consists of all students, regardless of their current grade status, who were enrolled in the school for five consecutive months following first entry into grade 9 (anywhere) or in the case of ungraded students with disabilities, who reached their seventieth during the school year and did not subsequently transfer to another high school, become incarcerated, leave the country, or die.

Graduation Rate Cohort Examples

• Students not included in the West High School cohort: –

CURRENT:

A student who entered grade 9 at the school in September 2002 and dropped out prior to October 2004 –

NEW:

A student who entered grade 9 in September 2003 and enrolled in the school less than five months prior to dropping out.

Graduation Rate Cohort Examples

• Students included in the West High School cohort: –

CURRENT:

A student who entered grade 9 in September 2002, was enrolled in the school in October 2004, and dropped out of the school after October 2004 and did not reenter a degree-granting program.

–

NEW:

A student who entered grade 9 in September 2003, enrolled in the school for five months and dropped out prior to June 2007 and did not reenter a degree-granting program.

Implications of Changes

• Graduation Rate and ELA and math accountability cohort will become more independent.

• Accountability cohort will include more students who transfer into a school but will exclude more dropouts. The calculated Performance Index for most schools and districts is higher using this definition.

• Graduation cohort will include more students who drop out. The calculated Graduation Rate for most schools and districts is lower using this definition.

One Good Thing

• The “first test” as a senior rule has been eliminated.

• Students highest ELA and math performance during cohort period may be used for accountability beginning with 2002 cohort.

Accountability for Limited English Proficient Students

• 2002-2003 to 2005-2006: Accountability and participation for certain LEP students on Grade 4 and 8 ELA could be based on performance on New York State English as a Second Language Achievement Test (NYSESLAT). Students held accountable based on NYSESLAT tests were not required to take the regular ELA assessments.

• Beginning in 2006-2007: Only newly arrived English language learners (less than one year in the US) may take the NYSESLAT in lieu of regular Grade 3-8 ELA assessments to meet participation requirements. NYSELSAT scores no longer used to calculate Performance Index.

• Beginning in 2006-2007: Performance of former LEP students may be included in LEP subgroup for up to two years following exit from LEP status.

Students with Disabilities Participation in State Assessments

General assessment



based on grade the student is in, or



not assigned grade level (ungraded): based on chronological age (which is consistent with birth dates of non disabled peers)

Students with Disabilities Participation in State Assessments New York State Alternate Assessment (NYSAA)



not assigned grade level (ungraded): based on chronological age chart on field memo

NYSAA

•

Linked to grade level content standards

•

Changes in NYSAA administration format and timetable beginning this year

•

http://www.vesid.nysed.gov/specialed/alterasses sment/changesadmin.htm

Accountability for Students with Disabilities

• Modified Standards (2%) –

For 2005-06, Grade 3-8 ELA and Math:

•

34 index points added to SWD group if SWD subgroup met participation rate and SWD subgroup was only one to fail to make AYP.

•

If Performance Index with inclusion of 34 points meets or exceeds AMO, the school or district has made AYP on that criterion

IDEA District Accountability: Levels of Intervention • In need of assistance • In need of intervention • In need of substantial intervention

•

District in Need of Assistance

School district requires technical assistance in order to improve performance

•

Criteria:

–

Graduation rate is more than 18.5% and less than 35%; and/or

–

Drop out rate is 20% to less than 33%; and/or

–

Math and ELA State assessment results: score on performance index in two or three areas is below the State average for SWDs and district did not make AYP in these areas.

•

58 school districts are “in need of assistance”

District in Need of Intervention

• • •

Requires direct State involvement and oversight to improve performance Criteria:

–

Graduation rate is 18.5% or less; and/or

–

Drop out rate is 33% or higher; and/or

–

Math and ELA State assessment results: Score on performance index in four areas is below State SWD average and district did not make AYP in these areas.

17 school districts are “in need of intervention”

District In Need of Substantial Intervention

• •

Criteria:

–

History of chronic noncompliance or substantial failure to achieve progress and/or

–

Systemic agency-wide problem of noncompliance and poor performance of students with disabilities No school districts identified this year as in need of substantial intervention.

Are we done yet?

• More possible changes: – Increase in the graduation standard from 55% – Replacement of elementary and middle level science with attendance rate.

– Addition of high graduation and attendance rates as criteria for identification of School Under Registration Review.

– New Federal rules on accountability for SWDs.

– New racial/ethnic groups for accountability?

– Growth model?

Challenges Ahead

• Single Grade 3-8 Performance Index makes schools and districts responsible for more disaggregated groups.

• New standard setting for grade 3-8 assessments may challenge middle schools even more.

• Changes in testing practices for LEP students require that bilingual education programs emphasize rapid acquisition of English as well as fluency in the native language.

• New graduation standards will raise expectations for middle schools and force high schools to reveal “hidden students.”

Challenges Ahead: Lists, Lists, Lists

• SURR • SINI, DINI • SRAP, DRAP • IDEA Districts • Title III AMAO’s • Persistently Dangerous Schools • High School Initiative

Growth Models: The Holy Grail of Accountability

– Much controversy about Growth Models – Some claim good growth models exist (Sanders) – Some claim that the state of the psychometric art does not currently support valid and reliable growth models – Growth models are not easy: • Less sophistication required to make accurate determinations about status then growth.

• Acquisition of knowledge is not linear.

• Growth models break down more quickly as groups become smaller.

If You Take Away Nothing Else..

• If a schools fails to make AYP, it is most often because of the performance of its most vulnerable populations, but • Systemic problems most often manifest themselves in the most vulnerable populations.

More Information

Ira Schwartz, Coordinator Accountability, Policy, and Administration New York State Education Department Office of School Improvement and Community Services [email protected]

718 722-2796