Florida’s School Accountability System Overview and Updates Edward Croft, Florida Department of Education Bureau Chief, Accountability Reporting Accountability, Research and Measurement Florida Charter School Conference,
Download ReportTranscript Florida’s School Accountability System Overview and Updates Edward Croft, Florida Department of Education Bureau Chief, Accountability Reporting Accountability, Research and Measurement Florida Charter School Conference,
Florida’s School Accountability System Overview and Updates Edward Croft, Florida Department of Education Bureau Chief, Accountability Reporting Accountability, Research and Measurement Florida Charter School Conference, November 15, 2011 Florida’s School Grading System Purpose and Aims • Making school performance clear to the public • Universally understood metric (A-F) • Performance is based upon student outcomes. • System of rewards and supports • Primary and lasting goals: raising student achievement and success, bringing more opportunities to students, opening doors 2 Accountability, Research, and Measurement Context • Accountability systems in transition • FCAT cut scores increasing – raising standards • School Grades – Changes to add new statutorily required elements and new tests – End of Course (EOC) assessments – Middle school – acceleration measures • high school EOCs and industry certifications • Elementary and Secondary Education Act (ESEA) 3 Accountability, Research, and Measurement Transition in School Grades System • Transition to new tests occurring over the next 4 years • Designing the system now to accommodate the changes • Rule revision to lay out changes for 2011-12 and beyond • Develop a multi-year model to: – Implement statutory changes – Include new tests as they are available – Review and set school grading scale • Working closely with stakeholders Accountability, Research, and Measurement 4 School Grade Distribution – Graded Schools (2011 does not include high schools.) 70% 61% 60% 53% 51% 48% 50% 49% 46% 40% 30% 20% 10% 48% 52% 55% 58% 52% 45% 37% 30% 25% 13% 8% 3% 0% 1999 24% 24% 23% 23% 16% 11% 0% 2000 23% 23% 17% 13% 8% 23% 0% 3% 5% 1% 7% 2% 2001 2002 2003 2004 22% 22% 22% 21% 20% 8% 17% 8% 3% 4% 1% 3% 2005 2006 2007 23% 20% 17% 20% 14% 5% 6% 2% 2% 2008 2009 18% 19% 5% 2% 2010 18% 5% 1% 2011 -10% A B C D F 5 Accountability, Research, and Measurement Elementary School Grade Distribution (A-F) 2002 to 2011 80% 72% 70% 64% 60% 60% 56% 57% 59% 58% 54% 58% 50% 40% 30% 39% 28% 22% 20% 0% 20% 22% 8% 2% 2002 19% 20% 17% 23% 18% 10% 20% 16% 3% 1% 2003 4% 1% 2004 17% 18% 0% 2% 15% 3% 2% 2005 2006 2007 A B 5% 1% C 15% 18% 10% 3% 1% 2% 1% 2008 D 20% 2009 4% 18% 18% 5% 2% 2010 1% 2011 F 6 Accountability, Research, and Measurement Middle School Grade Distribution (A-F) 2002 to 2011 80% 70% 67% 60% 52% 50% 64% 60% 61% 59% 48% 45% 40% 36% 40% 30% 20% 25% 26% 29% 28% 24% 24% 27% 18% 10% 0% 4% 1% 2002 24% 12% 5% 4% 0% 3% 2003 2004 19% 20% 6% 18% 8% 2% 1% 0% 2% 2005 2006 2007 A B C 17% 17% 4% 1% 2008 D 17% 16% 17% 2% 1% 4% 2009 19% 17% 5% 1% 2010 1% 2011 F 7 Accountability, Research, and Measurement High School Grade Distribution (A-F) 2002 to 2010 60% 50% 48% 44% 40% 40% 42% 34% 38% 30% 30% 23% 20% 19% 16% 12% 10% 6% 23% 18% 21% 18% 3% 28% 27% 26% 26% 17% 15% 25% 24% 30% 17% 15% 10% 24% 22% 18% 15% 17% 4% 3% 4% 1% 0% 2002 2003 2004 2005 A 13% 8% 5% 4% 22% 14% 30% 2006 B C 2007 D 2008 2009 2010 F 8 Accountability, Research, and Measurement Charter School Grade Distribution (A-F) by Percent, 2004 to 2011* * Results for 2011 do not include high school grades. 70% 63% 60% 57% 50% 50% 40% 30% 39% 24% 20% 10% 13% 12% 11% 52% 48% 36% 22% 15% 21% 20% 14% 6% 12% 2005 2006 A 20% 21% 16% 19% 19% 12% 6% 8% 5% 5% 3% 0% 2004 57% 2007 B 2008 C 8% 14% 3% 4% 2009 D F 15% 17% 15% 6% 6% 5% 2010 2011 9 Accountability, Research, and Measurement Charter Schools, Reading Performance Elementary School Grades 10 Accountability, Research, and Measurement Charter Schools, Reading Performance Middle School Grades 11 Accountability, Research, and Measurement Charter Schools, Reading Performance High School Grades 38.3 12 Accountability, Research, and Measurement Reading Outcomes for Schools Graded “A” in 2010 “A” Schools with at “A” Schools with Least 50% of Students Less than 50% of at Level 3 and Up in Students at Level 3 in Reading Reading School Type Number Percent Number Percent Elementary 950 100% 0 0% Middle 352 100% 0 0% High* 98 81% 23 19% *Includes only regular high schools that received 1600-point scale high school grades. 13 Accountability, Research, and Measurement Reading Outcomes for Schools Graded “B” in 2010 School Type Elementary Middle High* “B” Schools with at Least 50% of Students “B” Schools with Less at Level 3 and Up in than 50% of Students Reading at Level 3 in Reading Number 362 95 Percent 100% 99% Number 1 1 Percent 0% 1% 81 50% 81 50% *Includes only regular high schools that received 1600-point scale high school grades. 14 Accountability, Research, and Measurement Reading Outcomes for Schools Graded “C” in 2010 School Type Elementary Middle High* “C” Schools with at Least 50% of Students “C” Schools with Less at Level 3 and Up in than 50% of Students Reading at Level 3 in Reading Number 338 64 Percent 93% 65% Number 25 35 Percent 7% 35% 1 2% 60 98% *Includes only regular high schools that received 1600-point scale high school grades. 15 Accountability, Research, and Measurement Overview of School Grading: Assessment Components (Comprising 100% of Elementary and Middle School Grades, and 50% of High School Grades) Accountability, Research, and Measurement 16 Current Elementary and Middle School Grades Model Reading Math Writing Science FCAT (100) 12.5% FCAT (100) 12.5% (100) 12.5% (100) 12.5% Performance FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% Learning Gains: All Students FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% Low 25% Learning Gains FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% (300) 37.5% (300) 37.5% Accountability, Research, and Measurement 17 Current High School Grades Model Reading Math Writing Science Performance FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% FCAT (100) 12.5% FCAT (100) 12.5% Learning Gains: All Students FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% Acceleration Grad Rate College Readiness Participation (175) 10.94% Overall (200) 12.5% Reading (100) 6.25% Performance (125) 7.81% At Risk (100) 6.25% Math (100) 6.25% (300) 18.75% (300) 18.75% (200) 12.5% Low 25% Learning Gains FCAT 2.0 (100) 12.5% FCAT 2.0 (100) 12.5% (300) 18.75% (300) 18.75% (100) 6.25% Assessment Components = 50% (100) 6.25% Acceleration Components = 50% 18 Accountability, Research, and Measurement Points for Performance (Four Components) One point for each percent of students meeting performance standards on the FCAT, as follows: – FCAT reading, math, and science = score at FCAT Level 3 or higher. – FCAT Writing = a score of 4 or higher on the essay component. (Increased from “3.5 or higher” in 2009-10). 19 Accountability, Research, and Measurement Defining Learning Gains Three Ways to Make Learning Gains: 1. Move up by one or more achievement levels. 2. Maintain a satisfactory achievement level. 3. For students who remain at FCAT Level 1 or 2, demonstrate more than one year’s worth of growth on the FCAT vertical scale. 20 Accountability, Research, and Measurement Additional Requirements Adequate Progress of Lowest Performing 25% in Reading and Mathematics • At least 50% of the low performers in a school must show learning gains in reading or math, or the school must show annual improvement in that percentage. • The school grade is lowered one letter grade if the requirement is not met - for schools that would otherwise be graded “C” or higher “Percent Tested” Requirement • 90% must be tested to receive a regular grade in lieu of an “I”. • 95% must be tested for a school to be eligible for an “A.” 21 Accountability, Research, and Measurement Current Grading Scale For Elementary and Middle Schools 800 possible points: A = ≥ 525 points B = 495-524 points C = 435-494 points D = 395-434 points F = < 395 points Accountability, Research, and Measurement 22 Students Included in the Calculation Performance Components • Students must be enrolled for the full year (present in Survey 2 and Survey 3) • Standard curriculum, Gifted, Speech Impaired, Hospital Homebound, English language learners with > 2 yrs. in ESOL • Current-year FCAT score in content area. Learning Gains • Students must enrolled for the full year. • Must have current and prior-year test scores (FCAT or Florida Alternate Assessment) in content area. 23 Reasons Why Some Schools Are Not Graded Statute provides conditions under which a school may not receive a school grade (Section 1008.34, F.S.) • Insufficient number of students tested (cell size requirements) • Alternative schools may elect to receive a school improvement rating instead • DJJ centers are not assigned school grades. • ESE centers – not enough standard curriculum students with FCAT scores (reading, math, writing, science). • Certain other types of schools that serve populations outside the tested grades. 24 Accountability, Research, and Measurement Reasons Why Some Schools Are Not Graded, and Cell-Size Issues • School grades includes only standard curriculum students in the performance components. AYP includes all students. This reduces the number of students counted for school grades • To receive a school grade schools must have a sufficient number of students tested in each “cell” of the formula • Cell sizes required for school grades are 30 for reading and math and 10 for other areas where only 1 grade is tested • Under AYP, minimum cell-size for the school = 11. • Cell-size criteria for AYP reporting are established in states’ federally approved accountability plans and may vary from state to state. • Cell-size criteria for school grades are established in the implementing rule (6A-1.09981). 25 Accountability, Research, and Measurement Cell Size Issues • Some competing factors considered when establishing cell size are: • sample size and reliability of measures • the desire to have uniform criteria for inclusion • the desire to include as many students and schools as possible in accountability determinations • the appropriateness of assessments for specific populations • characteristics of special populations. • Many schools do not receive a school grade because they do not meet the cell size requirements 26 Number of Florida Schools Receiving an AYP Rating vs. Number Receiving a School Grade 2003 2004 2005 2006 2007 2008 2009 2010 2011* # Receiving AYP Designation (Yes/No) 2,995 3,068 3,106 3,198 3,244 3,306 3,354 3,424 3,396 # Receiving a School Grade (A – F) 2,421 2,653 2,770 2,790 2,838 2,892 2,952 2,998 3,035 *Though complete high school grades for 2011 have not yet been issued, there is data available to determine the number of schools that will receive a grade. Difference 574 415 336 408 406 414 402 426 361 27 Accountability, Research, and Measurement High School Grades 28 High School Grades • State Assessment Based Components = 50% of the school grade for high schools (800 points). 29 HS Components Outside State Assessments = 50% of High School Grade (800 points) GRADUATION ACCELERATION (AP, IB, AICE, DE, Ind.Cert.) Overall Rate Participation 200 200 (in 2009-10) 175 (in 2010-11) 150 (in 2011-12) At-Risk Rate Performance 100 100 (in 2009-10) 125 (in 2010-11) 150 (in 2011-12) READINESS (ACT, SAT, CPT, P.E.R.T.) GROWTH/DECLINE Performance on Reading For each component, schools earn up to 20 points for GROWTH 100 (up to 40 points for a double-weighted component , like the graduation rate) Performance on Mathematics 100 For each component, schools lose 5 points for DECLINE (up to 10 points for a double-weighted component, like the graduation rate) Total Graduation Points Total Acceleration Points Total Readiness Points Total HIGH SCHOOL Points Possible (Non-Assessment Measures) 300 300 200 800 Accountability, Research, and Measurement 30 High School Grades: Changes in 2010-11 • More emphasis on acceleration performance (125 points in 2011 vs. 100 points in 2010) • Less emphasis on acceleration participation (175 points in 2011 vs. 200 points in 2010) • P.E.R.T. scores (as available) now included for Postsecondary Readiness. 31 Graduation Rate For 2009-10 and 2010-11, Florida is using the National Governors’ Association four-year graduation rate. Which students are included in the cohort (denominator)? Entering 9th graders in Year 1 of the 4-year cohort plus incoming transfers, minus exiting transfers and deceased students Who counts as a graduate? Standard diploma recipients and special diploma recipients Who counts as a non-graduate? Students in the adjusted cohort who did not receive a standard diploma or special diploma (i.e., dropouts, certificate of completion recipients, GED diploma recipients, other nongraduates) Accountability, Research, and Measurement 32 Graduation Rate Summary Information Graduation Rate Methods National Governors Association (NGA) Rate Students Not Included in the Calculation Graduates Non-Graduates Students who transfer to: • Other schools (public, private, or Dept. of Juvenile Justice facilities); • Home-education programs; • Adult education programs • Standard Diploma recipients • Special Diploma recipients • • • • • Standard Diploma recipients • • • • Dropouts Certificate of Completion recipients GED recipients Continuing enrollees who are not ontime graduates Deceased students New Federal Uniform Rate Students who transfer to: • Other schools (public or private) • Home-education programs Deceased students Dropouts Certificate of Completion recipients GED recipients Continuing enrollees who are not ontime graduates • Special Diplomas • Transfers to Adult education programs or Dept. of Juvenile Justice facilities who are not standard diploma recipients. Accountability, Research, and Measurement 33 Accelerated Participation •Based on AP, IB, AICE exams; Dual Enrollment courses; Industry Certification programs •Denominator = count of 11th and 12th grade students. •Students in grades 9 and 10 are included in the numerator if they have successful completions •Extra weighting for more than one exam/course. 34 Accountability, Research, and Measurement Accelerated Performance • Included students = Participants from the acceleration participation calculation. • Performance = Successful completions (i.e., creditearning scores on AP, AICE, IB, Industry Certification; and, grade of “C” or higher in dual enrollment courses). • Extra weighting for highest scores on AP, IB, AICE, and for certain Industry Certification programs. Accountability, Research, and Measurement 35 Postsecondary Readiness - Reading, Math Numerator Denominator Number of students “ready” on SAT, ACT, and/or CPT/P.E.R.T. any time during their high school careers On-time HS graduates who scored a Level 3+ on the 10th Grade FCAT • Calculated separately for reading and mathematics with max of 100 points each. • Beginning in 2011-12, by rule, the measure will apply to all on-time graduates. 36 Accountability, Research, and Measurement High School Grade Scale Grade Scale for High Schools = 1600 point scale: • • • • • A = At least 1,050 points, B = 990 to 1,049 points, C = 870 to 989 points, D = 790 to 869 points, F = Fewer than 790 points. Accountability, Research, and Measurement 37 Changes Coming for School Grades • FCAT 2.0 cut scores: increased rigor of test and achievement expectations. • Moving toward new assessments of reading and mathematics in 2014-15: PARCC • Adding new measures to the middle school grades (HS EOCs and Industry Certifications) • Assimilating EOCs into the school grades model as they are implemented Accountability, Research, and Measurement 38 Multi-year School Grades System • Transition to new tests occurring over the next 4 years • Designing the system now to accommodate the changes • Develop a multi-year model to: – Implement statutory changes – Include new tests as they are available – Review and set school grading scale 39 Accountability, Research, and Measurement Each time standards are raised, the number of lower performing schools has decreased in the following year. 2500 2,317 2,127 2,077 Number of Schools 2000 1,809 1,802 1,844 2,221 1,952 1500 1,447 1,004 1000 500 845 677 515 307 173 0 1999 2000 2001 2002 299 308 249 401 2003 233 2004 A and B Schools 143 2005 2006 2007 200 217 2008 2009 213 2010 D and F Schools 40 Accountability, Research, and Measurement School Grades in 2011-12 and Beyond – Key Issues • Changes already written in rule for 2011-12 to increase rigor for high school grades • Include learning gains for EOCs as required in statute • Learning gains on the Florida Alternate Assessment for students scoring at the lowest levels • Learning gains: structure for FCAT 2.0 reading and mathematics • Balance of performance and learning gains in new model(s); distribution/weighting of points for school grade components 41 Accountability, Research, and Measurement School Grades in 2011-12 and Beyond - Key Issues (continued) • Address the statutory requirement for greater emphasis on reading performance • High school science performance in 2011-12, Biology 1 EOC in its baseline year • Middle school grades – include high-school level EOCs and Industry Certifications (as available) • Banking middle school performance on HS level EOCs (Algebra 1 for 2011-12) for high school grades 42 Accountability, Research, and Measurement School Grades in 2011-12 and Beyond - Key Issues (continued) • Criteria for setting school grading scale • Review cell size requirements • Possibly include exceptional education students in proficiency calculations • Accountability for ESE centers • Accountability for DJJ Centers 43 Accountability, Research, and Measurement Impact of FCAT 2.0 Cut Scores on School Grades • Won’t know the impact until: – Cut scores are finalized – School Grades model for next year is finalized – Working on simulations to determine the impact of the raised scores on school grades 44 Accountability, Research, and Measurement AAAC Recommendation Learning Gains for FCAT 2.0 • Students who increase an achievement level • Students who maintain a Level 3 or higher achievement level • Students at achievement levels 1 and 2 must: – Demonstrate more than 1 year’s expected growth on the new FCAT 2.0 scale. – Under this model, Level 1 students would be expected to show more growth than Level 2 students. – Model still being reviewed, subject to change. • The requirements will be included in the revised school grades rule (6A-1.09981) • Review value added model at the school level next summer 45 Accountability, Research, and Measurement Scale Scores – Reading, Draft Proposed Rule FCAT 2.0 Reading Scale Score Cuts – Draft Proposed Rule, 10/7/11 300 275 Scale Score 250 238 227 225 230 198 175 252 237 243 221 210 200 246 258 216 208 192 200 222 207 228 213 264 268 271 249 253 256 238 243 234 218 182 222 228 Not determined 150 3 4 5 Achievement Level 2 Cut Score Achievement Level 4 Cut Score 6 7 Grade Level 8 9 10 Achievement Level 3 Cut Score Achievement Level 5 Cut Score Accountability, Research, and Measurement 46 Differences in Cut Scores (Subject to Change if Cut Scores Change.) Grade Level 1-2 Level 2-3 Level 3-4 Level 4-5 3 to 4 10 10 11 11 4 to 5 8 8 9 8 5 to 6 7 6 7 6 6 to 7 6 6 6 6 7 to 8 5 6 6 6 8 to 9 4 4 4 4 9 to 10 6 5 3 3 47 Accountability, Research, and Measurement AAAC Proposed Learning Gains Model for Florida Alternate Assessment Scores (for Students with Disabilities) • Students’ second-year score must increase relative to their first year score by more than the standard error of the difference of the two scores. • AAAC recommends setting a fixed score differential for the calculation to make it more understandable to stakeholders. • The fixed score differential would be derived from analysis of all state scores. • This provision is supported by the Leadership Policy Advisory Council (LPAC). • Task - Determine whether the value of the fixed score differential fluctuates among grade levels or achievement levels. 48 Accountability, Research, and Measurement AAAC Recommendation* Learning Gains for Algebra 1 • If a student scores at level 3 or higher on Algebra 1, the student is credited with a learning gain. • For levels 1 and 2, compare the t-score for FCAT 2.0 Mathematics in the prior year with the t-score for Algebra 1. The t-scores are scores set on a common scale between the two assessments. • Next summer, review the value added model for EOC learning gains at the school level. * Keep in mind that these are just recommendations and may (or may not) be adopted by the State Board. 49 Accountability, Research, and Measurement New Middle School Grades Component – AAAC Recommendation Middle School acceleration measure – 100 points • Structured similar to the high school measure • Participation in high school end-of-course assessments (50 points) Denominator = o Grade 8 students scoring at level 3 or higher on 7th grade FCAT + o 6th and 7th graders taking HS EOCs in the current year that have matched course records + o Other 8th graders taking HS EOCs that have matched course records • Performance on high school end-of-course assessments (50 points) Denominator = participants from the participation measure • Attainment of National Industry Certifications – AAAC recommends including with HS EOCs in the acceleration measures, when data become available. 50 Accountability, Research, and Measurement Year Administered to Students (Baseline Yr. in Red) Type of Assessment Assessment Area FCAT FCAT Writing FCAT 2.0 End-of-Course Assessments 2011-12 2012-13 2013-14 2014-15 Gr 4, 8, 10 Gr 4, 8, 10 Gr 4, 8, 10 FCAT 2.0 Reading Gr 3-10 Gr 3-10 Gr 3-10 FCAT 2.0 Mathematics Gr 3-8 Gr 3-8 Gr 3-8 FCAT 2.0 Science Gr 5, 8 Gr 5, 8 Gr 5, 8 Algebra 1 In Course In Course In Course Geometry In Course In Course In Course Biology 1 In Course In Course In Course In Course In Course In Course In Course Middle School Middle School US History Civics Gr 5, 8 Partnership for Assessment of Readiness for College and Careers (PARCC) English language Arts Gr 3-11 Mathematics High School Math EOCs (3 subjects) Gr 3-8 Postsecondary Education Readiness Test (PERT) Reading, Writing, Mathematics In course Gr 11 Gr 11 Gr 11 Gr 11 51 New EOCs Added to School Grades 201112 High School Algebra 1 (math component) Geometry (math component) Biology (science component) U.S. History (50 pts) Middle School Civics (100 pts) 201213 201314 201415 X ? X X X X 52 Accountability, Research, and Measurement ESEA Waiver Request (Four Areas to Address for State Requests) • 1: COLLEGE- AND CAREER-READY EXPECTATIONS FOR ALL STUDENTS • 2: STATE-DEVELOPED DIFFERENTIATED RECOGNITION, ACCOUNTABILITY, AND SUPPORT • 3: SUPPORTING EFFECTIVE INSTRUCTION AND LEADERSHIP • 4: Reducing Duplication and Unnecessary Burden on Local Education Agencies 53 Accountability, Research, and Measurement ESEA Waiver Request and Differentiated Accountability (DA) Request Submitted on Nov. 14, 2011 • Florida will need to revise its Annual Measurable Objectives (AMOs) for AYP reporting to conform with options presented by the U.S. Department of Education. • If the waiver request is approved, the AYP performance of subgroups and schools will be reported on annual reports but will not be used in determining school improvement requirements. • If our DA system is revised as planned, schools will be classified in DA based solely on their school grade: • Prevent = “C” schools • Correct = “D” schools • Intervene = “F” schools 54 Accountability, Research, and Measurement Timeline • May – Feedback from Leadership Policy Advisory Council (LPAC) • July – Feedback from Assessment and Accountability Advisory Committee (AAAC) • August – Feedback from LPAC • October – Feedback from AAAC and LPAC • November – Feedback from AAAC, Submit ESEA Flexibility Request • December – FCAT rule to the State Board to set cut scores • December – School Grades rule workshops • January – Receive USDOE determination on ESEA Flexibility Request • February – School grades rule to the State Board 55 Accountability, Research, and Measurement Department Contact Information for School Grades [email protected] Or call (850) 245-0411 56