Transcript EngageNY

Norms:
an established standard of behavior shared by members of a
group to which each member is expected to conform
Presuming Positive Intentions
EngageNY.org
2
A New Baseline:
Measuring Student Progress on
the Common Core Learning
Standards
August 2013
EngageNY.org
“I understand these scores are sobering
for teachers and principals. It's frustrating
to see our children struggle. But we can't
allow ourselves to be paralyzed by
frustration; we must be energized by this
opportunity. The results we've announced
today are not a critique of past efforts;
they're a new starting point on a roadmap
to future success.”
John B. King – Commissioner of Education
EngageNY.org
4
“Our students face very real challenges.
But it's better to have our students
challenged now - with caring teachers
there to help -- than frustrated later when
they start college or try to find a job and
discover they are unprepared.”
John B. King – Commissioner of Education
EngageNY.org
5
EngageNY.org
6
Common Core in New York
2010: Board of Regents adopts Common Core State
Standards
2013: Common Core Assessments in Grades 3 – 8 ELA and
Math are administered
2014: Roll-out of Common Core Regents Exams begins
• June 2014: ELA and Algebra I
• June 2015: Geometry and Algebra II
Class of 2017: First cohort of high school graduates required
to pass Common Core Regents Exams for graduation
Transition to New York Common Core Assessments is a
seven year phase-in.
EngageNY.org
7
A New Baseline
• This year’s grades 3-8 ELA and math proficiency percentages
should not be compared directly with prior-year results.

Unlike prior years, proficiency is now based on the
Common Core – a more demanding set of knowledge and
skills necessary for 21st century college and careers.
• These results present a new and transparent baseline from
which we can measure student progress and preparedness for
college and careers.
• School and district leaders are urged to be thoughtful to ensure
these proficiency results have no negative impact on students,
schools, districts, or teachers.
• No new districts will be identified as Focus Districts and no new
schools will be identified as Priority Schools based on 2012-13
assessment results.
EngageNY.org
8
State-Provided Growth Scores
New York’s growth scores are based on year-to-year
comparisons for similar students, all of whom
experienced New York’s Common Core assessments for
the first time in 2012-13.
The state-provided growth scores are based on year-toyear comparisons on scale scores, not performance levels.
Therefore, the state-provided growth scores resulted in
similar percentages of educators earning each rating
category* in 2012-13 compared to 2011-12.
*Highly Effective, Effective, Developing, Ineffective
EngageNY.org
9
State-Provided Growth Score
Comparison - 2012 and 2013
2011-12 Percent of
Teacher MGPs
N=33,129
2012-13 Percent of
Teacher MGPs
N=37,614
Highly Effective
6.7%
7.0%
Effective
77.2%
76.3%
Developing
10.1%
10.8%
Ineffective
6.0%
5.9%
HEDI Rating
Growth scores are expected to be released to districts the week of 8/19
EngageNY.org
10
Converging Evidence about
College Readiness
College and
Career Readiness
Whether the measure is
national or New York-specific,
there is converging evidence
about student preparedness
for college and careers.
EngageNY.org
11
Graduating College and
Career Ready
New York's 4-year high school graduation rate is 74% for All Students.
However, the percent graduating college and career ready is significantly lower.
June 2012 Four-Year Graduation Rate (2008 Cohort)
Graduation under Current Requirements
Calculated College and Career Ready*
% Graduating
% Graduating
All Students
74.0
All Students
35.3
American Indian
58.5
American Indian
18.8
Asian/Pacific Islander
81.6
Asian/Pacific Islander
56.5
Black
58.1
Black
12.5
Hispanic
57.8
Hispanic
15.7
White
85.7
White
48.5
English Language Learners
34.3
English Language Learners
7.3
Students with Disabilities
44.7
Students with Disabilities
4.9
*Students graduating with at least a score of 75 on Regents English and 80 on a Math Regents, which correlates with
success in first-year college courses.
Source: NYSED Office of Information and Reporting Services
EngageNY.org
12
New York 2011 NAEP Reading
31%
41%
4%
9%
24%
26%
33%
32%
Grades 4 and 8
Grade 4
Level 1
Grade 8
Level 2
Level 3
Level 4
13
New York 2011 NAEP Math
5%
7%
23%
20%
30%
31%
40%
44%
Grades 4 and 8
Grade 4
Level 1
Grade 8
Level 2
Level 3
Level 4
14
SAT and PSAT Benchmarks for
New York Students
• College Board and NAEP study determined scores
on SAT and PSAT/NMSQT that correspond with
college readiness for the nation.
• Criteria were adapted slightly to accommodate
New York students’ course-taking patterns.
• The results for all New York students who
graduated in 2010 and who took the SAT and
PSAT/NMSQT are on the following slide.
EngageNY.org
15
SAT and PSAT/NMSQT CCR
Benchmark Data: ELA
EngageNY.org
16
SAT and PSAT/NMSQT CCR
Benchmark Data: Math
EngageNY.org
17
Why Readiness Matters Underperformance Costs $1 Trillion
• America’s urban school districts underperform
compared with their suburban counterparts.
• America’s suburban school districts underperform
compared with their international counterparts.
• If American students performed at the same level in
math as Canadian students, we would add $1 trillion
annually to the economy.
Source: Levine, Arthur. “The Suburban Education Gap.” The Wall Street Journal. 2012.
http://online.wsj.com/article/SB10000872396390444223104578041181255713360.html
EngageNY.org
18
Why Readiness Matters Talent Dividend
If New York increased its college attainment
rate by just one percent – from 33.8 to 34.8
percent – the State would capture a $17.5
billion Talent Dividend.
Source: CEOs for Cities:
http://ceosforcities.org
EngageNY.org
19
Regents Reform Agenda
Implementing Common Core
standards and developing
curriculum and assessments
aligned to these standards to
prepare students for success in
college and the workplace
Highly Effective
School Leaders
Highly Effective
Teachers
College and
Career Ready
Students
Building instructional data systems
that measure student success and
inform teachers and principals how
they can improve their practice in
real time
Recruiting, developing, retaining, and
rewarding effective teachers and
principals
Turning around the lowestachieving schools
EngageNY.org
20
Process Point
The idea of these results as a baseline
assessment of our students’ capacity to meet
demands of College and Career.
What other data has been telling us and how it
aligns to results.
The Reform agenda work that has been leading
up to this moment.
EngageNY.org
21
SETTING PERFORMANCE STANDARDS FOR
COMMON CORE ASSESSMENTS
Common Core
Standards / CCR
Research-based
Methodology
NY Educator
Judgment
Cut
Scores
Standard Setting
Determination
EngageNY.org
22
Just as New York Educators are
Essential to Test Development…
New York educators are represented on the following panels:

New York State Content Advisory Panels
• Spans early childhood and P12 through CUNY, SUNY and cIcu faculty
Item Development, Item Review, Final Form Review
These panels are informing:

College and Career Ready Determinations

Test specifications, policies, and items

Policy-level and grade-level performance level
descriptions

EngageNY.org
23
…New York Educators are
Essential to Setting Standards
• 95 New York educators for Days 1 to 4
• 34 stayed for Day 5
• Variety of educators nominated and represented:





K-12 ELA and Math Teachers
BOCES
ELL and SwD specialists
Higher Education
K-12 Administration
• Panelists represented New York’s geographic and
demographic diversity
EngageNY.org
24
Days 1 to 4
95 panelists followed a research-based protocol:
•Worked in four groups (ELA 3-5, ELA 6-8, Math 3-5, or
Math 6-8).
•Defined expectations based on what students should know
and be able to do at each grade according to the demands
of the Standards.
•Reviewed the New York tests and external benchmark data
(NAEP, SAT, PSAT/NMSQT).
•Viewed test questions in easiest-to-hardest order and made
individual panelist judgments on where to place the cut
scores for proficiency levels.
•Discussed rationales for their judgments and viewed impact
data for each of four rounds of review.
EngageNY.org
25
Panelist Evaluation of
Standard-Setting Process
Over 90% of panelists at end of Day 4 said they would
defend the recommended cut scores. Of those in the
minority, none strongly disagreed with the recommended
standards (they only moderately disagreed).
“The standards are being set by a group that consists of
teachers, K-12, college professors and administrators. It
makes sense and it's transparent.”
“The collective experience and knowledge evidenced in
discussions and the outcomes of the tasks resulted in fair
and unbiased standards. Participants followed directions
carefully and judiciously.”
EngageNY.org
26
Day 5
• 34 of the 95 panelists remained and worked in two groups
(ELA 3-8 or Math 3-8)
• Panelists reviewed the results across all six grade levels to
ensure that the results made sense from a broader
perspective.
• Panelists were allowed to make small adjustments only
(within +/- 4 raw score points).

Adjustments were required to be grounded in the
expectations of the Common Core standards.
• Commissioner was presented with both sets of
recommendations – those from Day 4 and from Day 5.
• The results of Day 4 and Day 5 differed minimally.
EngageNY.org
27
Statement from National Experts
“In observing the training for the NY State Grades 3-8 ELA
and Math Common Core Tests Standard Setting on June 29,
2013, we were comfortable that the facilitators were following
best practices in implementing research-based procedures.
After observing a full standard-setting session, we are
confident that the recommended cut scores were derived
using a well-implemented process that followed the plan
presented to the NY technical advisory committee (TAC).”
Marianne Perie, Co-Director at the Center For Educational Testing and Evaluation,
University of Kansas
Michael Rodriguez, Campbell Leadership Chair in Education and Human Development,
University of Minnesota
New York State TAC
EngageNY.org
28
Performance Level Descriptors
EngageNY.org
29
The Commissioner accepted Day 5 performance
standard recommendations with no changes.
EngageNY.org
30
New Standards, New Tests,
New Scale
New Scale
New performance standards
100 – 425
NYS Level 4: Student excels in CCLS for
this grade level
NYS Level 3: Student is proficient in CCLS
for this grade level
NYS Level 2: Student is below proficient in
CCLS for this grade level (partial but
insufficient)
NYS Level 1: Student is well below
proficient in standards for this grade level
EngageNY.org
31
2013 Grades 3-8
English Language Arts
Results
EngageNY.org
In ELA, 31.1 percent of students in grades 3-8 across the
State met or exceeded the proficiency standard (NYS
Levels 3 or 4), reflecting a new baseline relative to the
Common Core Standards
The vertical lines indicate
years where changes
were implemented. In
2010, cut scores changed,
but the standards and
scale remained the same.
In 2013, the standards,
scale, and cut scores
changed to measure the
Common Core.
77.4%
53.2%
52.8%
55.1%
31.1%
Grades 3-8
2009
2010
2011
2012
2013
33
Grade 3
Grade 4
Level 1
Grade 5
Level 2
Level 3
Grade 7
10.3%
29.6%
36.6%
23.4%
8.0%
16.0%
13.6%
Grade 6
32.0%
36.6%
23.4%
41.6%
8.5%
21.7%
28.8%
34.1%
35.7%
9.3%
3.7%
35.7%
33.2%
27.4%
30.5%
39.2%
21.0%
In each grade level statewide, the majority of students
performed at NYS Levels 1 or 2 in ELA
Grade 8
Level 4
34
3.2 percent of English language learners met or exceeded
the ELA proficiency standard (NYS Levels 3 or 4) in grades
3-8
58.2%
55.7%
3.2%
11.7%
12.6%
14.3%
33.0%
36.4%
55.9%
80.1%
Beginning in 2013-14, data will be available for
students who received ELL services at any time
prior to test administration.
English Language Learners
2009
2010
Not English Language Learners
2011
2012
2013
35
62.4%
59.9%
5.0%
15.5%
14.5%
15.2%
35.9%
39.3%
60.2%
84.2%
5 percent of students with disabilities met or exceeded the
ELA proficiency standard (NYS Levels 3 or 4) in grades 3-8
Students with Disabilities
2009
General Education
2010
2011
2012
2013
36
Asian
2009
Black
2010
Hispanic
2011
American
Indian/Alaskan
Native
2012
White
31.1%
39.9%
53.2%
52.8%
55.1%
77.4%
85.9%
64.8%
64.2%
66.4%
41.3%
40.6%
43.1%
21.2%
68.9%
36.8%
37.2%
40.0%
17.7%
64.8%
34.4%
35.0%
37.2%
16.1%
64.3%
86.6%
67.9%
67.4%
70.1%
50.4%
The ELA proficiency results (NYS Levels 3 or 4) for
race/ethnicity groups across grades 3-8 reveal the
persistence of the achievement gap
Total Public
2013
37
Asian
Hispanic
Females
American
Indian/Alaskan Native
Males
34.9%
17.8%
24.9%
45.1%
Black
14.9%
20.5%
12.6%
19.8%
45.3%
55.6%
Across all race/ethnicity groups in grades 3-8, girls
performed better than boys on the ELA proficiency standard
(NYS Levels 3 or 4)
White
38
2009
51.9%
53.2%
52.8%
55.1%
77.4%
74.9%
75.0%
77.2%
31.1%
35.0%
22.7%
17.7%
Urban-Suburban
2010
61.5%
60.2%
62.4%
49.6%
47.5%
49.0%
43.1%
40.3%
42.0%
10.4%
26.4%
29.1%
27.8%
28.1%
Large City
84.2%
76.3%
70.9%
56.9%
68.8%
42.4%
43.9%
46.9%
New York City
91.8%
Across grades 3-8, lower-need communities continued to
outperform other areas of the State in ELA proficiency (NYS
Levels 3 or 4)
Rural
2011
Average
2012
Low
Total Public
2013
39
New York City
Buffalo
2009
2010
Syracuse
2011
53.2%
52.8%
55.1%
2012
16.4%
31.1%
39.2%
37.8%
40.7%
8.7%
25.5%
22.5%
24.2%
Rochester
77.4%
65.2%
52.7%
5.4%
25.3%
24.4%
20.7%
11.5%
27.7%
26.9%
27.9%
26.4%
42.4%
43.9%
46.9%
54.4%
56.0%
68.8%
A smaller percentage of students in grades 3-8 met or
exceeded the ELA proficiency standard (NYS Levels 3 or 4)
in the Big 5 cities than statewide.
Yonkers
Total Public
2013
40
31.1%
23.1%
Charter Schools
2009
55.1%
52.8%
53.2%
49.2%
43.9%
43.0%
76.1%
77.4%
English Language Arts 2009-2013
Charter Schools Comparisons
Grades 3-8 Combined
Percentage of Students Scoring at Levels 3 and 4
Total Public
2010
2011
2012
2013
41
2013 Grades 3-8
Math Results
EngageNY.org
In math, 31 percent of grades 3-8 students across the State
met or exceeded the proficiency standard (NYS Levels 3 or
4) in math, reflecting a new baseline relative to the Common
Core Standards
31.0%
61.0%
63.3%
64.8%
86.4%
The vertical lines indicate
years where changes
where implemented. In
2010, cut scores
changed, but the
standards and scale
remained the same. In
2013, the standards,
scale, and cut scores
changed to measure the
Common Core.
Grades 3-8 Math
2009
2010
2011
2012
2013
43
Grade 3
Grade 4
Level 1
Grade 5
Level 2
Grade 6
Level 3
Grade 7
7.2%
7.4%
20.2%
41.3%
31.2%
34.3%
20.3%
12.5%
18.1%
38.0%
40.5%
28.9%
8.9%
21.0%
30.3%
39.8%
12.9%
23.4%
34.7%
28.9%
12.3%
21.9%
35.4%
30.4%
In each grade level statewide, the majority of students
performed at NYS Levels 1 or 2 in math
Grade 8
Level 4
44
9.8 percent of English language learners met or exceeded
the math proficiency standard (NYS Levels 3 or 4) in grades
3-8
67.2%
65.9%
9.8%
32.7%
34.4%
32.3%
30.7%
63.5%
67.1%
87.9%
Beginning in 2013-14, data will be available for
students who received ELL services at any time
prior to test administration.
English Language Learners
Not English Language Learners
45
2009
2010
2011
2012
2013
35.5%
71.5%
70.0%
7.0%
28.5%
26.9%
24.6%
58.4%
67.7%
91.5%
7 percent of students with disabilities met or exceeded the
math proficiency standard (NYS Levels 3 or 4) in grades 3-8
Students with Disabilities
2009
General Education
2010
2011
2012
2013
46
Hispanic
2010
2011
American
Indian/Alaskan
Native
2012
White
31.0%
61.0%
63.3%
64.8%
86.4%
38.1%
20.9%
18.4%
Black
2009
49.5%
52.3%
53.8%
47.3%
50.2%
53.1%
40.9%
44.0%
46.1%
15.3%
Asian
92.2%
71.1%
73.3%
74.0%
81.6%
79.5%
75.0%
94.9%
81.7%
83.7%
85.4%
60.3%
The math proficiency results (NYS Levels 3 or 4) for
race/ethnicity groups across grades 3-8 reveal the
persistence of the achievement gap
Total Public
2013
47
Asian
Black
Hispanic
Females
American
Indian/Alaskan Native
Males
38.6%
21.5%
20.2%
18.7%
18.2%
14.2%
16.4%
37.5%
59.8%
60.8%
Results on the math proficiency standard (NYS Levels 3 or
4) in grades 3-8 were relatively comparable for girls and
boys across race/ethnicity groups
White
48
Large City
UrbanSuburban
2009
Average
2011
2012
86.4%
31.0%
61.0%
63.3%
64.8%
50.9%
32.3%
19.2%
Rural
2010
95.9%
80.8%
83.2%
84.1%
67.6%
69.7%
70.4%
54.3%
55.8%
56.6%
15.1%
9.0%
New York City
91.1%
85.8%
48.6%
49.1%
49.7%
31.1%
31.6%
32.5%
29.6%
54.0%
57.3%
60.0%
64.7%
81.0%
81.8%
Across grades 3-8, lower-need communities continued to
outperform other areas of the State in math proficiency
(NYS Levels 3 or 4)
Low
2013
Total Public
49
New York City
Buffalo
2009
Rochester
2010
Syracuse
2011
Yonkers
2012
2013
31.0%
61.0%
63.3%
64.8%
86.4%
14.5%
41.5%
40.4%
46.8%
73.8%
25.7%
25.3%
26.9%
6.9%
5.0%
28.0%
29.4%
27.3%
58.2%
63.4%
9.6%
29.8%
31.0%
29.9%
63.3%
29.6%
54.0%
57.3%
60.0%
81.8%
A smaller percentage of students in grades 3-8 met or
exceeded the math proficiency standard (NYS Levels 3 or
4) in the Big 5 cities than statewide.
Total Public
50
64.8%
63.3%
31.0%
31.3%
61.0%
68.7%
64.6%
59.9%
86.4%
89.4%
Mathematics 2009-2013
Charter School Comparisons
Grades 3-8 Combined
Percentage of Students Scoring at Levels 3 and 4
Charter Schools
Total Public
2009
2010
2011
2012
2013
51
Materials to Support Score
Interpretation and Use
Released
Annotated
Items
Performance
Level
Descriptions
Annotated
Score Report
Suggested
Data
Analyses
Available on EngageNY.org upon release of scores
EngageNY.org
52
Understanding the ELA
and Mathematics
Parent Reports
August 9, 2013
ELA and Math Parent Guides
EngageNY.org
54
Points to Consider
• How this information be used to help parents
understand the score reports?
• What information is provided to parents to
help them understand their child’s
performance beyond the Performance Level?
• Why is this new information valuable?
EngageNY.org
55
ELA Test- Parent Report
EngageNY.org
56
ELA Proficiency
Across Grades
EngageNY.org
57
EngageNY.org
58
Math Test- Parent Report
EngageNY.org
59
Math Proficiency
Across Grades
EngageNY.org
60
EngageNY.org
61
Domain Subscores
EngageNY.org
62
Points to Consider
• How can this information be used to help
parents understand the score reports?
• What information is provided to parents to
help them understand their child’s
performance beyond the Performance Level?
• Why is this new information valuable?
EngageNY.org
63
What is the Work?
Implementing the Common Core
Instructional Shifts Demanded by the Core
6 Shifts in ELA/Literacy
Balancing Informational and
Literary Text
Building Knowledge in the
Disciplines
Staircase of Complexity
Text-based Answers
Writing from Sources
Academic Vocabulary
6 Shifts in Mathematics
Focus
Coherence
Fluency
Deep Understanding
Applications
Dual Intensity
EngageNY.org
64
EngageNY.org
Resources for Professional Development
Parent and Family
Resources
Most relevant and
current information,
and newest materials
highlighted for easy
access.
One-stop location for
resources and
materials to support
implementation of the
Regents Reform
Agenda
EngageNY.org
44
Curriculum Modules
• Exemplary, comprehensive, optional, free
• High-quality, rigorous, deeply aligned to the Common
Core
• Address needs of students performing above and below
grade level, students with disabilities, and English
language learners
• Include performance tasks and other assessments that
measure student growth – daily, weekly, at the end of
each unit/module
• Ensure diversity of voices and perspectives in text
selection
• Contain notes for teachers, templates, handouts,
homework, problem sets, overviews
• Innovative creative commons license approach
EngageNY.org
66
Instructional Videos on EngageNY.org
EngageNY.org
67
Other Educator Resources
• Professional development videos developed with authors of
Common Core and PBS
• Tri-State / EQUiP rubrics to evaluate curricular materials
against the Common Core
• Curricular exemplars (sample lessons and instructional
materials) developed with feedback from the authors of
Common Core
• Grade- and subject-specific test guides and assessment
design information
• Sample assessment questions developed with feedback
from the authors of Common Core
• Network Team Institutes / Teacher & Principal Common
Core Ambassadors Program
EngageNY.org
68
68
Bilingual Common Core Progressions
• Analysis of the main
academic demand of
each standard
• Performance
indicators that
demonstrate how
students at each level
of language
progression meet the
standard using gradelevel text
• Analysis of the linguistic demand of each standard
• Scaffolds and supports that guide teachers for each proficiency
level
EngageNY.org
69
A New Baseline:
Measuring Student Progress on
the Common Core Learning
Standards
August 2013
EngageNY.org
Analysis of State
Test Data
Recommendations for Use
August 9, 2013
Materials to Support Score
Interpretation and Use
Released
Annotated
Items
Performance
Level
Descriptions
Annotated
Score Report
Suggested
Data
Analyses
Available on EngageNY.org upon release of scores
EngageNY.org
72
Suggested
Data
Analyses
EngageNY.org
73
Points to Consider
• What can I learn from this data?
• What types analyses will provide misleading
information?
EngageNY.org
74
EngageNY.org
75
Summary of
Recommended Uses
EngageNY.org
76
Recommended Use
1. ELA: Informational Reading Standards vs.
Literary Reading Standards
Your district may choose to create subscores to
compare the number of questions students
correctly answer related to these two types of
passages.
EngageNY.org
77
Recommended Uses
2. Math: Performance on each Major Cluster
Districts and schools may choose to compare
performance across major clusters.
EngageNY.org
78
Recommended Use
3. ELA or Math: Performance grouped
by Standard
Districts and schools may choose to compare
performance across individual standards.
EngageNY.org
79
Recommended Use
4. ELA or Math: Performance on Individual
Released Test Questions
Over reliance on inferences made from data at
the question level may lead to making shifts in
instructional emphasis that are not warranted.
EngageNY.org
80
Recommended Use
4. ELA or Math: Performance on Individual
Released Test Questions
Over reliance on inferences made from data at
the question level may lead to making shifts in
instructional emphasis that are not warranted.
EngageNY.org
81
Recommended Use
4. ELA or Math: Performance on Individual
Released Test Questions
Districts or schools may choose to report
information about performance on individual
questions.
Over reliance on inferences made from data at the
question level may lead to making shifts in
instructional emphasis that are not warranted.
EngageNY.org
82
Danger!
5. ELA or Math: Performance on Individual
Non-Released Test Questions
NYSED strongly discourages making inferences
– regardless of level of aggregation –
from single test questions that
were not released on EngageNY.org.
EngageNY.org
83
How to Use This Data
• Cross-reference standards encompassed in the
subscore with the curriculum used to teach
those standards.
• In identified growth areas, analyze curricular
materials using the Tri-State/EQIP Rubric and
the Publisher’s Criteria.
EngageNY.org
84
• Consider high-quality professional development
on the instructional shifts and the use of aligned
curriculum.
• Determine priorities for district goals based on
identified areas of growth.
• Consider professional development for teachers
targeted at the content addressed in standards
with weak student performance.
EngageNY.org
85
Summary of
Recommended Uses
EngageNY.org
86
Points to Consider
• What can I learn from this data?
• What types analyses will provide misleading
information?
EngageNY.org
87