Transcript CTE Growth October 9

Assessment & Evaluation, October 2012
District Accountability Handbook
CDE – School and District
Accountability guidelines
http://www.schoolview.org/Region
alTraining.asp
SPF Technical Guide
http://www.schoolview.org/Perfor
manceFrameworks.asp
Video/training materials on SPF
http://www.cde.state.co.us/media/
training/SPF_Online_Tutorial/playe
r.html
Creating Meaning:
How will I recognize “notable change”
in performance over time?
Many factors can influence the interpretation of a difference between two numbers. Leaders must
consider all before ascribing noteworthy status to a change in a percent, mean, or median.
How Certain We
Need To Be
Statistical conventions in scientific research often call for high levels of certainty (90%-99%)
that require very large differences for “significance.” If we are satisfied being "fairly certain,"
then lesser changes can be regarded as “notable.”
Pre-Existing
Differences
Between Groups
Statistical comparisons begin with an assumption that groups to be compared are the same in
every way except for the intervention. If two groups begin with very different characteristics,
then differences in % P/A or growth may be due pre-existing difference.
Size of Comparison A difference of 5 percentage points can be significant when both groups being compared are
large but insignificant when the groups being compared are small.
Groups
Multiple Points
over Time
A 2% or 3% change may not be statistically significant from one year to the next. However, if
percent values change by 2 or 3 points over several successive points in time, then the overall
change may be significant.
General “rules of thumb” for identifying
percent P/A changes that may be significant
between a moderate to high level of certainty:
Approx.
Number of
Students
Approx.
Percent change
Most
Classrooms
Most
Elementary
Grades
Most Middle
School
Grades
Most High
School
Grades
District
Results
About 20-30
students
About 75-125
students per
grade
About 400
students per
grade
About 600
students per
grade
About 4000
students per
grade
6-12 percentage
points
4-8 percentage
points
2-5 percentage
points
1-3 percentage
points
13-20
percentage
points
These are approximate guidelines. The larger the difference, the more certain we can
be that it is not due to chance.
Achievement
Growth
Percent
Proficient and
Advanced
Actual
Growth &
Target Growth
• Reading (TCAP,
Lectura, and
CoAlt)
• Writing (TCAP,
Escritura, and
CoAlt)
• Math (TCAP and
CoAlt)
• Science (TCAP
and CoAlt)
• TCAP Reading,
Writing and
Math
• Median Student
Growth
Percentiles
• Adequate
Median Student
Growth
Percentiles
Gaps
Growth Gaps
Median Student
Growth Percentiles
for disaggregated
groups:
• Poverty (FRL)
• Race/Ethnicity
• Disability
• English proficiency
• Below proficient
Postsecondary
and Workforce
Readiness
Colorado ACT
Graduation
Rate
Dropout Rate
To define the type of plan your school must
create for submission to CDE
I.
II.
III.
IV.
Performance
Improvement
Priority Improvement
Turnaround
Performance
 All
Improvement
Priority
Improvement
Turnaround
Cherry Creek Schools were assigned to either
Performance or Improvement plans (most to
Performance).
SPF Sample
SPF Sample
•
•
•
CELA growth is
added to the
growth section.
Distinguish
between adequate
growth targets and
actual median
percentile growth.
The State defines
minority students
as “non-white.”
District KPIs
provide a better
picture of growth
for students of
color.
KEY CONCEPTS FOR UNDERSTANDING GROWTH
Actual Growth and Adequate Growth
Catching Up and Keeping Up
For Students
SGP – Student Growth Percentile
AGP – Adequate Growth Percentile target
For Schools
MGP -Median Growth Percentile
Median Adequate Growth target
 For each student, the State calculates a Student
Growth Percentile (SGP) for the current year
 The SGP is based on the comparison of a student’s growth to
growth in his or her State “academic peer group”
 SGPs are normative
 Some students’ SGP calculations use TCAP/CSAP data
from several years, while others reflect only 2 data
points
Office of Assessment & Evaluation
Student Pctls
45
78
99 Student
growth
32 percentiles
11
91
55
67
43
10
77
Sorted Pctls
Median Pctls
Median Growth for
this school is 55
Search for the
middle value…
Adequate Growth Percentiles: Are Students Growing Enough?
• For each student, the State projects growth paths in order
to calculate an AGP for the current year.
• The AGP is based on the calculated projection of how
much a student needs to grow.
 Students Below Proficient need to grow enough to “Catch Up,” or
become Proficient in 3 years or by Grade 10
 Students Proficient or above need to grow enough to “Keep Up,”
or remain Proficient or above for 3 years or until Grade 10
Each year for each student, the State projects the growth
necessary for that student to score at or above Proficient in 3
years or by grade 10
6th grade
7th grade
8th grade
9th grade
10th grade
76 is the minimum-this
student’s adequate
growth TARGET
Proficient
95
85
80
76
Not Proficient
2009
2010
2011
2012
2013
Each year for each student, the State projects the growth
necessary for that student to continue to score at or above
Proficient for 3 years or until grade 10.
6th grade
7th grade
8th grade
9th grade
10th grade
Proficient
12
25
38
50
50 is the maximum this student’s adequate
growth TARGET
Not Proficient
2009
2010
2011
2012
2013
What does the “Adequate Growth
Percentile” calculation mean for
classrooms with a high proportion of
very Unsatisfactory or low scoring
Partially Proficient students?
What does the AGP calculation mean for
classrooms with high proportions of
Advanced or high Proficient students?
• BObj growth reports offer administrators and instructional leaders
a variety of ways to explore and disaggregate CSAP/TCAP growth
results from the Colorado Growth Model.
• Most BObj growth reports are designed in a “crosstabular” form
that allows the user to view results by row for students in each
prior year’s CSAP/TCAP performance level (A, P, PP, US) and for all
students (bottom row).
Low, Typical, and High Growth
The percent of students with Student
Growth Percentiles (SGPs) in Low, Typical,
and High ranges, and at or above the 50th
percentile (the State median or average).
Average Growth
The median is the middle value
in a rank ordered list of SGPs for
this group of students. The
larger the group of students, the
more reliable the information
from this statistic .
Student Growth Percentiles (SGP)
from the Colorado Growth Model
tell us how a student grew
compared to other students in the
State with the same CSAP
performance starting point.
% Target Gain
The percent of students who demonstrate
Adequate Growth defined by CDE, or
growth sufficient to Catch Up (reach
Proficient in 3 years or by grade 10) or
Keep Up (stay at or above Proficient for
three years or until grade 10).
Understanding Cherry Creek School’s
The new federal racial/ethnic categories prompt a change in
district reporting of results for our two racial/ethnic groupings
(“White/Asian” and “Black/Hispanic/Native American”):
Students in the Multi-racial category will be grouped:
• White/Asian: Students with any combination of White, Asian, and/or
Pacific Islander
• Black/Hispanic/Native American Indian: Any students with a
racial/ethnic identification that includes Black and/or Native American
NOTE: Regardless of how many races are chosen, Hispanic
students are not considered multi-racial
KPI Reports
Achievement Status &
Achievement Growth
Equity Among Student
SubGroups
Post-Secondary
Readiness
CSAP achievement status
CSAP achievement status differences
Colorado ACT performance~
on most recent tests
5 points
most recent tests
10 points
10 points
CSAP achievement growth differences
CSAP achievement growth
15 points
over 2 most recent consecutive tests
30 points
High School graduation rates~ prior
year
10 points
45 total points
for growth
Excellence
Equity
College
Prep
• TCAP Status
• TCAP Growth
• TCAP Status Equity
• TCAP Growth Equity
• ES/MS: 3rd Reading/ EXPLORE
• HS: COACT, Graduation Rate
KPI Reports,
Summary
page
Total points are summed across all elements. The final percent of points earned is the result of
total points earned divided by the total points possible. For middle & elementary schools, the
total number of points possible is 70. For high schools, the total possible is 80.
KPI Reports
The State longitudinal growth percentile allows us to compare the average growth of our
students with average growth for students in the State with similar starting points.
ELEMENT POINTS=
% Rubric Pts x 30
RUBRIC POINTS-15 points possible per content area, Total Possible=45.
The objective is for
CCSD schools to meet
or exceed the State
median growth
percentile for students
with similar
performance starting
points (i.e., median
growth percentiles at or
above 50).
Reading
School
N
1,000
X
Y
1,093
725
Z
State
50
Median
50.0
56.0
52.0
Writing
Pts
10
10
10
N
998
1,114
724
Median
53.0
55.0
55.0
Math
Pts
10
10
10
N
1,003
1,111
721
Median
57.0
55.0
55.0
Pts
10
10
10
Rating
>=60th pctl
Exceeds
15
Meets
10
Rubric50th - 59th pctl
Rating
pctl
>=60th pctl35th - 49thExceeds
35th - 49th pctl
<35th pctl
30
30
30
Rubric*
Pts
GROW TH=Median Percentile
50th - 59th pctl
Total Rubric % of Poss
Element Pts
Pts
Rubric Pts
Earned
(45 Possible) (Total 45) (Total Poss 30)
<35th pctlMeets
Pts
Approaching
15
5
Does Not Meet
10
0
Approaching
5
Does Not Meet
0
Cherry Creek Schools Office of Assessment & Evaluation CSAP_Longitudinal Growth on Data Access System from CDE student data file
Results include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing,
or were new to the school after October 1.
66.7%
66.7%
66.7%
20.00
20.00
20.00
School
X
Y
Z
Excellence: CSAP Performance Indicator
How is this KPI similar to or different
from the SPF performance indicator? What
areas are “red” or “blue” (weakest or
strongest) on each report?
Excellence: CSAP Growth Indicator
How is this KPI similar to or different
from the SPF performance indicator?
Note any similarities or differences in
the degree to which your school meets or
exceeds growth expectation on both
systems. What areas are “red” or “blue”
(weakest or strongest) on each report?
SPF, KPI
Reports,
Notes Page
KPI Reports
READING RUBRIC POINTS-15 points possible within each Student Subgroup, Total 90 Rubric Points
Possible
The objective is for all student
subgroups in CCSD to meet or exceed
the State median growth percentile
for students with similar performance
starting points (i.e., median growth
percentiles at or above 50).
N
Pts
N
Median
SGP
HISPAN Pts
N
Pts
Total RD
Rubric
Points
(60 Poss)
School
N
X
602
49.0
5
82
55.0
10
138
51.0
10
173
48.0
5
30
Y
809
54.0
10
108
67.0
15
90
55.5
10
83
55.0
10
45
Z
225
48.0
5
40
60.0
15
196
54.0
10
261
52.0
10
40
State
50
Pts
Median
SGP
ASIAN
Median
SGP
BLACK
Median
SGP
WHITE
TOTAL ELEMENT POINTS
(Reading shown. Rubric Points totaled across Reading, Writing,
and Math.)
Rubric*
Rating
Pts
>=60th pctl
Exceeds
15
50th - 59th pctl
Meets
10
35th - 49th pctl
Approaching
5
<35th pctl
Does Not Meet
0
RUBRIC POINTS-15
points possible per content
area & student subgroup,
Total
Possible=180
Total
Rubric
% of
ELEMENT
POINTS=
% Rubric
Pts x 15Pts
Element
Pts Earned-All Possible
Content &
Rubric Pts
School Ethnic Groups (Total 180)
Earned
(Total Poss
15)
School
X
115
63.89%
9.58
X
Y
130
72.22%
10.83
Y
Z
130
72.22%
10.83
Z
Calculations included for all four ethnic groups regardless of numbers of s
across all grades & schools. Student program labels are tak
very small N's
Cherry Creek Schools Office of Assessment & Evaluation CSAP Longitudinal Growth on Data Access System from CDE student data file
Results include all students with valid CSAP scores in 2007 & 2008 EXCEPT students who participated in CSAP-Alternate, were expelled or withdrew during testing, or were new to the school after October 1.
Calculations are included for all four ethnic groups regardless of numbers of students. Native American Indian student groups are not summarized due to very small N's across all grades & schools.
Student program labels are taken from the CSAP test booklets and preprinted labels.
District
Performance
Plan
Creating Meaning:
What types of information
should I look for
on reports?
• Remember the fable of
the 6 men in the dark
trying to describe an
elephant?
• What do we need to see in
order to get a “complete
picture” of summary
performance and growth?
Levels of Analysis
Overall
More
Specific
Details
Views of Results
Overall
Content
Standards
Entire
School
Benchmarks
Grade
Level
Classroom/
Student
TCAP BObj reports provide several “lenses” through
which to examine school results. From different
views, you can create a complete picture of summary
school performance and growth.
Average/Median or overall %P/A.
Distributions.
Trends.
Disaggregations.
Comparisons
All Schools Change Report (pdfs emailed and on COLE)
and
5-Year Performance Summary (BObj)
5-Year Ethnic Summary (BObj) and
Disaggregated Summary (BObj)
Standards & SubContent Area Summaries
In what ways do the following report “picks” provide:
1. Multiple Views Of Summary Results
Overall %P/A, Distributions, Trends, Disaggregations, Comparisons
Levels of Analysis
Overall
2. Multiple Levels of Summary Analysis
Standards
Benchmarks
Performance for Subgroups