Transcript Unified Improvement Planning: Preparing to Plan Webinar (School Level) Developed by : The Center for Transforming Learning and Teaching www.ctlt.org.

Unified Improvement Planning:
Preparing to Plan Webinar
(School Level)
Developed by : The Center for Transforming Learning
and Teaching
www.ctlt.org
Work Session Purpose
Ensure school planning teams
will have access to the
performance data they need to
engage in unified
improvement planning
Materials
• UIP Template (School-Level)
• UIP Handbook
• Inventory of Performance Data
• Configuring IE8 for the Data lab (Internet
Explorer)
• Organizing Data for Continuous Improvement
• UIP Fall Training Schedule
How this will work
• Sharing information
• Collecting your questions
• Specific times identified (mid-webinar) to
respond to questions that have come up
so far
• Recommended follow-up
Session Outcomes
Participate in
webinar.
• Describe the student performance data
needed to engage in unified
improvement planning (state and locally
available).
Access additional
resources.
• Describe the performance data needed
to identify trends in each performance
indicator area.
Complete followup activities.
• Identify local options for accessing
required state data views/ reports.
• Inventory locally available student
assessment results (and when they are
available during the year).
• Identify which views/reports to bring to
data analysis work sessions.
Agenda
Overview of
UIP data
needs
Identifying
Trends
Accessing
Performance
Data
Purposes of Unified Improvement Planning
• Align federal and state accountability systems.
• Provide a framework for performance management.
• Support school and district use of performance data to
improve system effectiveness and student learning.
• Shift from planning as an event to continuous
improvement.
• Give external stakeholders a way to learn about how
schools and districts are making improvements.
How will engaging in
unified improvement
planning result in
improvements in
performance?
Theory of Action: Continuous
Improvement
FOCUS
Monitor
Progress at
least
quarterly
State Performance Indicators
Achievement
Growth
Growth Gaps
Percent
proficient and
advanced
Normative
and CriterionReferenced
Growth
Growth Gaps
• Reading
(CSAP, Lectura,
and CSAPA)
• Writing (CSAP,
Escritura, and
CSAPA)
• Math (CSAP
and CSAPA)
• Science (CSAP
and CSAPA)
• CSAP Reading,
Writing and
Math
• Median Student
Growth
Percentiles
• Adequate
Median Student
Growth
Percentiles
Median Student
Growth
Percentiles for
disaggregated
groups:
•
•
•
•
Poverty
Race/Ethnicity
Disabilities
English Language
Learners
• Below proficient
Postsecondary
and Workforce
Readiness
Colorado
ACT
Graduation
Rate
Dropout
Rate
Colorado Unified Planning Template
Major Sections:
I. Summary Information about the school or
District
II. Improvement Plan Information
III.Narrative on Data Analysis and Root
Cause Identification
IV.Action Plan(s)
Unified Improvement Planning Processes
Gather and
Organize
Data
Review
Performance
Summary
Describe
Significant
Trends
Data Analysis
(Data Narrative)
Progress
Monitoring
Prioritize
Performance
Challenges
Identify Root
Causes
Set
Performance
Targets
Identify Major
Improvement
Strategies
Identify
Interim
Measures
Identify
Implementation
Benchmarks
Target Setting
Action Planning
Gathering and Organizing Data
• To support local efforts to
– Develop Unified Improvement Plans
– Monitor progress continuously (at least quarterly)
• What data?
– Required State Data Reports/Views
– Local Data Sources
• UIP Handbook, p. 5-6: Gathering and Organizing
Relevant Data
Multiple measures must be
considered and used to understand
the multifaceted world of learning from
the perspective of everyone involved.
-Victoria Bernhardt
Demographics
School
Processes
Provides information that
allows for the prediction of
actions, processes,
programs that best meet the
needs of all students.
Student Learning
Perceptions
Victoria Bernhardt
For what are multiple measures
used in UIP?
• Review current performance and prior year’s
targets
• Analyze data to identify trends
• Prioritize performance challenges
• Identify root causes
• Identify interim measures (and monitor changes
in student performance during the year)
• Identify implementation benchmarks (and
monitor implementation of action steps)
UIP Data Use
Types of data (intersections)
needed
Review current performance and prior year’s Performance data (intersected with
targets
demographic data)
Analyze data to identify significant trends
Performance data (intersected with
demographic data)
Prioritize performance challenges
Performance data (intersected with
demographic data)
Identify root causes of performance
Process and perception data (intersected
challenges
with demographic data)
Establish annual performance targets
Performance data (intersected with
demographic data) and state and local
expectations
Identify interim measures and monitor
Performance data (intersected with
changes in student performance.
demographic data)
Identify implementation benchmarks and
Process and perception data (intersected
monitor implementation of action steps.
with demographic data)
Inventory Local Performance Data
• Inventory of Performance Data Sources (spread sheet
included in the materials for this webinar).
• Components (see Legend)
–
–
–
–
–
–
–
Content Area
Assessment
Grade Levels
Which Students
Content Focus
Metrics
Questions
• Follow-Up: Complete an inventory of
performance data available to your school.
Your questions?
Agenda
Overview of
UIP data
needs
Identifying
Trends
Accessing
Performance
Data
Trends
• Include all performance indicator areas.
• Include at least three years of data.
• Consider data beyond that included in the
school performance framework (gradelevel data).
• Include positive and negative performance
patterns.
• Identify where the school did not at least
meet state and federal expectations.
Trends Could be:
Stable
Increasing
Decreasing
Increasing then decreasing
Decreasing then increasing
Stable then increasing
Stable then decreasing
Increasing then stable
Decreasing then stable
• Include:
–
–
–
–
–
–
Trend Statements
Measure/Metric
Content Area
Which students (grade-levels, disaggregated groups)
Direction
Amount
Time period
• Examples
– The percent of 4th grade students who scored proficient or
advanced on math CSAP declined from 70% to 55% to 48%
between 2009 and 2011.
– The median growth percentile of English Language learners in
writing increased from 28 to 35 to 45 between 2009 and 2011.
– Our dropout rate has been stable (15, 14, 16) and much higher
than the state average between 2009 and 2011.
Developing Trend Statements
What
measure/ What
Performance
data
content
Indicator
source? area?
Which
metric(s)?
Academic
Growth
Gaps
Colorado
Median
Growth
Growth
Model Reading Percentile
Academic
Growth
Colorado
Growth
Model Math
Which
Direction of
Which disaggregated
trend?
students?
groups? Comparison? Amount?
9th and
10th
graders
Students on
2008-09 to
an IEP
decreasing 55 to 45 2010-11
Median
Growth
Percentile 6th graders All students increasing
Students in
Academic
Percent
Middle
Growth Gaps Colorado
catch-up
School
disaggregated Growth
groups
Model Writing growth (grades 6-8)
Over what
time
period?
ELLs
2008-09 to
35 to 43 2010-11
Trend Statement
The median student growth percentile
reading for 9th and 10th graders on an
decreased from 55 to 45 between the 2
09 and 2010-11 school years.
The median student growth percentile
math for 6th graders increased from 35
43 between 2008-09 and the 2010-1
school year.
The percentage of middle school stude
receiving english language services mak
catch-up growth in writing was stable
stable then 26%, 28%,
between 2008-2009 (26% to 28%) an
increasing
40% 2008-2010 increased from 2009 to 2010 (28%, 40%
Your questions?
Agenda
Overview of
UIP data
needs
Identifying
Trends
Accessing
Performance
Data
Performance Data Sources
CDE
•
www.schoolview.org
– Data Center (graphs and charts) – 2011 data available August 3rd
– Data Lab (charts) – 2011 data available August 3rd
• requires changes in IE8 security settings
• Math, reading, writing (no science)
– The Colorado Growth Model (public and student-level data access)
•
PDF files
–
•
•
School Growth Summary Report
CEDAR (export into Excel)
Flat-files provided directly to district
District data reporting tools
Data Lab
• Requires changes in Internet Explorer
security settings
• On-line Tutorial on how to use it
• On-line FAQ helps address issues that
may arise
• To download files to excel you may need
to hold ctrl down while clicking on
download
Poll
• How do you access performance data for
your school?
Small N?
• What if summary reports have little or no data?
• CDE does not report data for small N to protect
student privacy.
• Options?
– Student-Level Data
– Summary statistics for smaller N
• Accessed through
– District data reporting tool
– Downloading student-level records from CEDAR
– The Colorado Growth Model web-based application
(student-level)
Accessing State Data
• Username and password required to:
– Download data files from CEDAR
– Access student-level data through the Colorado
Growth Model web-based application
• Who provides/changes usernames and
passwords?
– District assigned Local Access Manager (LAM)
– State does not provide or change school-level
usernames and passwords.
– Poll: Who has a username and password?
Notes for Alpine Users
• Users can select the N for summary statistics
(as low as 1)
• Users identify the “groups” of students for whom
they want to run reports.
• Training Videos include – how to create groups.
Your questions?
What performance data views/reports
do we need?
• How do we analyze data with action in mind?
• Would action steps target growth separate from
achievement?
• In what categories do we take action?
– Content areas (math, reading, writing, science)
– Disaggregated groups of students (low performing,
low growth, race/ethnicity, ELL, IEP. . .)
Levels of data/levels of challenges
System
Aggregated
Program
(Tier I)
Standard/
Sub-Content
Disaggregated group
Program
(Tier II/
Tier III)
Classroom
Student work
Individual
A path through the data. . .
Review the SPF Report to identify where performance did not
at least meet expectations (federal/state/local)
Select one content area on which to focus
Look for
and
describe
positive
and
negative
trends
Consider performance
(achievement/growth) by
grade level for 3+ years
Consider performance by
disaggregated group by grade
level for 3+ years
Within grade-levels consider
achievement by
standard/sub-content area
Disaggregate groups further
Look across groups
Consider cross-content area performance (3 + years)
Consider PWR metrics over 3+ years
Your questions?
Academic Achievement
• CSAP performance by grade level
– % proficient and advanced
– % and number scoring at each performance level
• Available from CDE:
–
–
–
–
Data Center (http://www.schoolview.org/performance.asp )
Data Lab (http://www.schoolview.org/performance.asp )
Download files from CEDAR
Flat-files provided to districts
• District data tool
Subject
Count Group
Math
Data Center
Grade
Ppy
Py
Cy
CyYr
Unsatisfactory
3
8.16%
6.90%
7.48%
2010
Math
Partially Proficient
3
30.61%
14.94%
16.82%
2010
Math
Proficient
3
43.88%
50.57%
48.60%
2010
Math
Advanced
3
16.33%
25.29%
27.10%
2010
Math
No Score
3
1.02%
2.30%
0.00%
2010
Math
Unsatisfactory
4
7.34%
5.68%
12.24%
2010
Math
Partially Proficient
4
41.28%
31.82%
29.59%
2010
Math
Proficient
4
42.20%
44.32%
44.90%
2010
Math
Advanced
4
7.34%
17.05%
12.24%
2010
Math
No Score
4
1.83%
1.14%
1.02%
2010
Math
Unsatisfactory
5
2.68%
13.89%
5.56%
2010
Math
Partially Proficient
5
50.89%
48.15%
36.67%
2010
Math
Proficient
5
31.25%
33.33%
35.56%
2010
Math
Advanced
5
15.18%
4.63%
21.11%
2010
Math
No Score
5
0.00%
0.00%
1.11%
2010
From the Data Lab
Academic Subject Grade
Year
Name
2008
Math
N Count Percent N Count Percent N Count Percent N Count Percent N Count Percent N Count Percent N Count
Proficienc Proficient Unsatisfa Unsatisfa PartialPro PartialPro Proficient Proficient Advanced Advanced NotScore NotScore Total
y
Advanced ctory
ctory
ficient
ficient
d
d
Grade 3
98
60.2
8
8.2
30
30.6
43
43.9
16
16.3
1
1
98
2008
Math
Grade 4
109
49.54
8
7.3
45
41.3
46
42.2
8
7.3
2
1.8
109
2008
Math
Grade 5
112
46.43
3
2.7
57
50.9
35
31.2
17
15.2
0
0
112
2009
Math
Grade 3
87
75.86
6
6.9
13
14.9
44
50.6
22
25.3
2
2.3
87
2009
Math
Grade 4
88
61.36
5
5.7
28
31.8
39
44.3
15
17
1
1.1
88
2009
Math
Grade 5
108
37.96
15
13.9
52
48.1
36
33.3
5
4.6
0
0
108
2010
Math
Grade 3
107
75.7
8
7.5
18
16.8
52
48.6
29
27.1
0
0
107
2010
Math
Grade 4
98
57.14
12
12.2
29
29.6
44
44.9
12
12.2
1
1
98
2010
Math
Grade 5
90
56.67
5
5.6
33
36.7
32
35.6
19
21.1
1
1.1
90
Academic Growth
• The Colorado Growth Model by grade level
– Median Student Growth Percentile
– % Catch-up, %Keep-up, %Move-up
• Available from CDE:
–
–
–
–
–
–
The Colorado Growth Model (web-version)
Data Center
Data Lab
School Growth Summary Report (pdf)
Download from CEDAR
Flat files provided to district
• District data tool
Schools within a District
Students in a Grade in a School
Standard/ Sub-Content Area
• CSAP Achievement by Standard or SubContent Area by grade-level
– % proficient and above
• Available from CDE:
– Flat files provided to district
– Download from CEDAR
• District Data Tools
Disaggregated Groups
• More detailed than the SPF/DPF
• SPF/DPF Disaggregated Groups:
– Minority
– Free/Reduced
– ELL
– IEP
– Below Proficient
Disaggregated Group Performance
• CSAP and Colorado Growth Model by grade-level
– Achievement: %proficient and advanced, % performing at each
level
– Growth: median student growth percentile, %catch-up/keepup/move-up
• Available from CDE:
–
–
–
–
–
–
School Growth Summary Report
The Colorado Growth Model (web-version)
Data Center
Data Lab
Download from CEDAR
Flat File provided to district
• District data tool
Academic
Year
Subject Grade
Name
Data Lab
ELL
N Count Percent Growth N Median Catchup Percent
Keepup
Percent
Proficienc Proficient Count
Growth Denomina Catchup Denomina Keepup
y
Advanced
Percentile
tor
tor
N<16
N<20
N<16
N<16
-
2008
Math
Grade 3
ELL
2008
Math
Grade 3
NON-ELL
90
61.11
N<20
-
N<16
-
N<16
-
2008
Math
Grade 4
ELL
27
33.33
26
29
N<16
-
N<16
-
2008
Math
Grade 4
NON-ELL
82
54.88
77
37
24
12.5
53
43.4
2008
Math
Grade 5
ELL
16
25
N<20
-
N<16
-
N<16
-
2008
Math
Grade 5
NON-ELL
96
50
91
48
29
20.7
62
40.3
2008
Math
Grade 6
ELL
40
45
37
44
17
11.8
20
40
2008
Math
Grade 6
NON-ELL
236
47.46
221
44
104
12.5
117
27.4
2009
Math
Grade 3
ELL
20
50
N<20
-
N<16
-
N<16
-
2009
Math
Grade 3
NON-ELL
67
83.58
N<20
-
N<16
-
N<16
-
2009
Math
Grade 4
ELL
N<16
-
N<20
-
N<16
-
N<16
-
2009
Math
Grade 4
NON-ELL
80
63.75
77
44
26
26.9
51
47.1
2009
Math
Grade 5
ELL
23
26.09
21
32
N<16
-
N<16
-
2009
Math
Grade 5
NON-ELL
85
41.18
79
27
35
2.9
44
34.1
2009
Math
Grade 6
ELL
N<16
-
N<20
-
N<16
-
N<16
-
2009
Math
Grade 6
NON-ELL
96
53.12
91
44
42
19
49
53.1
2010
Math
Grade 3
ELL
17
64.71
N<20
-
N<16
-
N<16
-
2010
Math
Grade 3
NON-ELL
90
77.78
N<20
-
N<16
-
N<16
-
2010
Math
Grade 4
ELL
23
30.43
22
20
N<16
-
N<16
-
2010
Math
Grade 4
NON-ELL
75
65.33
71
30
17
23.5
54
37
Disaggregating Disaggregated
Groups
• Minority (Asian, Black, Hispanic, Native American,
White)
• ELL (FEP, LEP, NEP, monitoring status)
• IEP (limited Intellectual capacity, emotional
disability, specific learning disability, hearing
disability, visual disability, physical disability,
speech/language disability, deaf-blind, multiple
disabilities, infant disability, autism, traumatic brain
injury)
Disaggregating Disaggregated Group
Performance
• CSAP and Colorado Growth Model by grade-level
– Achievement: %proficient and advanced, % performing at each
level
– Growth: median student growth percentile, %catch-up/keepup/move-up
• Available from CDE:
– Data Center (race/ethnicity only)
– Data Lab (race/ethnicity only)
– Download from CEDAR
– Flat File provided to district
• District data tool
Data Lab
Academic
Year
2008
2008
2008
2008
2008
2008
2008
2008
2008
2008
2008
2008
2008
2008
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2009
2010
2010
2010
2010
Subject
Name
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Math
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
Grade
3
3
3
4
4
4
5
5
5
5
6
6
6
6
3
3
3
4
4
4
4
5
5
5
5
6
6
6
6
3
3
3
3
Ethnicity
N Count
Proficiency
Asian
Hispanic
White
Asian
Hispanic
White
Asian
Black
Hispanic
White
Asian
Black
Hispanic
White
Asian
Hispanic
White
Asian
Black
Hispanic
White
Asian
Black
Hispanic
White
Asian
Black
Hispanic
White
Asian
Black
Hispanic
White
N<16
42
52
N<16
55
52
N<16
N<16
50
55
N<16
17
106
144
N<16
43
43
N<16
N<16
39
45
N<16
N<16
51
52
N<16
N<16
46
54
N<16
N<16
46
54
Percent
Proficient
Advanced
50
69.23
45.45
51.92
34
54.55
23.53
41.51
53.47
65.12
86.05
53.85
68.89
37.25
36.54
34.78
61.11
65.22
85.19
Growth N
Count
N<20
N<20
N<20
N<20
53
48
N<20
N<20
49
54
N<20
N<20
100
137
N<20
N<20
N<20
N<20
N<20
39
43
N<20
N<20
47
48
N<20
N<20
42
53
N<20
N<20
N<20
N<20
Median
Growth
Percentile
35
29
51
42
42
45
42
42
28
32
39
52
-
Catchup
Denominator
Percent
Catchup
Keepup
Denominator
Percent
Keepup
N<16
N<16
N<16
N<16
21
N<16
N<16
N<16
22
N<16
N<16
N<16
55
55
N<16
N<16
N<16
N<16
N<16
17
N<16
N<16
N<16
23
24
N<16
N<16
26
23
N<16
N<16
N<16
N<16
14.3
27.3
10.9
14.5
35.3
4.3
8.3
11.5
30.4
-
N<16
N<16
N<16
N<16
32
34
N<16
N<16
27
41
N<16
N<16
45
82
N<16
N<16
N<16
N<16
N<16
22
32
N<16
N<16
24
24
N<16
N<16
16
30
N<16
N<16
N<16
N<16
46.9
32.4
25.9
39
20
34.1
31.8
56.2
29.2
41.7
50
50
-
Post-Secondary and Workforce
Readiness
• Graduation Rate, Drop-out Rate, and
Colorado ACT Composite
• By disaggregated student groups
• Available from CDE:
– Data Center
– Download from CEDAR
– Flat File provided to district
• District data tool
Data Center
Entity
Ethnicity
Ppy
Py
Cy
CyYr
District
District
District
District
District
American
Indian or
Alaska Native
Asian
Black
Hispanic
White
0.33
0.90
1.00
0.60
0.70
0.67
0.89
0.70
0.60
0.80
0.71
0.90
0.65
0.65
0.80
2010
2010
2010
2010
2010
State
State
State
State
State
American
Indian or
Alaska Native
Asian
Black
Hispanic
White
0.57
0.83
0.64
0.56
0.82
0.56
0.86
0.64
0.58
0.82
0.50
0.82
0.64
0.56
0.80
2010
2010
2010
2010
2010
Organizing Data for Continuous
Improvement
• Resource: Organizing Data for Continuous Improvement
• Components:
– Major steps in identifying trends
– Measures and metrics
– Critical questions to “drill down” for each step
– Data reports/views that will allow planning teams to answer the
critical questions.
Your questions?
Give us Feedback!!
• Poll
– + the aspects of this webinar that you liked or
worked for you.
– The things you will change in your practice or
that you would change about this webinar.
– ? Question that you still have or things we didn’t
get to today about gathering and organizing
performance data
–
Ideas, ah-has, innovations