Transcript Breaking Down the Data: Developing a Deeper Understanding at the District Level Daniel J.

Breaking Down the Data: Developing a
Deeper Understanding at the District
Level
Daniel J. Losen
Independent Consultant
The Good News
 Data can be used to locate solutions as well as problems.
 Trend data can help identify educational leaders and policies
that have major impact.
 Non-special education data are particularly useful.
 The data analysis can be simplified for use at the district and
school level.
 Data can be used to address external explanations.
Problems
 Most districts do not have an abundance of educators who are





knowledgeable about analyzing racial and ethnic data.
Training Educators is important to expand capacity.
Keeping data simple is requires unpacking more complex
indicators used in state and federal monitoring.
Over-burdened data specialists.
Resistance to looking at and discussing racial data.
Privacy issues at the building level.
Recommendations:
Look at two or more years worth of data.
Look at disparities in key disability categories together.
Compare building level data.
Look at grade-level data (and compare at building level)
Disaggregate data by free and reduced lunch status.
Disaggregate by gender.
Emphasize differences.
Look at raw numbers first.
Look at risk.
Compare differences in risk more directly.
Look at several categories together (reading achievement alongside
suspensions, next to identification).
 Look at rates of referral to special education.











Data Analysis to Identify Problems and
Trigger Funds for Solutions
Principle #1: Look at the data in its simplest form first and ask basic
questions.
 Numbers: Are there high numbers of children of a given racial
group being identified.
 Unit of analysis: Percent = Risk: How many (per hundred) of the
enrolled Latinos are labeled as having SLD.
 Risk? If I picked a Latino child from this district at random, what is
the likelihood that he or she would be labeled SLD
 Ask: How does the risk for Latinos in my district compare?
 Explore different comparison groups.
Comparison Groups
 Compare to whites in the district.
 Compare to other groups in the district with similar socio-
economic status (i.e. Blacks and Latinos).
 Compare to state AND national averages for all students.
 Ask if this seems high? Low?
 Compare percentage point differences in risk value.
District’s Risk for “Mental Retardation” by
Racial/Ethnic Groups
0.8
0.8
0.7
0.6
0.6
Amer Ind
0.5
0.5
A/PI
0.4
Black
0.3
0.3
Latino
0.2
0.2
0.1
0
White
District’s Risk for Emotional Behavioral Disability
by Racial/Ethnic Groups 2004-05
2.5
2.2
2
1.8
Amer Ind
1.5
1.3
A/PI
Black
1
0.7
Latino
0.5
White
0.1
0
EBD
Before risk ratios:
What does risk mean?
 Native American children are over three times as likely as
White children to be labeled ED.
 Black and Latino children are about twice as likely to be
labeled ED as Whites.
 Asian American children have very low identification rates
for ED compared to all other groups.
 There were large differences between the racial groups but
were the rates “high?”
District’s Risk for Specific Learning Disability by
Racial/Ethnic Groups 2004-05
6
5.1
5
4.6
4.5
Amer Ind
4
2.8
3
A/PI
Black
2
Latino
1
White
1
0
LD
Q: Why is it important to analyze risk as well as risk ratio:
Risk ratios alone leave out important information.
What Does it Mean?
 About one in 20 Black, Native American and Latino children
identified for SLD…
 ….at almost twice the rate of Whites.
 Asian Americans seem to be identified at a very different
rate. Concerned?
 Each district and each group tells a different story.
What is Revealed?
 Far more children are identified as SLD.
 Some disability categories in the district have far more
children in them than others (compare MR with SLD).
 White and Asian identification rates are consistently lower in
these “subjective” categories.
Look at Identification For Different
Disability Categories Together
 Then add restrictiveness of educational setting, and
discipline.
 Then put those next to regular education data.
 Also compare to “medically diagnosed” disability categories.
Make Comparisons
 Broaden groupings
 Comparisons get easier to understand if made on a regular
basis
 Keep the method of comparison the same to the extent
appropriate
Principle # 2 Continued: Look At
Multiple Categories
 Look at many data sources before deciding that there is no




problem, especially where some data suggests a problem
exists.
Multiple data sources are better at building confidence in
results.
Look at patterns with other categories in regular
and special education.
Discipline? Pay close attention to the categories of
disciplinary action.
Don’t include data that are not revealing of
differences or necessary (i.e. not every category all
the time).
Why Discipline Data?
 Racial Disparities in Discipline is a required indicator for
IDEA monitoring and enforcement.
 As numbers go down for identification, or restrictiveness
of setting look to see if they go up in regular education
suspension and expulsion.
 Disciplinary exclusion from school can have a long-term
negative impact on academic outcomes.
Building Level Data: Look for Red Flags
Risk Comparisons
 Use graphs, as problems are often easy to spot visually.
 Principle # 2 Look at More Than One Category and More Than One
Area
 Analysis for action, not paralysis by analysis….
 Seek to understand the data connections.
Patterns of Racial Disparity in Indiana 2006-2007
(U.S. Dept of Ed.)
New Haven: Middle Schools
 J. Robinson Middle: 200 of 265 Black males suspended in 02



03
= 75% of the enrolled Black males.
Roberto Clemente Middle: 85 of 205 Latino Males:
= 41.5% of the enrolled Latino males
73.6% of Black males at Troup Middle School were
suspended at least once.
Principle # 3: Look at data over time
for trends and anomalies
 Did a policy change?
 Did leadership change?
 Did a dramatic change in one area correspond with a
dramatic change in a different area?
Chicago Male Suspension Rates By
Race Over Time (Source: Illinois State Board of Education)
Male Students Supsended
18.9
20
18
16
13.9
14
12
10
8.3
8
6
5.9
5.7
4.3
8
7.6
Black
3.9
Hispanic
4
2
0
2002
White
2003
Percent of Enrollment K-12
2004
Use Data to Find Solutions
 Look at simple data first. If educators at the school and district




level understand the data they are more likely to own the problem
but also may find a school leader with a solution.
You cannot dispel myths and misconceptions with fancy numbers.
The opportunity to show the connections to general education is
strengthened by looking at the data side by side.
The stronger the connection between special education and
regular education racial data, the harder it is to locate the problem
as a problem only within special education or outside the school.
The remedy may need to be a coordinated effort addressing
several areas. Discomfort with data could lead to remedies that
are more narrow than the context demands. (We only care about
the things we can count).
Use Data to Evaluate Interventions
 Process of continuous evaluation over time
 No clear programmatic solution – research on what works
 Qualitative factors matter, too
Getting and Using Data
 Simple analysis frequently repeated (Principle #1) will help
establish a sound practice.
 Comfort with looking at data on race, disability, and gender
is easier if the analysis remains similar.
Looking at More Categories Can Dispel Common
Misconceptions and Locate Solutions
 It’s not about race, just poverty.
 It’s not us, other districts identified them.
 Principle #2 Look at more categories and areas:
 i.e. test out the theories with the district’s data on poverty with race, and
on district’s rates compared to those entering already identified.
 Misconceptions linger when left unspoken or unaddressed.
 And exceptions are important to acknowledge.
 Teacher support: Going beyond the numerical disparities to resource
distribution.
Trends Over Time
Principle 3: Look at data over time for positive trends and to
continuously evaluate remedies.
 Finding buildings that are consistently successful (i.e.
principals)
 Providing more teacher support and training and then
tracking the results
Poverty and Race? (Actual District)
White #
%
Black #
%
Enroll White White Enroll Black Black
ment With With ment With With
Hisp. # H
Enroll With
ment
%H
With
Poor
222
58
26.1
290
97
33.5
243
23
9.5
Non
3188
328
10.3
136
40
29.4
98
12
12.2
Percent with Disabilities by Economic
Status by Race/Ethnicity
30
25
20
Asian/PI
Black
Hispanic
Am. Ind.
White
15
10
5
0
Not Poor
Poor
We have a problem,
we are trying to fix it, and
it’s hard work!
DPI EOCA Presentation
January 24, 2006
Presenter – Jack Jorgensen
Executive Director, Department of Educational Services
Madison Metropolitan School District
31
Comparing Schools With Similar
Demographics
 Similar demographics can suffice where differences appear
large.
 Classroom level data should be considered.
 Qualitative analysis is critically important.
 Grade level data are important, across schools.
Problems To Watch For
 Greater inclusion and lower risk for identification, but higher
rates of suspension.
 Dramatic changes in identification rates, and lower
achievement scores.
 Lower suspension rates, with lower attendance.
Looking at Rates of Referral
 Centralize referral process
 Helps improve quality of RTI and Early Intervening Services
 Initially high rates of referral resulted in high rates of
identification/elligibility.
 Then high rates of referral that resulted in non-identification.
 Ideally, fewer referrals for evaluation, but higher percentages
of referred identified.
 Madison Wisconsin example:
First – Defining The Problem
1998-1999 School Year
 A disproportionate number of students of color,
especially black males, were being referred to
special education (6% vs. 2% for white males).
 A disproportionate number of students of color,
especially black males were being placed into
special education (80% placement rate).
35
A Multi-faceted Response
 Aug. 1999 – Moved the responsibility for
conducting initial evaluations from schoolbased staff to a centrally coordinated IEP
system (CCIS)
 Oct. 2001 – Superintendent initiated the
development of an early intervention system
 Sept. 2002 – Initiated a study group on
disproportionality of minority students in
special education
 Sept. 2003 – District-wide conversations on race
and equity were initiated
36
Centrally Coordinated IEP System (CCIS)
 A centrally coordinated IEP system (CCIS) for
processing and completing initial special education
referrals of students ages 5-21
 Special Education program support teacher is
assigned to the IEP Team as: a) special ed. teacher, b)
IEP chairperson and c) LEA representative
 General education teacher and other staff (e.g.,
psychologist, social worker, nurse, etc.) were
appointed as appropriate from the student’s school of
attendance
37
Six Years of Data - Have we improved?
 Rate of referrals
 Rate of placements
 Looking at the data by ethnicity, socio-economic
status and gender
 How CCIS is impacting our district’s overall
prevalence rate
38
Centrally Coordinated IEP System
(CCIS)
39
The End
 Questions?
 My Contact Information:
 [email protected]
 [email protected]
 Cell: 617-285-4745
 Office: 781-861-1222
K-12 Referrals to Special Education
Total Number Referred and Proportion of Total Enrollment
600
548
500
400
2.2%
583
2.4%
3.0%
608
532
511
519
2.1%
2.1%
2.1%
496
2.5%
2.5%
Referrals as % of Student Enrollment
Number of Students Enrolled
700
2.0%
2.0%
300
1.5%
1.0%
200
100
0.5%
0
0.0%
98-99
99-00
00-01
01-02
02-03
03-04
04-05
CCIS
School Year
Total Students Referred
Referrals as a % of Total Enrollment
Poverty, Race and K-12 CCIS Referrals for 2004-05
African American
White
Afr Amer Low
Inc
25%
Afr Amer Not
Low Inc
75%
93%
120%
100%
80%
60%
40%
20%
0%
10%
White Low
Inc
White Not
Low Inc
90%
96%
97%
99%
Afr Amer
7%
4%
Low Inc Ref
3%
Low Inc Not Ref
1%
Not Low Inc Ref
Not Low Inc Not Ref
White
C C IS K -1 2 In itia l R e fe rra ls - P la c e m e n ts in to S p e c ia l E d u c a tio n
T o ta l N u m b e r a n d P ro p o rtio n o f P o p u la tio n R e fe rre d
700
100%
90%
600
80%
70%
500
64%
108
175
70%
64%
62%
61%
56%
219
60%
400
204
184
50%
205
216
300
40%
440
200
408
30%
389
328
327
314
280
20%
100
10%
0
0%
9 8 -9 9
9 9 -0 0
0 0 -0 1
C C IS
N o . o f R e fe rra ls P la c e d
0 1 -0 2
0 2 -0 3
0 3 -0 4
School Y ear
N o . o f R e fe rra ls N o t P la c e d
P la c e m e n t R a te
0 4 -0 5
P la c e m e n t a s % o f T o ta l R e fe rra ls
T o ta l S tu d e n ts R e fe rre d a n d P la c e d
80%
K-12 Placements as Percentage of Total Referred in Subgroup Ethnic/Racial
Subgroups
100%
90%
Percent of Total Subgroup
80%
70%
n=124
Afr Amer
60%
n=118
50%
n=25
40%
n=10
30%
20%
10%
0%
98-99
99-00
00-01
01-02
CCIS
School Year
02-03
03-04
04-05
Asian
Hispanic
White
Student Subgroup
Placement Rate Change
Between
1998-99 and 2004-05
White, Female, Low Income
-22%
White, Female, Not Low Income
-13%
White, Male, Low Income
-2%
White, Male, Not Low Income
-28%
African American, Female, Low Income
-22%
African American, Female, Not Low Income
-30%
African American, Male, Low Income
-15%
African American, Male, Not Low Income
-26%
Placements in Special Education
2004-05 School Year
S/L Only Initial Referrals Placed
(n=173)
17%
CCIS Pre-K Initial Referrals
Placed (n=189)
18%
CCIS K-12 Initial Referrals
Placed (n=280)
27%
In-State/Out-of-State Transfer
Students (n=396)
38%