Transcript Advanced Data Based Decision Making

Advanced Data Based
Decision Making
Kimberly Ingram, Ph.D.
Professional Development Coordinator
Oregon Dept. of Education
February 2008
Southern Oregon PBS Network Conference
Agenda

Part 1

Look at your SET data – Is your Correction Feature at
80%
• If Yes, the next few slide will be very meaningful
• In No, the next few slides will be meaningful and we need to
discuss ways to enhance that Feature – “corrections Packet”

Examine Data Decision Rules for
• School-wide Interventions
• Targeted Group Interventions
• Individualized Interventions


Practice Data-based Decision-Making
Additional Data for further prevention efforts
• Ethnicity reports
• Special education

Part 2 – Prepare for First Day of School
CONTINUUM OF
SCHOOL-WIDE
POSITIVE BEHAVIOR
SUPPORT
Tertiary Prevention:
FBABSP for Students
with High-Risk Behavior
~5%
~15%
2-5 ODR
Primary
Prevention/Universal
Interventions:
School/ClassroomWide Systems for
All Students,
Staff, & Settings
Secondary Prevention:
Specialized Group
Systems for Students
with At-Risk Behavior
~80% of Students
0-1 Referrals
School-wide Positive Behavior Support
Supporting
Decision
Making
INFORMATION
Supporting
Staff Behavior
SYSTEMS
PRACTICES
Supporting
Student Behavior
Improving Decision-Making
From
Solution
Problem
Problem
To
Problem
Solving
Information
Solution
Using Data for On-Going
Problem Solving
Start with the decisions not the data
 Use data in “decision layers” (Gilbert, 1978)

Is there a problem? (overall rate of ODR)
 Localize the problem

• (location, problem behavior, students, time of day)

Get specific
Use data to guide asking of “the right questions”
 Don’t drown in the data
 It’s “OK” to be doing well
 Be efficient

Using Discipline Data
for Decision Making
Use Referral data to Inform
Intervention

In order to maximize school resources, it is
important to know where the majority of behavior
problems are occurring

Prevention measures, such as:
•
•
•
•

Re-teaching expectations
increasing supervision and monitoring
increased use of acknowledgments, or
environmental restructuring
are often the best interventions for misbehavior
especially when referrals are not successfully
addressing the problem
Using Discipline Data


There are many different data systems for tracking,
organizing, and presenting discipline data:
You can either make your current system work for
you, or


SWIS (School Wide Information System) is one of the
best systems for flexibility in manipulating data and
ease of presenting data to maximize the use of your
data
eSIS has some similar graphing abilities (Big 5, and a
few others). It is not as flexible as SWIS, however, it
can still offer excellent data for decision-making
Key features of data
systems that work.
The data are accurate and valid
 The data are very easy to collect (1% of staff
time)
 Data are presented in picture (graph) format
 Data are current (no more than 48 hours old)
 Data are used for decision-making

• The data must be available when decisions need to be
made (weekly?)
• Difference between data needs at a school building
versus data needs for a district
• The people who collect the data must see the
information used for decision-making.
Using Data for
Decision Making
PBS Teams use data for
School-wide, universal, interventions
 Targeted group, secondary,
interventions
 Individual, tertiary, interventions

School-wide Positive
Behavior Support
Systems
Classroom
Setting Systems
School-wide
Systems
SW v. Individual
Majors + Minors
Majors Only
#
%
#
%
1-2
89
20%
44
10%
3-5
27
6%
10
2%
>5
30
7%
4
1%
What about CLEO?
12 By Dec. 2000 – Jun. 2001
 19 By Sep. 2001 – Dec. 2001

Suspensions/Expulsions Per Year
2000-01
2001-02
Events Days Events Days
In School Suspensions
Out of School Suspensions
Expulsions
0
1
0
0
1
0
2
3
0
2
2.5
0
CLEO: # By/Day/Month
CLEO: # By by Type
CLEO: # BI by Location
1. School-wide systems if…

Elementary:
> 1 ODR per day per month
per 300 students (majors
only)

Middle:
> 1 ODR per day per month
per 100 students (majors
only)


>40% of students received
1+ ODR
>2.5 ODR/student

Modify universal
interventions
(proactive schoolwide discipline) to
improve overall
discipline system

Teach, precorrect, &
positively reinforce
expected behavior
SWIS summary 06-07 (Majors Only)
1974 schools; 1,025,422 students; 948,874
ODRs
Grade
Range
Number of
Schools
Mean
Mean ODRs
Enrollment per 100 per
per school school day
K-6
1288
446
.34 (sd=.37)
(1 / 300 / day)
6-9
377
658
.98 (sd=1.36)
(1/ 100 / day)
9-12
124
1009
.93 (sd=.83)
(1/ 107 / day)
K-(8-12)
183
419
.86 (sd=1.14)
(1/ 120 / day
SWISTM summary 05-06 (Majors
Only)
1668 schools, 838,184 students
Grade
Range
Number of
Schools
Number of
Students
Mean ODRs
per 100 per
school day
K-6
1010
439,932
.37 (sd=.50)
Mean = 435
6-9
9-12
K-(8-12)
312
104
239
205,129
1.01
Mean = 657
(sd=1.06)
102,325
1.16
Mean = 983
(sd=1.37)
90,198
1.09
Mean = 377
(sd=1.56)
Interpreting Office Referral
Data:
Is there a problem?

Absolute level (depending on size of school)
Middle, High Schools (> 1 per day per 100)
 Elementary Schools (> 1 per day per 300)


Trends
Peaks before breaks?
 Gradual increasing trend across year?


Compare levels to last year

Improvement?
Change
0 Report Options
1.44
1.48
1.25
1.40
0.92
0.94
0.80
0.25
0.50
0.45
0.00
Elementary School with 250 students
Average Referrals per Day per Month
Middle School of 600 students
Change
11Report Options
8
9
4
6.2
4.5
0
0.00
13
14
11
12
Middle School with 500 students
Change
Report Options
0 Middle
14
6.00
3.00
0.00
school with 500 students
Change
51
8
7
15
1
9
0
14.00
6.00
3.00
0.00
12
0Report Options
3
Change
5.5
6.00
3.00
12.00
11.00
9.00
13.00
8.00
15.00
10.00
7.00
5.00
0.00
2
0
3
4
.3Report Options
Middle School with 500 students
Is there a problem?
Change
Report Options
4.20
6.20
3.40
0.00
Middle school with 500 students (Dec)
Is there a problem?
0.00.00
0
Middle School with 500 students
Is there a problem?
0
4.20
6.20
3.40
15.00
9.00
13.00
8.52
8.00
7.00
0.00
Middle School with 500 students (Dec 04-05)
Is there a problem?
Change
0 Report Options
9.20
11.50
7.20
6.20
3.40
15.00
9.00
13.00
6.52
8.00
7.00
0.00
Middle School with 500 students (Feb 3, 04-05)
2. Classroom system if…



>60% of referrals come
from classroom
>50% of ODR come
from <10% of
classrooms
Several teachers not
writing referrals at all

Enhance universal
&/or targeted
classroom
management
practices


Examine academic
engagement &
success
Teach, precorrect
for, & positively
reinforce expected
classroom behavior
& routines
3. Non-classroom systems if…


>35% of referrals
come from nonclassroom settings
>15% of students
referred from nonclassroom settings

Enhance universal
behavior
management
practices


teach, precorrect for,
& positively reinforce
expected behavior &
routines
increase active
supervision (move,
scan, interact)
Referrals by Location
Change
12
3
7
5
4
9
0
1
2
0Report Options
1
4
Change
84Report Options
38
0
1
3
2
4
7
Middle School
Change
1 Report Options
3
0
2
4
9
24
23
31
12
Elementary School
Referrals by Time
4. Targeted group interventions
if….

>10-15 students
receive >5 ODR

Provide functional
assessmentbased, but groupbased targeted
interventions

Standardize &
increase daily
monitoring,
opportunities &
frequency of positive
reinforcement
5. Individualized action team
system if...


<10 students with >10
ODR
<10 students continue
rate of referrals after
receiving targeted
group support

Provide highly
individualized
functional-assessmentbased behavior
support planning
Referrals by Student
Student Referral Report
Staff
Time
Location
Problem
Behavior
Motivation
Others
Involved
Admin
Decision
1 03/10/2004
43866
12:15PM
Plygd
Agg/Fight
Unknown mot
Peers
Parent
2 03/01/2004
62390
12:30PM
Unknown
loc
Disrespt
DK
Peers
Parent
3 02/10/2004
47522
01:30PM
Class
Agg/Fight
Unknown mot
UnknownO
Office AD
4 12/18/2003
47522
10:30AM
Class
Agg/Fight
Unknown mot
Peers
Out-sch
susp
5 12/08/2003
47522
10:00AM
Class
Other
behav
Unknown mot
None
Out-sch
susp
6 12/08/2003
47522
01:15PM
Class
Disrespt
Ob p attn
None
Office AD
7 11/20/2003
62390
10:00AM
Unknown
loc
Agg/Fight
Unknown mot
Peers
Out-sch
susp
8 11/20/2003
47522
10:30AM
Class
Agg/Fight
Unknown mot
Peers
Out-sch
susp
Date
School Example
A middle school getting ready to
implement targeted group
interventions. They had been
implementing school-wide
interventions for one school year.
Some Questions ABC Middle
School had re: student needs
How many students in the middle of the triangle?
 How many need at the top of the triangle?
 How many students in the targeted group have 2,
3, 4, 5, thru 25 referrals?
 What types of behaviors are targeted group
students and tip of triangle students engaging in?
 What percent of students in targeted and tip are
Sped?
 What percent of students in targeted and tip met
AYP the previous year?

ABC Middle School
541 students
 1314 total number of referrals for SY 0405
 Pre and Post Set completed
 Team attended 4 PBS trainings
throughout year and implemented along
the way
 Team leader attended district leadership
meeting consistently throughout year

le
dg
em
en
t
Features
M
ea
n
Co
rre
ct
io
ns
Ev
al
ua
tio
n
M
an
ag
em
en
Di
t
st
r ic
tS
up
po
rt
Ac
kn
ow
ta
ug
ht
de
f in
ed
Ex
pe
c
Ex
pe
c
% of features implemented
SET - Gardiner M.S.
100
80
60
Pre - Fall '04
40
Post - Spring '05
20
0
Triangle Data
0-1 referral: 381 (65% of students)
 2-5 referrals: 124 (21% of students)
 6+ referrals: 82 (14% of students)

Break down of all referrals (1314) by
behaviors










Disrespect: 354
Disruption: 310
Tardy: 274
Inappropriate Language: 80
Fighting/Aggression: 73
Skip: 56
Harassment: 37
Theft: 28
Other: 82
Miscellaneous (drugs, lying, prop. damage,
weapons, vandalism): 20
‘At-Risk’ Group = 2-5 referrals
409 referrals generated by this group
(31% of all referrals)

Students with
2 referrals: 46
 3 referrals: 37
 4 referrals: 22
 5 referrals: 19

Total
_______
124 students [12 (10%) on IEPs]
Sample of Behaviors from ‘At-Risk’
Group – 8 students total, 4 boys/4 girls, 2
on IEPs,
Tardy: 9
 Disruption: 8
 Disrespect: 3
 Other (gum chewing): 3
 Aggression/Fighting: 1
 Combustible: 1
 Tobacco: 1
 Weapons: 1
 Drugs: 1
_________
Total = 28 referrals

‘Tip of Triangle’ – 6+ referrals
859 referrals generated by this group
(65% of all referrals)

Students with







6 referrals: 11
7 referrals: 11
8 referrals: 9
9 referrals: 9
10 referrals: 11
11 referrals: 4
12 referrals: 7

Students with







13 referrals: 3
14 referrals: 5
15 referrals: 5
17 referrals: 3
18 referrals: 2
19 referrals: 1
24 referrals: 1
_________
Total = 82 students
[18 (22%) on IEPs]
ABC MS –
SPED Students and AYP

78 students in special education (14% of
student body)

39 students (50%) of sped students with 2
or more ODRs
• 38 students (97%) of sped students with 2 or
more ODRs did not meet AYP in 1 or more
subjects
Additional Information

Ethnicity Reports


Available on SWIS and eSIS
Special Education
Ethnicity Reports

Rationale
The power of information
 The risks and ethics of dis-proportionality


Format
Multiple reports are needed for decisionmaking
 SWIS currently provides the numbers and
output.

Ethnicity Reports

Key Questions

What proportion of enrolled students in
school are from each ethnicity?

What proportion of referrals are contributed
from students in each ethnicity?

What proportion of students with at least one
referral are from each ethnicity?

What proportion of students within each
ethnicity have received at least one office
discipline referral?
Ethnicity #1
Ethnicity #2
Ethnicity #3

Data are good…but only as good as
systems in place for
PBS
 Collecting & summarizing
 Analyzing
 Decision making, action planning, &
sustained implementation

Monthly E-mails (December)
Dear Staff,
Thought I’d send this along before we go home ‘till
1999.
Through 11/30/98 there were 179 referrals involving
62 students (6.7%). 858 students (99.3%) have no
referrals.
27 students (2.9%) are responsible for 80% of all
referrals through 11/30. The top 13 have earned 59%
of the referrals.
Thank you for your efforts this fall in helping to carry a
positive surge in momentum through the year’s end.
Have a refreshing break.
Happiest Holiday Wishes!
Monthly E-mails (February)
Ever have that feeling like you wondered if someone had gotten
the license plate of the truck that hit you? February had a bit of
that feel to it. Approximately 1/3 of the year’s referrals to date
(143 out of 457) took place in February…In perspective, the
month was truly out of character with the rest of the year. Thank
you for your perseverance.
85% of our students continue their good work and have no
referrals.
The 457 referrals (9/98-2/99) are down 22% from the 581
referrals last year.
In April we will be seeking staff input through our EBS survey to
help build a focus for next school year. Keep up your good work-
Summary
Transform data into “information” that
is used for decision-making
 Present data within a process of
problem solving.

Use the trouble-shooting tree logic
 Big Five first (how much, who, what, where,

when)
Data should be collected to answer
specific questions
 Ensure the accuracy and timeliness of
data.
