Transcript Using Data for Decision-making Rob Horner, Anne Todd, Steve Newton, Bob Algozzine, Kate Algozzine.

Using Data for Decision-making
Rob Horner, Anne Todd, Steve Newton,
Bob Algozzine, Kate Algozzine
Goals

Define four roles for data use in a problem-solving
model

Define the key features of a problem statement

Define the process for identifying the data needed
for decision-making

Define the process by which data are used to
identify, refine, and problem-solve.
Main Ideas

Decisions are more likely to be effective and
efficient when they are based on data.

The quality of decision-making depends most
on the first step (defining the problem to be
solved)

Define problems with precision and clarity
Main Ideas

Data help us ask the right questions…they do
not provide the answers: Use data to




Identify problems
Refine problems
Define the questions that lead to solutions
Data help place the “problem” in the context
rather than in the students.
Main Idea

The process a team uses to problem solve is
important:

Roles:


Facilitator; Recorder; Data analyst; Active member
Organization




Agenda; Old business (did we do what we said we would
do); New business; Action plan for decisions.
What happens BEFORE a meeting
What happens DURING a meeting
What happen AFTER a meeting
Main Ideas


Build “decision systems” not “data systems”
Use data in “decision layers”


Is there a problem? (overall rate of ODR)
Localize the problem





(location, problem behavior, students, time of day)
Get specific
Don’t drown in the data
It’s “OK” to be doing well
Be efficient
?
Beh
Proble m
Location
?
Time
of
Day
Student
Setting
A
B
C
D
E
F
G
H
I
J
Loc ations
1
2
3
4
5
6
7
8
9 10
Tim es
A
B
C
D
E
F
G
H
I
Students
1
2
3
4
5
6
7
8
J
9 10
K
K
11
Using Data


Do we have a problem?
Refine the description of the problem?


Test hypotheses




What behavior, Who, Where, When, Why
“I think the problem on the playground is due to Eric”
“ We think the lunch period is too long”
“We believe the end of ‘block schedule” is used
poorly”
Define how to monitor if solution is effective
Identifying problems/issues

What data to monitor




What question to answer


Do we have a problem?
What questions to ask of Level, Trend, Peaks




ODR per day per month
OSS, ISS, Attendance, Teacher report
Team Checklist/ SET (are we doing what we planned to do?)
How do our data compare with last year?
How do our data compare with national/regional norms?
How do our data compare with our preferred/expected status?
If a problem is identified, then ask

What are the data we need to make a good decision?
Using Data to Refine Problem
Statement

The statement of a problem is important for teambased problem solving.


Everyone must be working on the same problem with the same
assumptions.
Problems often are framed in a “Primary” form, that
creates concern, but that is not useful for problemsolving.


Frame primary problems based on initial review of data
Use more detailed review of data to build “Solvable Problem
Statements.”
Solvable Problem Statements
(What are the data we need for a decision?)

Solvable problem statements include
information about the five core “W”
questions.





What is problem, and how often is it happening
Where is it happening
Who is engaged in the behavior
When the problem is most likely
Why the problem is sustaining
Primary versus Precision Statements

Primary Statements





Too many referrals
September has more
suspensions than last
year
Gang behavior is
increasing
The cafeteria is out of
control
Student disrespect is
out of control

Precision Statements

There are more ODRs
for aggression on the
playground than last
year. These are most
likely to occur during
first recess, with a large
number of students, and
the aggression is related
to getting access to the
new playground
equipment.
Primary versus Precision Statements

Primary Statements





Too many referrals
September has more
suspensions than last
year
Gang behavior is
increasing
The cafeteria is out of
control
Student disrespect is
out of control

Precision Statements

There are more ODRs
for aggression on the
playground than last
year. These are most
likely to occur during
first recess, with a
large number of
students, and the
aggression is related to
getting access to the
new playground
equipment.
Precise or Primary Statement?

Children are using inappropriate language
with a high frequency in the presence of both
adults and other children. This is creating a
sense of disrespect and incivility in the school

James D. is hitting others in the cafeteria
during lunch, and his hitting is maintained by
peer attention.
Precise or Primary Statement?

ODRs during December are higher than in any other
month.

Minor disrespect and disruption are increasing over
time, and are most likely during the last 15 minutes
of our block periods when students are engaged in
independent seat work. This pattern is most
common in 7th and 8th grades, involves many
students, and appears to be maintained by escape
from work (but may also be maintained by peer
attention… we are not sure).
Precise or Primary Statement?

Three 5th grade boys are name calling and
touching girls inappropriately during recess in
an apparent attempt to obtain attention and
possibly unsophisticated sexual expression.

Boys are engaging in sexual harassment
Organizing Data for Decision-making


Compare data across time
Moving from counts to count/month
Total Office Discipline Referrals
Total Office Discipline Referrals as of January 10
Change
Report Options
3.49
2.75
2.5
2.7
1.8
1.4
0
.00
SWIS summary 06-07 (Majors Only)
1974 schools; 1,025,422 students; 948,874 ODRs
Grade Range
Number of
Schools
Mean Enrollment Mean ODRs per
per school
100 per school
day
K-6
1288
446
.34 (sd=.37)
(1 / 300 / day)
6-9
377
658
.98 (sd=1.36)
(1/ 100 / day)
9-12
124
1009
.93 (sd=.83)
(1/ 107 / day)
K-(8-12)
183
419
.86 (sd=1.14)
(1/ 120 / day
Application Activity: Absolute Value
Is there a Problem?
Compare with national average:
Middle School of 625 students?
625/100 = 6.25
6.25 X .98 = 6.12
# per day
per 100 students
per School Day
Referrals
Office Discipline
Office Discipline Referrals per Day per Month per 100 Students
16
14
12
10
8
Series1
6
4
2
0
Sept
Oct
Nov
Dec
Jan
Feb
March
April
May
June
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Compare with National Average
Last year
150 / 100 = 1.50 Elementary
1.50 X .34 School
= .51 with 150 Students
2
1.5
1
0.5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Compare with National Average
1800 / 100
Last year
= 18 High18School
X .93 of= 1800
16.74
students
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
This Year
20
Middle School of 700 students
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Middle
School
Office Re fe rrals pe r Day pe r M onth
This Year
A v e R efer r als per D ay
N= 495
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
School Months
Mar
Apr
May
Is There a Problem? #2
Middle
School
Absolute - Trend - Compare
N= 495
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
T his year (Middle)
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Middle
School
N= 495
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Middle
School
N= 495
Office Re fe rrals pe r Day pe r M onth
A v e R efer r als per D ay
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
What are the data you are most likely to need to
move from a Primary to a Precise statement?

What problem behaviors are most common?


Where are problem behaviors most likely?


ODR per time of day
Who is engaged in problem behavior?


ODR per Location
When are problem behaviors most likely?


ODR per Problem Behavior
ODR per student
Why are problem behaviors sustaining?

No graph
What other data may you want?




ODR by staff
ODR by IEP
ODR by grade
ODR by gender by grade
Test precision problem statement

Use precision problem statement to build and
test hypotheses.




Problems are most common in D-Hall wing
Problems are more likely during second recess
Problems are most common during assembly schedule
Problems are more likely during state testing periods
What behaviors are problematic?
N um ber of R efer r als
Re fe rr als pe r Prob Be havior
50
40
30
20
10
0
L a n g Ac h o l Ars o n Bo m bCo m b sDe f i a nDi s ru p tDre s sAg g / f g tT h e f tHa ra s sPro p D Sk i p T a rd y T o b a c Va n d W e a p
Types of Problem Behavior
What behaviors are problematic?
N um ber of R efer r als
Re fe rr als pe r Prob Be havior
50
40
30
20
10
0
L a n g Ac h o l Ars o n Bo m bCo m b sDe f i a nDi s ru p tDre s sAg g / f g tT h e f tHa ra s sPro p D Sk i p T a rd y T o b a c Va n d W e a p
Types of Problem Behavior
What behaviors are problematic?
N um ber of R efer r als
Re fe rr als pe r Prob Be havior
15
10
5
0
L a n g Ac h o l Ars o n Bo m bCo m b sDe f i a nDi s ru p tDre s sAg g / f g tT h e f tHa ra s sPro p D Sk i p T a rd y T o b a c Va n d W e a p
Types of Problem Behavior
Where are the problems occurring?
N um ber of O ffic e R efer r als
Referrals by Location
50
40
30
20
10
0
B ath R B us A
B us
C af
C lass C omm
Gym
H all
School Locations
Libr
P lay G S pec
Other
Where are the problems occurring?
N um ber of O ffic e R efer r als
Referrals by Location
50
40
30
20
10
0
B ath R B us A
B us
C af
C lass C omm
Gym
H all
School Locations
Libr
P lay G S pec
Other
N um ber of R efer r als per S tudent
Who is contributing to the problem?
Referrals per Student
20
10
0
Students
N um ber of R efer r als per S tudent
Who is contributing to the problem?
Stude nts pe r Num be r of Re fe rrals
20
10
0
Students
When are the problems occurring?
N um ber of R efer r als
Re fe rrals by Tim e of Day
30
25
20
15
10
5
0
7 : 0 0 7 : 3 0 8 : 0 0 8 : 3 0 9 : 0 0 9 : 3 0 1 0 : 0 01 0 : 3 01 1 : 0 01 1 : 3 01 2 : 0 01 2 : 3 0 1 : 0 0 1 : 3 0 2 : 0 0 2 : 3 0 3 : 0 0 3 : 3 0
Time of Day
When are the problems occurring?
N um ber of R efer r als
Re fe rrals by Tim e of Day
30
25
20
15
10
5
0
7 : 0 0 7 : 3 0 8 : 0 0 8 : 3 0 9 : 0 0 9 : 3 0 1 0 : 0 01 0 : 3 01 1 : 0 01 1 : 3 01 2 : 0 01 2 : 3 0 1 : 0 0 1 : 3 0 2 : 0 0 2 : 3 0 3 : 0 0 3 : 3 0
Time of Day
Using Data to Build Solutions

Prevention: How can we avoid the problem context?



Who, When, Where
Schedule change, curriculum change, etc
Teaching: How can we define, teach, and monitor what we want?


Teach appropriate behavior
Use problem behavior as negative example

Recognition: How can we build in systematic reward for desired behavior?

Extinction: How can we prevent problem behavior from being rewarded?

Consequences: What are efficient, consistent consequences for problem
behavior?

How will we collect and use data to evaluate (a) implementation fidelity, and
(b) impact on student outcomes?
Solution Development
Prevention
Teaching
Reward
Extinction
Corrective Consequence
Data Collection
Examples

Phoenix Elementary

What is national comparison?





265/100 = 2.65
2.65 X .34 = .90
Absolute level compared with last year, compared
with teacher/staff impressions, compared with
family impressions, compared with student
impressions.
Where, what, when, who , why
Hypotheses?
Solutions
Phoenix Elementary
Using Data For Decision-Making
You are the EBS team for Phoenix Elementary.
265 students k-5





Do you have a problem?
Where?
With Whom?
What other information might you want?
Given what you know, what considerations
would you have for possible action?
M e a n S tu d e n t C o n ta cts p e r
Phoenix
Student DisciplineContacts
5
4
3
Year 2
Year 1
2
1
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
June
N u m b e r o f Stu d e n t C o n ta
Phoenix Elementary
Locations
140
120
100
80
Year 1
60
Year 2
40
20
0
Playgd
Class
Restrm
Location
Caf
Other
Phoenix Elementary ODR per Student
Major ODRs Year 2 Only
14
12
10
8
6
4
2
Students
49
46
43
40
37
34
31
28
25
22
19
16
13
10
7
4
0
1
Number of Student contacts
16
7:
00
7:
30
8:
00
8:
30
9:
00
9:
30
10
:00
10
:30
11
:00
11
:45
12
:15
12
:45
1:
15
1:
45
2:
15
2:
45
3:
15
Number of Referrals
Phoenix Elementary ODR per Time of Day
30
25
20
15
10
5
0
Time of Day
Problem Statement


Do we have a problem?
Build a precise problem statement
Solution Development
Prevention
Teaching
Reward
Extinction
Corrective Consequence
Data Collection
Trevor Test Middle School
565 students
Grades 6,7,8
Office Re fe rrals pe r Day pe r M onth
Re fe rrals pe r Prob Be havior
20
N um ber of R efer r als
A v e R efer r als per D ay
This Year
15
10
Lang.
5
0
Defiance
Sept
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Disruption
School Months
50
40
30
20
10
0
L a n g Ac h o l Ars o n Bo m bCo m b sDe f i a nDi s ru p tDre s sAg g / f g tT h e f tHa ra s sPro p D Sk i p T a rd y T o b a c Va n d W e a p
Re fe rrals by Location
Harrass Skip
80
Re fe rrals by Tim e of Day
60
40
20
0
B ath R B us A
B us
C af
C lass C omm
Gym
H all
Libr
P lay G S pec
School Locations
Other
N um ber of R efer r als
N um ber of O ffic e R efer r als
Types of Problem Behavior
50
12:00
40
30
20
10
0
7 : 0 0 7 : 3 0 8 : 0 0 8 : 3 0 9 : 0 0 9 : 3 0 1 0 : 0 01 0 : 3 01 1 : 0 01 1 : 3 01 2 : 0 01 2 : 3 0 1 : 0 0 1 : 3 0 2 : 0 0 2 : 3 0 3 : 0 0 3 : 3 0
Time of Day
Cafeteria Class Commons
Hall
N um ber of S tudents
Stude nts per Num be r of Re fe rrals
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Number of Referrals
Precise Problem Statement &
Hypothesis Development

Many students from all grade levels are engaging in
disruption, inappropriate language and harassment in
cafeteria and hallway during lunch, and the behavior
is maintained by peer attention

A smaller number of students engage in skipping
and noncompliance/defiance in classes, (mostly in
rooms 13, 14 and 18), and these behaviors appear to
be maintained by escape.
Solution Development
Prevention
Teaching
Reward
Extinction
Corrective Consequence
Data Collection
Solution Development:
For disruption in hall and cafeteria
Prevention
Teaching
*Teach behavioral expectations in
cafeteria
*Maintain current lunch schedule,
but shift classes to balance numbers.
Reward
Establish “Friday Five”: Extra 5 min
of lunch on Friday for five good
days.
Extinction
Encourage all students to work for
“Friday Five”… make reward for
problem behavior less likely
Corrective Consequence
Active supervision, and continued
early consequence (ODR)
Data Collection
Maintain ODR record and
supervisor weekly report
Langley Elementary
School
478 Students
K-5
Precision Statement/Hypothesis

What
Where
When
Who
Why
What other info needed?

Possible Solutions?





Solution Development
Prevention
Teaching
Reward
Extinction
Corrective Consequence
Data Collection
Sandhill High school
354 students
Sandhill High School: 354 students
Sandhill High School
Sandhill High School
Precision Statement/Hypothesis

What
Where
When
Who
Why
What other info needed?

Possible Solutions?





Solution Development
Prevention
Teaching
Reward
Extinction
Corrective Consequence
Data Collection