Transcript Using Data to Make Decisions

The Nuts and Bolts of
Data Collection
and Analysis
Positive Behavioral
Interventions and
Supports
Joan Ledvina Parr
[email protected] 410-887-1103
OUTCOMES
PRACTICES
Process for Supporting Social Competence and
Academic Achievement
Supporting
Staff
Behavior
OUTCOMES
School-wide
Classroom
Non-classroom
Individual
Supporting
Decision
Making
Office Discipline Referrals
Staff Input
Academic Progress
Attendance
PRACTICES
Supporting
Student Behavior
Define behavior expectations
Specify routines
Teach
Acknowledge
Correct
Follow up and feedback
Reinforcement
Generalization
Are you….
• Swimming in data?
• Drowning in data?
• In a maze of data and can’t
find your way out?
• Lost and don’t know
what to do?
Data Needs to be Your Friend
Without data, you are just
another person with an
opinion…..
Data Can Be Your Friend
• Key Features of Data Systems that Work:
– Data are accurate
– Data are easy to collect
– Data are used for decision making
• Data must be available when decisions need to
be made
• People who collect the data must see the
information used for decision making
Objectives—to Help you Find your Way
• Review the types of data you will
want to consider
• Review how to access STARS data
and PBIS reports
• Learn how to use the Big 5 Generator
• Understand how to use data to
make decisions
Where to Begin……
Review End of the Year Data Reports
• Office referrals and suspensions
– PBIS reports in STARS
• PBIS Self-Assessment (Staff Survey)
• Benchmarks of Quality
– PBIS team completed
• Analysis of your School’s Resources
– Triangle of Tier 1, 2, and 3 interventions
• Final Step: Develop an Action Plan
• Bonus Step: Learn about the Big 5 Generator
What Data Should be Collected for
Decision Making?
• Office Discipline Referrals
– Measure of overall environment—
– Referrals are affected by:
• Student behavior
• Staff behavior
• Administrative context
• Teacher Referrals
– (Minors, Focus Room, etc.)
• Suspensions
• Attendance
Office Discipline Referral Process/Form
• Coherent system in place to collect office
discipline referral data
• Faculty and staff agree on categories
• Faculty and staff agree on process
• Office Discipline Referral Form should
include all needed information:
– Name, Date, Time, Staff Referring
– Problem Behavior, Location, Motivation
– Others Involved, Detailed Information as needed
STARS Discipline Module
• Data Entry
– Office Discipline Referrals (Majors)
– Teacher Referrals (Minors)
• Reports
– PBIS Reports: The Big 5
•
•
•
•
•
Location
Problem Behavior
Time of Day
Student
Referrals per Day by Month
– Variable Reports
STARS—3 of the Big 5 Reports
• Location
• Problem Behavior
• Time of Day
• Date Range
– From ……To
• Referral Type
– All (major and minor)
– Major
– Minor
• Show All Categories
with _____ or more
referrals
STARS—4th of the Big 5 Reports
• Student
• Date Range
• Referral Type
• Show All Students with
_____ or more referrals
• Sort on:
–
–
–
–
–
–
Student name
Age
Grade
Section
Major
Minor
Student Report—Factors to Sort on
•
•
•
•
•
Student Name—Lists students alphabetically
Age—Lists by age of students
Grade—Lists by grade of students
Section—Lists by section of students
Majors—Lists students’ names in order according to
their number of majors
• Minors—Lists students’ names in order according to
their number of minors
** You can limit your list by requesting only students with
_____ or more referrals or by asking for only majors or
only minors
STARS—5th of the Big 5 Reports
• Average Referrals per Day by Month
– Provides a chart with the number of days school
was in session, number of referrals (majors and/or
minors), as well as the calculation of Referrals per
Day by Month
The Big 5 Generator
•
•
•
•
Excel spreadsheet
Record STARS data by Month
Label each document by Month
Cut and paste graphs into document
– Location
– Problem Behavior
– Time of Day
– Students
– Average Referrals per Day by Month
Analysis:
What systems are problematic?
• Referrals by location?
– Are there specific problem locations?
• Referrals by problem behavior?
– What problem behaviors are most common?
• Referrals by student?
– Are there many students receiving referrals or only
a small number of students with many referrals?
• Referrals by time of day?
– Are there specific times when problems occur?
In
Ag app
gr ro.
es
sio Lan
n/ g
Di Figh
sr
es t
pe
ct
Ly
Ha ing
Di rra
sr ss
up
t io
n
Ta
rd
y
Pr
op
Sk
Fo Dam ip
rg
er age
y/
Th
ef
Dr t
es
In
s
na
pr Te
o.
ch
Ou Affe
c
t
Ga Bo t.
ng un
Di d s
sp
To lay
ba
c
Al co
co
ho
Dr l
Co ug s
m
bu
s
Bo t
m
b
Ar
W son
ea
po
ns
Ot
Un he
kn r
ow
n
Referrals by Problem Behavior
Office Disciplinary Referral Problem Behaviors
140
120
100
80
60
40
20
0
October
Cl
a
Pl ssro
ay o
g m
Ha C rou
ll/ om nd
Br m
ee on
Ba C zew s
t h af ay
/R ete
es ri
tro a
om
G
Bu Lib ym
s L ra
Pa oad ry
rk in
in g
gL
Sp
ot
ec
i B
Of al E us
f-C ve
a m nt
St pu
ad s
iu
Lo
ck Of m
O
Un th er fic
kn er Ro e
ow Lo om
n cat
Lo io
ca n
tio
n
Referrals by Problem Location
Office Disciplinary Referral Problem Locations
35
30
25
20
15
10
5
0
Seri es 1
Referrals by Time of Day
Office Disciplinary Referrals By time
10
9
8
7
6
5
October
4
3
2
1
0
7:00 7:30 8:00 8:30 9:00 9:3010:0010:3011:0011:3012:0012:301:00 1:30 2:00 2:30 3:00 3:30 4:00 4:30
What Does the Data Tell Us
If many students are making same mistake,
consider changing system….not students
Start by teaching, monitoring &
rewarding…before increasing punishment
Guidelines:
IF:
Focus On:
More than 35% of students receive 1 or more referrals
Average referrals per student > 2.5
Schoolwide
Systems
More than 35% of referrals come from non-classroom settings
More than 15% of students who receive a referral are referred from
non-classroom settings
Non-Classroom
Systems
More than 50% of referrals are from classroom settings
More than 40% of referrals come from less than 10% of the
classrooms
Classroom
Systems
If 10 or more students have 5 or more referrals
Targeted Group
Interventions
Less than 10 students with 10 or more office referrals
Less than 10 students continue w/referrals after targeted interv
Small number of students destabilizing overall functioning
Individual Student
Systems
Looking at the Overall Numbers
• What are the trends from year to year?
Create Graphs from the Big 5 Generator,
Excel or Power Point Applications
Office Referrals per Year
1200
1000
800
600
400
200
0
2004- 2005- 2006- 2007- 2008- 2009- 20102005 2006 2007 2008 2009 2010 2011
Looking at the Overall Numbers
• What are the trends within the year?
Create Graphs from the Big 5 Generator,
Excel or Power Point Applications
Average Referrals per Day by Month
35
30
25
20
15
10
5
0
A
S
O
N
D
J
F
M
Average Ref per Day SY 0809
A
M
J
Another View of Summary Data
• Consider the triangle percentages….
– Percentage of students receiving 0-1 referrals (green)
– Percentage of students receiving 2-5 referrals (yellow)
– Percentage of students receiving 6 or more referrals
Comparing Percentages of Students in
the Green, Yellow, and Red Zones
School Name
100%
90%
80%
10.0%
18.0%
70%
60%
50%
40%
30%
72.0%
School Name
100%
90%
4.0%
14.0%
80%
70%
Red
60%
Yellow
50%
Green
40%
30%
20%
20%
10%
10%
0%
0%
82.0%
Next Steps: Tracking Data
Comparing academic and behavior data
State-Wide
Assessment:
Basic
Borderline
Proficient
or
Advanced
Classroom
Performance:
Below
grade level
Approaching
grade level
On or
above
grade level
Discipline:
1-5%
1-5%
5-10% 5-10%
80-90%
80-90%
6+ referrals
2-5 referrals
0-1 referral
Days Absent
STUDENT
NAME
GRA
DE
Last
Report
Current
Days Tardy
Last
Report
Current
# Referrals
Last
Report
Current
CLASS
LEVEL
Read
ing
Math
MSA
Read
ing
Math
GRADES
1
2
3
4
#
Red Zone Students:
(6+ referrals)
M
a
j
o
r
s
Grade
#
Mi
no
rs
Total
ref
err
als
Suspensions
Interventions in place
PreK
0
6
6
0
Parent conference
K
1
5
6
1
SST, beh chart, Couns., gradual entry
1
3
18
21
3
IEP, FBA, beh chart, CICO,
3
0
7
7
0
CICO, SST, beh chart, couns.
3
0
10
10
0
CICO, beh chart,SST, couns
4
1
7
8
1
SST, FBA, CICO, Couns.
5
0
12
12
0
IEP, FBA, couns interv., beh chart
#
Yellow Zone
Students:
(2-5 referrals)
M
a
j
o
r
s
Grade
#
Mi
no
rs
Total
ref
err
als
Suspensions
Interventions in place
PreK
0
2
2
0
SST, 504, Couns. Interv.
K
0
3
3
0
Parent contact, SST requested
1
0
4
4
0
parent conference, beh chart
2
0
3
3
0
IEP, couns, beh chart
3
1
2
3
1
CICO, Couns., beh chart, referred to IEP
4
0
2
2
0
Parent contact
5
1
2
3
1
CICO, beh chart, parent contact
Analysis:
Review Results of Self-Assessment
• Staff completed the Self-Assessment
at the end of the year
• Note: change of website:
www.pbssurveys.org is now www.pbisassessments.org
• Generate the reports using the handout provided
– Individual Summary Report (one year)
– Comparative Summary Report (compares years)
– Analysis of Schoolwide System Report (critical
features)
– Individual Item Scores (item analysis)
Sample of Item Analysis
80
15
%
62
5%
%
36
%
59
2%
%
36
%
72
5%
%
28
%
41
0%
%
34
%
25
%
%
10. School administrator is an active participant on the behavior support team.
16
11. Data on problem behavior patterns are collected and summarized within an
on-going system.
13
12. Patterns of student problem behavior are reported to teams and faculty for
active decision-making on a regular basis (e.g. monthly).
14
13. School has formal strategies for informing families about expected student
behaviors at school.
19
14. Booster training activities for students are developed, modified, & conducted
based on school data.
22
20
%
64
%
34
%
53
%
34
%
%
52
%
32
%
%
49
%
45
%
%
%
33
%
%
Analysis:
Benchmarks of Quality
• Review the BOQ completed by your team in
April
• Use the BOQ Planning Form (in your
handouts) to guide your thoughts about
improvements needed
Consider your School’s Resources
• Complete the Triangle Activity (included in
your handouts)
• Consider what your school provides students at
the Tier 1, Tier 2, and Tier 3 levels
• Consider what your school might want to
implement to provide greater support for
students
INTENSIVE
TARGETED
UNIVERSAL
Groups targeting social skills, friendship, and/or anger
management skills
School health services Bullying Prevention
School counseling services
Second Step Student Intervention Plans Schoolwide PBIS Check-in/Check-out FBAs/BIPs
Section 504 Plans and/or IEPs
Behavioral contracting
Health Education Voluntary State Curriculum
Alternative programs School-based mental health services
Character Education
Expanded School Mental Health Initiatives and Interagency Partnerships
Have a Vision
(aka Your PBIS Action Plan)
When you know where you are headed, you can guide
students toward their own success.
Without a destination in mind and plan how to get
there, you may arrive at a place you don’t want to be.
Develop your School PBIS Action Plan
•
•
•
•
•
•
•
•
Use your data
Know your resources
Consider your plan for additional interventions
Map out your plan (PBIS Action Plan included in
your handouts—electronic version available)
Develop a schedule of meetings
Delineate roles and responsibilities
Plan for data analysis and discussion
Strengthen your team with multiple levels of
talent
Final Considerations
• We can’t “make” students learn or behave
• We can create environments to increase the
likelihood students learn and behave
• It is all about providing and supporting the
systems so that adults can change their behavior
to implement the practices that will bring about
OUTCOMES
change in student behavior
PRACTICES
Some Final Thoughts
on the Road to Success
All of us will have set-backs on the journey
Allow yourself plenty of time to get
there
Remember to bring the kids along
Remember,
Building a PBIS Continuum is a
Marathon not a Sprint
For Additional Information
• Joan Ledvina Parr
– PBIS Facilitator / School Psychologist
– [email protected]
410-887-1103
• Debely Fenstermaker
– PBIS Coach / School Psychologist
– [email protected] 410-887-7566
• Margaret Grady Kidder
– PBIS Coordinator / Coordinator of Psychological Services
– [email protected] 410-887-0303