School-wide Evaluation Tool (SET)

Download Report

Transcript School-wide Evaluation Tool (SET)

School-wide
Evaluation Tool (SET)
MN SW-PBIS Training
Kevin Filter
1
Virtual Training Materials
Please have the following materials printed
out and available during the training
session.
Set Evaluation Tool http://rainbowresearch.org/clients/pbis/doc
s/manual10_evaltool.pdf
Case Study 1 –
http://rainbowresearch.org/clients/pbis/doc
s/manual05_casestudy1.pdf
SET Overview
Purpose
1. Assess the extent to which the critical
features of SW-PBIS are in place in a
school

Results can serve as a basis for determining
annual goals in school
3
SET Overview
Purpose (continued)
2. Determine whether technical support has
resulted in changes in school adoption of
SW-PBIS
3. Serves as a partial basis for determining
whether adoption of SW-PBIS leads to
valued changes in school

Context for SWIS data
4
SET Overview
Scoring
 Each item is scored as a 0, 1, or 2 based on
information gathered as described below
 Seven sections with two to eight items each
5
SET Overview
Procedures and Data Sources
 Interview administrator (20 minutes)
 Interview teachers and team members (2-3
minutes each)
 Interview students (1-2 minutes each)
 Observe posted products in school
 Review permanent products
6
SET Scoring Guide Sections
7
Scoring Sections
A. Expectations Defined
B. Behavioral Expectations Taught
C. On-going System for Rewarding
Behavioral Expectations
D. System for Responding to Behavioral
Violations
8
Scoring Sections
(cont’d)
E. Monitoring and Decision-Making
F. Management
G. District-level Support
9
Collecting Data
10
Administrator Interview Guide
(20-30 minutes) (Page 4 of SET)
3 sections
 Make this a fluent conversation and record
responses as they come up
 Try to ask open-ended questions and narrow
questions if necessary
 This may also be a chance for the administrator to
brag… that’s a good thing!
 Don’t ask for details if some basic features are not
in place (“skip to” rules)
 Tip: ask who’s on PBS team
11
Administrator Interview Guide
Let’s talk about your discipline system
1) Do you collect and summarize office discipline referral information? Yes No If
no, skip to #4.
2) What system do you use for collecting and summarizing office discipline
referrals? (E2)
a) What data do you collect
b) Who collects and enters the data?
3) What do you do with the office discipline referral information? (E3)
a) Who looks at the data?
b) How often do you share it with other staff?
4) What type of problems do you expect teachers to refer to the office rather than
handling in the classroom/
specific setting? (D2)
5) What is the procedure for handling extreme emergencies in the building (i.e.
stranger with a gun)? (D4)
Administrator Interview Guide
Let’s talk about your school rules or motto
6) Do you have school rules or a motto? Yes No If no, skip to #
10.
7) How many are there?
8) What are the rules/motto? (B4, B5)
9) What are they called? (B4, B5)
10) Do you acknowledge students for doing well socially? Yes No
If no, skip to # 12.
11) What are the social acknowledgements/ activities/ routines
called (student of month, positive referral, letter
home, stickers, high 5's)? (C2, C3)
Administrator Interview Guide
Do you have a team that addresses school-wide discipline?
If no, skip to # 19
12) Has the team taught/reviewed the school-wide program with
staff this year? (B3) Yes No
13) Is your school-wide team representative of your school staff?
(F3) Yes No
14) Are you on the team? (F5) Yes No
15) How often does the team meet? (F6
16) Do you attend team meetings consistently? (F5) Yes No
17) Who is your team leader/facilitator? (F4)
18) Does the team provide updates to faculty on activities & data
summaries? (E3, F7) Yes No
If yes, how often?
Administrator Interview Guide
Do you have a team that addresses school-wide discipline?
If no, skip to # 19
19) Do you have an out-of-school liaison in the state or district to
support you on positive behavior support
systems development? (G2) Yes No
If yes, who?
20) What are your top 3 school improvement goals? (F1)
21) Does the school budget contain an allocated amount of
money for building and maintaining school-wide
behavioral support? (G1) Yes No
Transition to “wandering” stage
1. Gather permanent products
2. Find a place to finish the SET (a small
private room)
3. Transfer some information onto the
Interview and Observation Form (page 6)
Interview and Observation Form
(page 5 of SET)
 15 students and 10 staff (3 team members)
 Catch teachers and students in hall or other non-
disruptive area
 Select times when it will be easy to talk to
students and teachers (i.e., lunch)
 Introduce yourself and thank everyone!
 Look for posted materials during or after
interviews
12
Interview and Observation Form
(cont’d)
For all of the following, compare responses
to what administrator said

What are school rules?

What type of problem would you refer to
office?

Categories vs. examples

Procedure for a stranger with a gun

Who is team leader?
13
Interview and Observation Form
(Observation Items)
Documented crisis
plan posted
Rules & expectations
posted
What do they look like?
 Contents
What do they look like?
What does posted mean?
 Clearly visible
What does posted mean?
 Clearly visible
15
16
Permanent Products
 Ask administrator for these materials during
interview
 List of materials on page 0 of SET
 Hold on to them and review them when
scoring the SET (after all interviews)
 Best to review them at school in a staff
lounge or other semi-private space
18
Case Study #1
Case Study Exercise
(30 minutes)
 Use Case Example #1 to independently
review a case study
 After reading the case, complete a scoring
guide (the blank scoring guide is in the
complete SET download)
 Discuss the case study exercise as a whole
group when we come back together
20
Materials for the Case Study
Exercise
Set Evaluation Tool http://rainbowresearch.org/clients/pbis/doc
s/manual10_evaltool.pdf
Case Study 1 –
http://rainbowresearch.org/clients/pbis/doc
s/manual05_casestudy1.pdf
Case Study Exercise Review
Next Steps in Training
Reliability check
 Conduct SET with me or another reliable
evaluator
 Once reliable (85%), you are eligible to
conduct SETs
24
What’s Next
Establish reliability
Conduct SETs
Contact Pat Seppanen for co-observation
opportunities in your area:
[email protected]
Independently Completing
a SET Evaluation
Logistics of Completing a SET
SET Implementation Guide (Page 1 of SET)
 Make initial contact
 Give school the SET overview (page 1)
 Confirm date to conduct SET
 Administrator on site
 Answer questions related to SET
21
Site Visit Logistics and Tips
 Send a friendly reminder a few days before visit
 Encourage administrator to announce the SET visit
and logistics to staff on day of visit (avoid surprises!)
 When scheduling, make sure that you can begin with
20-30 minute administrator interview
 Bring a calculator
 Please avoid evaluating your own school
22
After the SET
 Log on to www.pbssurveys.org and complete
the SET on-line
 Copy and paste graph into report template
 Complete the report and give it to (a) the
school and (b) Rainbow Research within one
week
Report Template
Report Template
Report Template
What’s Next
Establish reliability
Conduct SETs
Contact Pat Seppanen for co-observation
opportunities in your area:
[email protected]
Contact
Pat Seppanen,
Rainbow Research
[email protected]
612-824-0724, ext 204
Kevin Filter
MN State Univ, Mankato
[email protected]
(507) 389-5828
23