Transcript Slide 1

Maryland’s State Implementation
Strategies, Successes, and Outcomes
www.pbismaryland.org
Susan Barrett
[email protected]
1
ACKNOWLWEDGEMENTS
• Maryland State Department of
•
•
•
•
•
Education
Sheppard Pratt Health System
Johns Hopkins University
24 Local School Systems
University of Oregon
University of Connecticut
2
Pennsylvania
D.C.
3
494
500
369
400
300
190
200
100
283
276
150
125
68
35
42
61
30
133
183
81
0
FY
99
FY
00
FY
01
FY
02
Schools Trained
FY
03
FY
04
FY
05
FY
06
Coaches Trained
4
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
5
Political Support
PBIS was part of the top ten MSDE state supplemental budget
request for FY’ 00-FY’ 07.
Section 7-304.1 Of the Annotated Code of Maryland
•
•
•
In this section, “Positive Behavioral Interventions and Support Program”
means the research-based, systems approach method adopted by the
State Board to build capacity among school staff to adopt and sustain the
use of positive, effective practices to create learning environments where
teachers can teach and students can learn.
Each county board of education and the Board of School Commissioners
of Baltimore City shall require an elementary school that has a
suspension rate that exceeds 18 percent of the elementary school’s
enrollment to implement:
–
A Positive Behavioral Interventions and Support Program; or
–
An alternative behavioral modification program in collaboration with
the Department.
The State Board shall adopt regulations to implement the provisions of
this section.
6
Section 7-304.1 Of the
Annotated Code of Maryland
(a) In this section, “Positive Behavioral Interventions and Support Program”
means the research-based, systems approach method adopted by the
State Board to build capacity among school staff to adopt and sustain
the use of positive, effective practices to create learning environments
where teachers can teach and students can learn.
(b) Each county board of education and the Board of School
Commissioners of Baltimore City shall require an elementary school
that has a suspension rate that exceeds 18 percent of the elementary
school’s enrollment to implement:
(1) A Positive Behavioral Interventions and Support Program; or
(2) An alternative behavioral modification program in collaboration with
the Department.
(c) The State Board shall adopt regulations to implement the provisions of
this section.
7
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
8
State Leadership Team
Visibility:
MSDE Marketing Department is developing a tiered
marketing/advertising plan targeting key PBIS
stakeholders to generate more awareness for the
initiative.
Target Audiences: Parents and Community
Members/Leaders, District Leaders, Instructional
Leaders, State Leaders, Teachers, Students
Politicians
9
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
10
State Leadership Team
Funding:
As the PBIS initiative prepares for a largescale expansion, we are in progress on a
framework to identify and secure a broad
base of funding support to ensure the
infrastructure required to maintain the
consistency and quality of technical assistance
and evaluation support to districts.
11
Funding
• PBIS has been one of the top ten line
items of the Maryland State Department of
Education supplemental budget request
FY’ 00- FY’ 07.
12
Funding -Reality
• Title IV-Safe and Drug Free Schools
• Special Education
• HIV prevention
• Drop Out Prevention
• Sheppard Pratt
13
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
14
Schools Trained and Active
140
126
126
120
94
85
100
91
67
80
60
54 51
60
30
40
20
49
20 12
17
14
3
0
1999
2000
2001
2002
2003
Trained
Active
2004
2005
2006
15
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
16
State Leadership Team:
Diversified
•
•
•
•
•
•
•
•
•
MSDE Division of Special Education/ Early Intervention Services
MSDE Division of Student and School Services
Sheppard Pratt Health System
Johns Hopkins University
LSS Behavior Support Coaches
Juvenile Justice
Governor’s Office of Children
University of Maryland
Department of Health and Mental Hygiene
Management Team-Core Group
MSDE, JHU, Sheppard Pratt
17
Mental Health Integration Grant
• MARYLAND STATE DEPARTMENT OF EDUCATION
• THE UNIVERSITY OF MARYLAND, BALTIMORE, ON BEHALF
•
•
•
•
•
•
•
OF ITS CENTER FOR SCHOOL MENTAL HEALTH ANALYSIS
AND ACTION
DEPARTMENT OF JUVENILE SERVICES
GOVERNOR’S OFFICE FOR CHILDREN
MENTAL HYGIENE ADMINISTRATION
THE MENTAL HEALTH ASSOCIATION OF MARYLAND
THE MARYLAND COALITION OF FAMILIES FOR CHILDREN’S
MENTAL HEALTH
MARYLAND ASSEMBLY ON SCHOOL-BASED HEALTHCARE
SHEPPARD PRATT HEALTH SYSTEM
18
State Leadership Team
Functions
• Set policy
• Provide support for local leadership
• Influence System of Change at District
•
•
•
•
Level
Assess Training Needs
Event Coordination
Provide Training and Technical Assistance
Monitor Outcomes
• Features of implementation
• Referrals
• Other indicators
19
Building Local Capacity
Roles and Responsibilities
20
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
21
Behavior Support Coaches
• 283 Behavior Support Coaches
– Itinerant positions/funded by LSS
• Meet 5 times/year
– Networking
– Regional Meetings
– Workgroups
• Coaches Reception
• Coaches Newsletter/Coaches Calendar
• Coach = Recruiter
8 LSS Coordinators
22
Roles of Coach
• Sustainability & Accountability
•
•
•
•
•
•
•
•
Hands-on technical assistance
Guide problem solving
Local training
Team start-up & sustainability
Public relations/communications
Support local leadership
Local coordination of resources
Provide prompts & reinforcers
23
Behavior Support Coach Activities
•
•
•
•
•
•
FTE allocated to complete tasks
Consistently attend team meetings
Assist team with data-based decisionmaking, planning, and implementation
Attend Regional/State Coaches
meetings/trainings
Send information to PBIS State/District
Coordinator (e.g., checklists, action plans,
etc.)
Assist with dissemination activities (e.g.,
presentations, case studies, articles, etc.)
24
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
25
Maryland
Annual Events
•
•
•
•
Spring Forum (April 6, 2006)
July Institute (July 10, July 17)
Coaches Meetings (5/year)
Regional Team Leader/Coach Meetings
(2/year)
• Schools serving students with special
needs - MANSEF (2/year)
• High Schools – (2/year)
26
Spring Forum
•
•
•
•
Purpose: Recruitment
Date: April (set one year in advance)
Overview & School Presentations
Participants:
– Administrators from “potential new” schools
– “Key” system personnel
– Potential coaches
Planning Phase I Begins
27
Summer Training
• New Coaches
• New Teams
• Returning Teams
• Exemplar Schools
• Receptions and Poster
Sessions
28
Other Training Events
• Local Coordinators/Trainers
• State Coaches Meetings
• Regional Meetings
• Targeted Group
• Behavioral Basics
• School Wide Information System
(SWIS)
• SWIS facilitator
• School-wide Evaluation Tool (SET)
29
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Active Coordination
Training
Coaching
Evaluation
Local School Teams/Demonstrations
30
Evaluation Capacity
Establishing Measurable Outcomes
1. What schools have been trained and are
active?
2. How well are schools implementing
PBIS?
3. What impact does PBIS have on student
behavior? Achievement?
31
Monitoring Outcomes
• Team Implementation Checklist
• SWIS
• SET
• Coaches Checklist
• Staff Survey
• Satisfaction Surveys
• Implementation Phases Inventory (IPI)
32
Evaluation Tools
Access 2003 Database
• Data entry/storage
• Report Generation
33
Evaluation Tools
Maryland website www.pbismaryland.org
Various levels: Any user
Team/coach
LSS Point of Contact
State Team
Maryland Forms Matrix
Access Database
SWIS
PBS surveys (www.pbssurveys.org)
34
35
How Well are Schools
Implementing?
• Systems-wide Evaluation Tool (SET)
– Annually
– 7 Features of SW Implementation
• Implementation Phases Inventory (IPI)
– Semi-annually
– Levels of SW: Preparation, Initiation,
Implementation, and Maintenance
36
School-Wide Evaluation Tool
35 Coaches trained as SET assessors
15 Contractual SET assessors
•
•
•
•
•
•
97 SETs completed 2004
154 SETs completed 2005
104 schools have at least two SET scores
80% Total score is considered
Maintenance Phase (IPI)
All regions met 80% criterion across
schools
69% increase after one year of
implementation
37
SET Scores by Region
100
90
80
70
60
50
40
30
93
91
86
90
84
43
42
29
25
Pre
Western
Special
Southern
Eastern
Central
20
10
0
Post
38
IPI
% of schools at phase
11.1%
16.0%
53.8%
19.1%
P reparatio n
Initiatio n
* IPI: Implementation Phases Inventory
Implementatio n
M aintenance
39
PROJECT TARGET
Evaluating PBIS in Maryland
Project Target
Dr. Catherine Bradshaw
Center for the Prevention of Youth Violence
Johns Hopkins University
40
Randomized Trial of PBIS:
Project Target
Project Target
Sample
• 37 Elementary Schools
– Allegany(7), Anne Arundel(4), Baltimore (13),
Charles(11), & Washington(2)
– 3,057 Staff & 20,246 Students
Design
• Randomly Assigned
•
– 21 PBIS
– 16 Comparison
Baseline plus 4 years
41
Funding : NIMH & CDC to the Johns Hopkins Center for the Prevention of Youth Violence
Data Collected
• Disruptive behavior
Project Target
– Teacher Observation of Classroom Adaptation (TOCA; Werthamer-Larsson et al., 1991)
– Student Interactions in Specific Settings (SISS; Cushing & Horner, 2002)
– Office discipline referrals (SWIS; School-Wide Information System)
• Academic information
– Attendance
– Academic performance
• School climate
– School Climate Survey (Haynes, Emmons, & Comer, 1994)
• Organizational health
– Organizational Health Inventory (Hoy et al., 1990)
• Implementation fidelity
– System-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)
– Staff Survey (Sugai, Todd, & Horner, 2000)
42
Implementation of PBIS
100
90
80
70
60
50
40
30
20
10
0
PBIS
Comparison
84
90
*
*
62
56
37
43
Baseline
Year 1
Year 2
Notes. No significant differences between groups at baseline, but differences at all other years, p<.05.
43
Impact of PBIS on Organizational Health
3.15
PBIS
Comparison
3.12
Mean OHI
3.1
3.05
3.01
3.012
3
2.95
2.94
2.9
Baseline
Year 3
Note. Adjusted means from 3-level model in Mplus. Intervention effect on slope of overall OHI significant at p<.05.
44
Summary of Preliminary
Findings from Project Target
• PBIS schools have high program fidelity
• PBIS increased organizational health
• Some positive effects on student outcomes
– Behavior problems in classroom
– Reduced office discipline referrals
– Reduced suspensions
Project Target
45
PBS Systems Implementation Logic
Visibility
Funding
Political
Support
Leadership Team
Action Plan
Training
Coaching
Evaluation
Local School Teams/Demonstrations
46
Anticipated Growth
50% of MD Schools will be trained by 2010
Chart Title
800
600
400
364
434
504
574
644
714
200
0
FY 05
FY 06
FY 07
Anticipated Growth at 5%
FY 08
FY 09
FY 10
Linear (Anticipated Growth at 5%)47
5 YEAR GOALS
• Goal 1: Increase state coordination by
identifying regional coordinators for Southern,
Central, Eastern and Western Maryland.
• Goal 2: Increase evaluation capacity by
expanding web site functionality i.e. online
tutorials, data submission and retrieval.
• Goal 3: Increase local training capacity by
identifying training coordinator(s).
• Goal 4: Increase visibility by identifying an
individual to write grants, develop marketing
material and interface with outside agencies.
48
Lessons Learned
•
•
•
•
•
•
•
Pacing
Be Patient
Exemplar
Feedback
Social Marketing-target audiences
Role of Coach
Process
49