Transcript The Importance of Coaching in Implementation of Evidence-based Practices Rob Horner University of Oregon www.pbis.org.

The Importance of Coaching in
Implementation of Evidence-based Practices
Rob Horner
University of Oregon
www.pbis.org
Goals

Current assumptions/research about
coaching

Define our experience with coaching in
PBS implementation

Implications for building district capacity
Coaching Defined

Coaching is the active and iterative delivery of:
◦ (a) prompts that increase successful behavior, and
◦ (b) corrections that decrease unsuccessful behavior.
◦ Coaching is done by someone with credibility and
experience with the target skill(s)
◦ Coaching is done on-site, in real time
◦ Coaching is done after initial training
◦ Coaching is done repeatedly (e.g. monthly)
◦ Coaching intensity is adjusted to need
Outcomes of Coaching
Fluency with trained skills
 Adaptation of trained concepts/skills to
local contexts and challenges

 And new challenges that arise
Rapid redirection from miss-applications
 Increased fidelity of overall
implementation
 Improved sustainability

 Most often due to ability to increase coaching
intensity at critical points in time.
Training Outcomes Related to Training Components
Training Outcomes
Training
Components
Presentation/
Lecture
Knowledge of
Content
Skill
Implementation
Classroom
Application
10%
5%
0%
Plus
Demonstration
30%
20%
0%
Plus
Practice
60%
60%
5%
Plus Coaching/
Admin Support
Data Feedback
95%
95%
95%
Joyce & Showers, 2002
Coaching
within SWPBS Implementation

Context:
◦ 9000 schools implementing SWPBS nationally

Defining the Role
 Internal vs External
Selecting Coaches
 Training and support for coaches
 Assessing Impact

Visibility
Political
Support
Funding
Leadership Team
Active Coordination
Training
Coaching
Behavioral
Expertise
Local Demonstration Schools
Evaluation
Coaching vs. Training

Coaching involves active collaboration
and participation, but not group
instruction.
◦ Small group
◦ Build from local competence
◦ Sustainable
Who should be a coach?

Internal vs External
 Internal coaches are employed in the school where
they provide support
 External coaches are employed outside the schools
where they provide support (e.g. by district, region,
state).
Who should be a coach?
Internal Coach
External Coach
Advantages
Knowledge of school
Staff relationships
Regular access
Independent
Outside perspective
Multiple schools experience
Disadvantages
Conflicting roles
Narrow range of
experiences
Limited knowledge of school
Limited relationships
Less frequent access
Who should be a coach
Coaching Competencies
Necessary
Preferred
Participate in team training
Knowledge about SWPBS
Able to attend team meetings at
least monthly
Knowledge about behavior support
practices (targeted, individual)
Effective working with adults
Skilled in collection and use of data
for decision-making.
Knowledgeable about school
operating systems
Professional Commitment
What Coaches Do




Work with team during initial SW-PBS training
Meet with new teams monthly on-site
Telephone/email contact as needed
“Positive” nag





Self-assessment (EBS Survey, Team Checklist)
Action planning
Activity implementation
On-going evaluation
 School self-evaluation efforts
 State-wide Initiative evaluation efforts (SET)
Guide State-wide initiative
 Feedback to Taskforce
What Coaches Do
Dissemination of outcomes and effects
 SWIS Facilitation

◦ Implement and support use of data-based
decision making.
Commitment of Coaches

Team Support
◦ First Year (1-2 teams) (participate in training and planning)
◦ Second Year (Maintain initial teams, start 3-5 teams)
◦ Future Years (10-15 teams total)

FTE commitment
◦ 20-50%

Roles/Background
◦ Behavior Specialists, Special Education Teachers
◦ Consultants, Administrators
◦ School Psychologists, Counselors, Social Workers
Guiding Principles for Effective Coaching

Build local capacity
 Become unnecessary…but remain available

Maximize current competence
 Never change things that are working
 Always make the smallest change that will have the biggest impact

Focus on valued outcomes
 Tie all efforts to the benefits for children

Emphasize Accountability
 Measure and report; measure and report; measure and report.

Build credibility through:
 (a) consistency, (b) competence with behavioral principles/practices, (c)
relationships, (d) time investment.

Precorrect for success
Specific Expectations
◦ Attend and participate in team training
◦ Meet with your team(s) at least monthly
 Provide technical assistance as needed
◦ Monitor and report on team efforts
 Team Checklist
 EBS Survey/ SET/ ISSET
 Annual Profile/Summary Data
◦ Present on School-wide PBS at district, state, national
forums.
◦ Assist district to build capacity for sustained
implementation (re-define your role over time)
◦ Meetings with Coordinator and Taskforce for
purposes of state-wide planning
Assist Teams in Using Data for
Decision-making
Using Team-Checklist and EBS Survey data
for Team Action Planning
 Using SET/ TIC data for evaluation
 Using ODR/ Academic (ORF) data for
assessment, planning and reporting.
 Keeping faculty involved through regular
data reporting.

Examples
Illinois
 North Carolina
 Michigan

PBIS in Illinois
Lucille Eber Ed.D.
IL PBIS Network
July 17, 2008
Developing Local Systems of Care
for Children and Adolescents with
Mental Health Needs and their Families
Training Institutes
Nashville,TN
PBIS Schools Over Ten Years:
number of schools
Trained & Partially or Fully Implementing
1000
900
800
700
600
500
400
300
200
100
0
874
520
394
587
654
444
303
120
184
23
Year Year Year Year Year Year Year Year Year Year
1
2
3
4
5
6
7
8
9
10
9/98 9/00 9/01 6/02 6/03 6/04 6/05 6/06 6/07 6/08
# IL PBIS Schools & # External Coaches
June 30, 2008
The Organization of PBIS in Illinois
900 schools implementing SWPBS
ISBE
Coordination
Chicago
Coordinators
North
Coordinators
Central
Coordinators
South
Coordinators
46 Coaches
(10)
495 Coaches
(84)
193 Coaches
(20)
105 Coaches
(29)
33 Schools
525 Schools
203 Schools
127 Schools
Elementary
Middle
Capacity
Schools per Coach per Region
Schools per Coach by Region
Schools/External
Schools/ Internal
Schools per Coach
8
7
6
5
4
3
2
1
0
Chicago
North
Central
South
Illinois Suspension Rates per 100
9
8
PBS slope = -1.15
7
6
5
PBS
Non PBS slope = -.37
4
Non-PBS
Linear (PBS)
3
2
1
0
2003-04
2004-05
2005-06
Illinois Suspension Rates per 100 for
Black and Hispanic Students
14
PBS Slope = -1.85
12
10
8
Non PBS Slope = -,34
PBS
Non-PBS
Linear (PBS)
6
4
2
0
2003-04
2004-05
2005-06
North Carolina
Positive Behavior Support Initiative
Partners’ Update
February 2009
Heather R. Reynolds
NC Department of Public Instruction
Bob Algozzine
Behavior and Reading Improvement Center
http://www.dpi.state.nc.us/positivebehavior/
North Carolina
Positive Behavior Support Initiative
State PBS Coordinator
Heather R Reynolds
North Carolina
Positive Behavior Support Initiative
Office discipline
referral data
(majors) from
schools
implementing
PBS in North
Carolina [07-08]
compare
favorably with
national
averages.
North Carolina
Positive Behavior Support Initiative
Office Discipline Referral Risk in North Carolina
100
90
80
70
60
50
40
30
20
Levels of
behavior risk in
schools
implementing
PBS were
comparable to
widely-accepted
expectations and
better than those
in comparison
schools not
systematically
implementing
PBS.
10
0
2004-05 (N=21)
2005-06 (N=35)
2006-07 (N=66)
2007-08 (N=110)
Comparison (N=5)
6+ ODR
5
3
4
4
10
2-5 ODR
12
9
11
11
23
0-1 ODR
83
88
85
85
67
North Carolina
Positive Behavior Support Initiative
100
[A]chievement
causes
[B]ehavior?
95
90
[B]ehavior causes
[A]chievement?
85
EOG Reading
80
75
Reading
Linear (Reading)
70
rxy = -.44
(n = 36)
65
60
55
50
0.00
0.10
0.20
0.30
0.40
0.50
ODRs
0.60
0.70
0.80
0.90
1.00
[Context causes
[A]chievement
and [B]ehavior?.
Steve Goodman
[email protected]
www.cenmi.org/miblsi
Goals
1.
Share information about Michigan’s
Integrated Behavior and Learning
Support Initiative (MiBLSi)
2.
Provide examples of improving the
quality and quantity of the data
collected
3.
Provide examples of acting upon
project data to improve outcomes
Participating Schools
300
250
2000 Model Demonstration Schools (5)
2004 Schools (21)
200
2005 Schools (31)
150
2006 Schools (50)
2007 Schools (165)
100
50
0
2003-2004
2004-2005
Existing SchoolsNew Schools
2005-2006
2006-2007
Project Data:
Outcomes, Process and System
Development
Major Discipline Referrals per 100 Students per Year
(Schools implementing > 80% on Team Implementation Checklist)
DIBELS Instructional Recommendations and Major
Discipline Referral per Cohort per Year
Major Discipline
Referrals
160
90
140
80
70
120
DIBELS
Benchmark
100
60
50
80
40
60
30
40
20
20
10
0
0
2003-2004
Cohort 1 (n=16 schools)
2004-2005
2005-2006
Cohort 2 (n=24 schools)
2006-2007
Major Discipline Referrals per 100 students
Percent Students at Benchmark (schoolwide)
100
Percentofstudents
Participating School Example:
Fourth Grade Reading MEAP Results
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
2000
Began MiBLSi
Implementation
2001
School District
2002
2003
Year
2004
2005
Improving the quality and
quantity of project data
Percent of Process and System Data Collected by
Cohort
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Cohort 1
Cohort 2
Cohort 3
Cohort 4.1 Cohort 4.2 Cohort 4.3
Project
Average
Improving the accuracy and
Consistency of Recording Office
Discipline Referrals
Example Situation
Developing Fluency with
Discipline Referral
Categories
Example Exercise 2:
Match the example
situation below to the
correct problem
behavior on the
discipline categories
answer sheet. Write
the letter in the
column for Exercise
2.
A.
Issac picked up his lunch and notice that another student was sitting in his
usual spot at the table. Issac squeezes into his spot forcing the student to
abruptly slide off the seat and onto the floor.
B.
Ian comes to school wearing a T-shirt with a sexually suggestive picture.
C.
During the test, Timothy is looking at Jennifer’s paper and copying her
answers.
D.
Taneshia walks to school each morning. She plays in the yard across from
the school until the bell rings. After she sees all the student run into the
building, she walks towards the school, late again.
E.
During lunch, Franz collects the empty milk cartons from his classmate. He
sets them on the floor in a line and then stamps on the cartons making a loud
noise with each stomp.
F.
Alexandra has been following Ben around the playground. She sometimes
makesExercise
various1noises (that Exercise
seems to2bother him) Exercise
when she
3 is near. Sometime
she tells him how she and her brother will stop him on the way home from
school and beat him up.
Problem Behavior
Inappropriate language
G.
Fighting physical injury
H.
Defiance/Disrespect
Lying/Cheating
Susan and Jill are best friends. Jill lives several houses down from the
school. During lunch Jill talks Susan into coming over to see her new puppy.
The school has a rule about staying on school property at lunch.
Billy’s teacher asks the class to take out their math book. Billy groans and
says that math is stupid while not taking out his book. His teacher then tells
him he needs to get his book from his desk. Billy says he hates math, he
hates school and he is not going to do it.
I.
Mr. Walter happens to walk into the boy restroom to find Matt and Chris
Harassment/Tease/Taunt
wadding up paper towels and stuffing them into the toilet.
Disruption
J.
Excessive Tardiness
Truancy
K.
Property Damage
Forgery/Theft
Other
Rebecca has been having difficulty with her assignments at school. Her
teacher, Ms. Swanson, sent a note home with Rebecca to be signed by her
parents and returned the next day. However, when the note was returned the
signature was Rebecca’s mother’s name but written in Rebecca’s
handwriting.
The first grade class writes in a journal. The topic focuses on a particular
letter each day. During the letter “F” day, Sarah writes a word she has
learned from some tough kids on the playground. She shows her journal to
all of the students around her.
Acting on the Data to Improve
Classroom Management
Major Discipline Referrals by Location
100%
Began focusing on
classroom management
support 2005-2006
90%
80%
70%
60%
50%
40%
30%
20%
10%
200304 ,N=28
200405 ,N=36
200506 ,N=77
200607 ,N=108
200708 ,N=205
Special Event
Playground
Other
Office
Library
Hall
Gym
Common
Classroom
Cafe
Bus Zone
Bus
Bathroom
0%
MEAP- 4th grade Reading Assessment
29 Elementary Schools In Michigan
Schoolwide: Over 55%
of major discipline
referrals from
classroom
Schoolwide: Under
55% of major discipline
referrals from
classroom
Probability of scoring
below 75% proficiency
on 4th grade MEAP
(Reading): .78
Probability of scoring
above 75% proficiency
on 4th grade MEAP
(Reading): .75
Improving Targeted Student
Intervention
Interviews with staff and self assessment
indicate a need to develop targeted
support systems
Checklist for Individual Student Support Systems
(CISS) Results from Cohort 4 (n=34 schools)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Foundatons
Targeted
Intensive
Overall
Improving Targeted Student
Intervention Strategies

Building Leadership Teams:
◦ “Quick Sort” process for identifying
students and linking to interventions

Focused training for practitioners:
◦ Using Behavior Education Program
(check in- checkout)
Supporting Coaches

Conducting Self-Assessment to identify
needs

Providing support based on results
◦ Coach training 2 – 4 time per year
◦ Coach manual
◦ Coach website
◦ Coach conference (March 13-14)
Coaches Self-Assessment
adapted from: Sugai, Todd and Horner, 2006
100%
90%
Percent Responded
80%
70%
60%
50%
40%
30%
20%
10%
0%
Item
Example of the Impact of Coaching on Student Outcomes:
Average Major Discipline Referrals per Day per Month
Coach returns
from leave
7
6
Avg.ReferralsperDay
5
4
3
2
1
0
SepOctNov
DecJanFebMarAprMay
05-0606-07
Working with School Teams to use data
Data Review/Action planning with building
leadership teams
 Pre-training coaches/principals in the data
review content
 Providing worksheets to guide data review
process

Team Evaluation of Outcome, Process and Systems
Data
EBS Team Implementation Checklist by Feature
Lesson Learned

Implementation cannot be faster than
your school staff capacity to implement

Teams need to be taught how to
analyze and use data

Emphasis on directing resources to
need and removing competing activities
Building Capacity Statewide

Districts need capacity to:
◦ Deliver regular training on core content
 Annual orientation
 Staff development
◦ Incorporate expectations in regular staff
evaluations
◦ Provide expertise for more intense support
need requirements.
 Regular meetings with building personnel around “emerging
challenges.”
Avoid passing the planning buck
State asks districts to build a plan
 Districts ask schools to build a plan
 Schools ask teachers to build a plan

“Even if you’re on the right track,
you’ll get run over if you just sit
there”
- Will Rogers