Impacting Outcomes for Students with the Most Challenging Behaviors through Schoolwide PBS: OSEP’s Model Demonstration Projects

Lucille Eber, Statewide Director Illinois Statewide Technical Assistance Center PBIS Network Jennifer Doolittle, Project Officer Office of Special Education Programs

      

Timeline of OSEP’s PBS Investments

1997:

IDEA reauthorized

1998:

OSEP funds the Center on Positive Behavioral Interventions and Supports (PBIS).  Provides a framework for schoolwide behavior support practices

2002:

OSEP funds the Center for Evidence-based Practice: Young Children with Challenging Behavior

2004:

“PBS Blueprint for Practice” emphasizes developing State and district level infrastructures that can promote large-scale implementation and sustainability

2006:

Tertiary Behavior Model Demonstration Sites

2008

: Technical Assistance Center for Social-Emotional Intervention for Young Children (TACSEI)   Scaling up evidence-based behavior support practices at the preschool level IDEA performance measures for social-emotional outcome for birth to five

The Future

  Continue to scale-up school-wide PBS Assist schools to identify and implement more tertiary-level, intensive behavior support strategies.

OSEP’s Model Demonstration Projects (MDPs)

 Model Demonstration Coordinating Center    Creating cross-site evaluation plans, instruments, and procedures to assess the context, implementation, and efficacy of each model and across models within a cohort.

Identifying key issues in translating research to practice and analyzing and synthesizing data across MDPs and cohorts Applying multiple analytic methods to answer evaluation questions regarding both the efficacy of models and bridging the gap between research and practice.

 Producing high-quality, useful, and accessible products that communicate findings to key audiences.

    

Cohorts

Each year a new cohort of three to four MDPs is added; each cohort has a different focus: 2006: 1st cohort is focusing on progress monitoring for preschool through 4th grade in general and special education classrooms.

2007: 2nd cohort is developing tertiary behavior interventions for elementary/middle schools for students with challenging behaviors.

2008: 3rd cohort will develop, implement and evaluate early childhood language interventions.

Documenting model development, model implementation, and model outcomes for each cohort, and analyzing MDP experiences and results across cohorts  Help OSEP bridge the gap between identifying evidence-based practices and achieving their widespread use.

Tertiary Behavior MDPs



Illinois PBIS Network and the University of Kansas



The University of Oregon



The University of Washington

      

Tertiary Behavior MDPs Big Ideas

School-wide intensive PBS approach guided by a three-tiered prevention model Individualized, function-based behavior support to children who exhibit the most challenging behaviors and have not been responsive to primary or secondary prevention efforts Emphasis on collecting and using data for decision-making Systematic strategies for professional development TA that improves student behavior to help schools promote student learning and other positive outcomes Cost effective and efficient process for school districts to implement and sustain A response to intervention (RtI) logic model to show student progress

RTI for Behavior and Academics

 Key components  Evidence-based instruction and supports for all students      Universal screening to determine which students are not meeting benchmarks Targeted instruction and support that goes beyond what all students receive Progress monitoring Intensive and individualized instruction and support for students who are still not making progress - special education?

Decision points throughout

CEC



Boston



April 4, 2008

Lucille Eber, Illinois PBIS Network Wayne Sailor, University of Kansas

The Kansas-Illinois SW-PBS Tertiary Demonstration Center: A Response to Intervention (RtI) Continuum of Support Model

K-I Center Team Leaders

• Jamie Bezdek, University of Kansas • Kimberli Breen, IL PBIS Network • Jen Rose, Loyola University-IL PBIS Network • Amy McCart, University of Kansas

Evaluation:

• Kelly Hyde (SIMEO) • Holly Lewandowski (PoI and SWIS data)

Big Ideas for this Session

1. How the K-I Center is applying the RtI approach to both behavior and academics to ensure tertiary capacity 2. Implementation experiences and data from IL (Year One and Year Two) 3. What the K-I Center hope to “deliver” in terms of knowledge, tools etc.

Key Questions

Does building a school-wide system of PBIS increase school’s abilities to effectively educate students with more complex needs?

What systems, data and practice structures are needed to ensure that positive behavior support being applied in needed dosage for ALL students?

School-Wide Systems for Student Success A Response to Intervention Model

Academic Systems

Tertiary Interventions •Individual Students •Assessment-based •High Intensity Secondary Interventions •Some students (at-risk) •High efficiency •Rapid response •Small Group Interventions • Some Individualizing 1-5% 5-10% 1-5% 5-10%

Behavioral Systems

Tertiary Interventions •Individual Students •Assessment-based •Intense, durable procedures Secondary Interventions •Some students (at-risk) •High efficiency •Rapid response • Small Group Interventions • Some Individualizing Universal Interventions •All students •Preventive, proactive 80-90% 80-90% Universal Interventions •All settings, all students •Preventive, proactive

Positive Behavior Interventions & Supports A Response to Intervention (RtI) Model

Universal

School-Wide Assessment SWIS & other School-wide data BEP & group Intervention data School-Wide Prevention Systems

Secondary

Small

group interventions (CICO, SSI, etc)

Group interventions with individualized focus (CnC, etc)

Tertiary

Simple individual interventions (Simple FBA/BIP, schedule/curriculum changes, etc) Functional assessment tools/ Observations/scatter plots etc.

SIMEO tools: HSC-T, RD-T

Multiple-domain FBA/BIP Wraparound Revised March 2008 IL-PBIS Adapted from T. Scott, 2004 3.5.08

Core Features of a Response to Intervention (RtI) Approach

• Investment in prevention • Universal Screening • Early intervention for students not at “benchmark” • Multi-tiered, prevention-based intervention approach • Progress monitoring • Use of problem-solving process at all 3-tiers • Active use of data for decision-making at all 3-tiers • Research-based practices expected at all 3-tiers • Individualized interventions commensurate with assessed level of need

Continuum of Support for Secondary-Tertiary Level Systems

1.

2.

3.

4.

5.

Group interventions

(BEP, social or academic skills groups, tutor/homework clubs, etc)

Group Intervention with a unique feature

student, (BEP individualized into a Check & Connect; mentoring/tutoring, etc.) for an individual

Simple Individualized Function Based Behavior Support Plan

for a student focused on one specific behavior (simple FBA/BIP-one behavior; curriculum adjustment; schedule or other environmental adjustments, etc)

Complex Function-based Behavior Support Plan across settings

(i.e.: FBA/BIP home and school and/or community) Wraparound: behaviors More complex and comprehensive plan that address multiple life domain issues across home, school and community (i.e. basic needs, MH treatment, as well as behavior/academic interventions) multiple 3.8.08

ILLINOS SW-PBS History • The Context for Implementing the Tertiary Demo process……

Implementing Tertiary Demos: The IL Context for

IL PBIS Schools Over Nine Years:

Trained & Partially or Fully Implementing

700

654

600

587

500

520

400

444 394

300

303

200 100 0

184 120 23

Year 1 9/98 Year 2 9/00 Year 3 9/01 Year 4 6/02 Year 5 6/03 Year 6 6/04 Year 7 6/05 Year 8 6/06 Year 9 6/07

IL PBIS Expansion History

June 30, 2005

 

444 schools in 143 districts 92 new schools trained June 30, 2006

  

520 schools in 155 Districts 97 new schools trained in FY06 12 new districts June 30, 2007

  

654 schools in 170 72 schools trained in FY07 15 new districts January, 2008

  

744 Schools in 196 Districts Approximately 90 schools trained in 1 st Approximately 26 new districts in 1 st half of FY08 half of FY08

Illinois PBIS Schools

Mean Percentage of Students with Major ODRs 2006-07, Statewide

100%

12% 5%

90% 80% 70%

14% 12% 83% The differences between fully and partially implementing schools were statistically significant in all three levels of ODRs 74%

60% 50% Partially Implementing (n=58) 0-1 ODRs Fully Implementing (n=141) 2-5 ODRs 6+ ODRs (0-1 ODR, Mann Whitney U=3035.0, p=0.004; 2-5 ODR, Mann-Whitney U=3050.0, p=0.005; 6+ODR, Mann Whitney U=3062.0, p=0.005).

Illinois PBIS Schools

Illinois PBIS Schools Completing School Profile Forms & Implementing Secondary/Tertiary Interventions across Five Years

250 200 150 100 50 0

149 170 187 151 195

2002-2003 2003-2004 2004-2005

Years

2005-2006 2006-2007 Schools Completing Profiles # of Schools Reporting Small Group and Indiv idual Interv entions

Illinois PBIS Schools

Comparison of Partial & Fully Implementing Schools on Suspensions/Expulsions FY07 per 100 Students

12 10 8 6 4 2 0

11.2

5.7

Partial (n=58)

Implementation

Full (n=140)

Illinois PBIS Schools 100 80 60 40 20 0

Comparing School Safety Survey Partial vs. Full Implementation 46 73 41 78

Partial (n=20) Full 80/80 (n=62)

Partial vs Full Implementation

Risk factor Protection factor

Illinois PBIS Schools

Illinois 2005-06 Proportion of Students who Meet or Exceed Third Grade ISAT Reading Standard

70 68 66 64 62 60 58 56 54

59.3

Partial (n = 89 schools)

68.4

Full (n = 120 schools) The difference between the two types of schools was

significant

(t=3.72, df=159, p<0.001).

Level of PBIS Implementation

Illinois 2005-06 Proportion of Schools that Met AYP

100 90 80 30 20 10 0 70 60 50 40

65.6

86.6

Partial (n = 160 schools) Full (n = 157 schools)

Level of PBIS implementation

Illinois PBIS Schools

Findings suggest that fully implementing PBIS schools met AYP at a significantly higher percentage than partially implementing

schools (χ2=19.17, df=1, p<.001).

Illinois PBIS Schools

Small Group & Individual Interventions Rated "High" or "Very High" in Fully & Partially Implementing PBIS Schools 2006-07 142

150 100 50 0

42

Fully Implementing Schools (n= 70 schools) Partially Implementing Schools (n= 24 schools)

Level of implementation

Illinois PBIS Schools

Small Group Interventions Rated as "Very High” & "High“ in Fully & Partially Implementing PBIS Schools 2006-07 73

80 70 60 50 40 30 20 10 0

21

Fully implementing (n = 47 schools) Partially implementing (n = 14 schools)

Level of implementation

Illinois PBIS Schools 80 70 60 50 40 30 20 10 0

Individual Interventions Rated “Very High” & “High” in Fully & Partially Implementing PBIS Schools 2006-07 69 21

Fully implementing (n = 44 schools) Partially implementing (n = 14 schools)

Level of implementation

Technical Features: Database Development SIMEO Database

(Systematic Information Management of Education Outcomes)

online data collection and graphing database system for individual student receiving intensive level planning and supports

Challenges….

Tertiary Demos

Mean Percentage of Students for Tertiary Demo School A by Major ODRs 2005-06 Grades K-6 (285 students)

100% 80% 60% 40% 20% 0%

10.53% 20.00% 69.47% 53.57% 36.61% 9.82%

% Students In Group 0 or 1 2 to 5 % ODRs From Group 6+

Tertiary Demos 100% 80% 60% 40% 20% 0%

Mean Percentage of Students for Tertiary Demo School A by Major ODRs 2006-07 Grades K-6 (293 students) 5.46% 17.75% 76.79% 38.26% 46.38% 15.36%

% Students In Group 0 or 1 2 to 5 % ODRs From Group 6+

Examples of Ineffective or Weak Secondary/Tertiary Systems

• Referrals to Sp.Ed. seen as the “intervention” beyond universals • FBA seen as required “paperwork” vs. a critical step in designing effective interventions

Some “Big Picture” Challenges?

• Low intensity, low fidelity interventions for behavior/emotional needs • Habitual use of restrictive settings (and poor outcomes) for youth with disabilities • High rate of undiagnosed MH problems (stigma, lack of knowledge, etc) • Changing the routines of ineffective practices (systems) that are “familiar” to systems

Tertiary Demos

Investing in Tertiary Demos?

• ISBE is investing • ICMHP is investing • USDOE-OSEP is investing

We need replicable Systems, Data, and Practices so we can effect long term change.

800 600 400 200 0

Secondary Training Events A Two Year Comparison

60 40 20 0 2005-06 2006-07 Secondary Participants Secondary Trainings

1000 800 600 400 200 0

Tertiary Training Events A Two Year Comparison

2005-06 2006-07 Tertiary Participants Tertiary Trainings 30 20 10 0

A Focus on Tertiary Impacts Implementation at All Levels

• Notable progress was observed in tertiary demo schools’ implementation of PBIS. • Building-based teams met frequently to action plan and significant gains were made during year one.

• The Illinois PBIS Phases of Implementation Tool is being used by schools to self-assess their systems, data and practices and guide their implementation.

• As schools invest in developing tertiary structures, they are also taking steps to improve their universal and secondary systems.

IL Tertiary Demo 40 35 30 25 20 15 10 5 0

Tertiary Demo School Reduces ODRs & Increases Simple Secondary Interventions 36 5 5 1

Aug to Nov 2006 2-5 ODRs 6+ ODRs Aug to Nov 2007 CICO*

*CICO = Check in, Check Out

Wraparound

Results of Implementation of Wraparound within SW-PBS in IL

• Three year pilot • Enhance SOC wraparound approach – data-based decision-making as part of wraparound intervention • Development of strength-needs data tools • Web-based system

IL PBIS Tertiary Demos-07 150 125 100 75 50 25 0

Tertiary Interventions Linked to Immediate & Sustainable ODR Decreases 125 50

Time1 (N=26) Time 2 (N=26) Office Discipline Referrals

2

Time 3 (N=10)

Wraparound-07

Immediate & Sustainable Change Noted in Placement Risk High Risk

2 1.75

1.5

1.25

Low/No Risk

1

1.78

1.5

1.3

Baseline (n = 19) Time2 Time 3

Wraparound-07 1 0 4 3 2

School Risk Behaviors Substantially Decline for Student Engaged in Wrap 3.87

2.84

2.37

1.38

0.79

Baseline (n=19) Time 2 (n=19)

0.5

Time 3 (n=8) ODRs OSSs

IL Wraparound Data-07

Positive Classroom Behavior & Academic Achievement Linked Always

4

2.8

3

2.21

2.45

2.83

2

2.13

2.19

Never

1 Baseline (n=26) Time 2 (n=26) Classroom Behav ior Functioning Time 3 (n=12) Academic Achiev ement

IL PBIS Tertiary Demos

Shift in Responsibility for Individual Student Data Management at Tertiary Demo Sites

30 25 20 15 10 5 0 FY 05 (N=18) PBIS Network Staff FY 06 (N=26) School Social W orker PBIS Coach FY 07 (N=26) Other School Personnel

“Andy”

Using Data to Keep the Team Moving “Celebrate Success of current plan”

“Andy”

Next Steps

• Team will use data to plan for academic supports.

• Lower intensity of some supports • Transition from frequent adult feedback to self-monitoring to boost self-confidence and feel/be less dependent on adults.

“Mary Ellen”

Home, School, Community Tool

SIMEO -Educational Information Tool

3.5

3 2.5

2 1.5

1 0.5

0 Baseline Time 2 Time 3 attends school participates in extracurricular activities appropriate behavior in unsupervised settings Time 4

“AJ”

• 0 ODRs (from 3) • 0 Time-outs (from 22) • Passing grades in all classes (from D’s & F’s) • Parents report that: “this is different” • Improved partnerships with community service providers • MH partner participating and providing effective strategies

“Henry”

Reason Referred to Tertiary Supports “Henry”, an elementary school student, had:

  extremely poor attendance failing grades    poor homework completion trouble with the law in the community and had a court assigned probation officer and a mandated Department of Children and Family Services (DCFS) counselor

“Henry”

Engagement and Team Preparation

• Wraparound team initially included – Henry – his mother – the school social worker – his primary classroom teacher – school principal, bi-lingual liaison – district SWPBS tertiary-tier coach • Team met consistently to identify strengths, big needs and develop a wraparound plan for Henry.

“Henry”

Identifying Strengths and Big Needs

• Henry’s strengths identified by team included: – a good relationship with his teacher – responsiveness to positive attention from adults he likes – leadership among his peers – and effective self-advocacy • Henry’s “big needs” as identified by the wraparound team: – Henry needs to feel as if he fits in with the other kids at school – Henry needs to feel successful at school – Henry to be invested in his education

“Henry”

Ongoing Plan Implementation and Refinement

• Henry was included in the “Check-and-Connect” intervention which was being delivered to other students in the school • In addition, Henry and his teacher talked about individual behavior goals listed on his daily point card. • He was put on “safety patrol” in which he was an older youth selected to be a positive role model to help monitor and improve his behavior in the hallways. • Because Henry’s voice was important, the suggestion that he work with younger students in their classrooms was set aside due to Henry’s lack of interest.

“Henry”

Progress Monitoring

• Henry’s progress was monitored through: – Office discipline reports – Attendance – Grades – DIBEL scores – Check In Check Out behavior card points – SIMEO Tools

“Henry”

Initial Outcomes

• From second quarter to third quarter, with wraparound in progress, Henry’s grades and attendance began to increase – Spelling: 15% to 40% – Math: 15% to 48.5% – Reading: 20% to 63% – His DIBELS score increased from 55 words per minute in the fall to 67 words per minute in the winter. – Attendance:15% in 1 st quarter 60% in 2 nd quarter, 75% in 3 rd quarter

Henry’s Risk of Placement Data Referral-Disposition Tool (SIMEO)

Henry’s Improved Behavior and Emotional Functioning at Home

Henry’s Improved Behavior and Emotional Functioning at School

Henry’s Improved Behavior and Emotional Functioning In the Community

Educational Environment Data (EE)

• A key item in IL State Performance Plan for feds • More districts to be “flagged” for monitoring • Tertiary demo activities focus on IL SPP data points

Sparta School District

Six Year Comparison of Least Restrictive Environment

150 100 50 0 1999-00 2000-01 2001-02 2002-03 2003-04 2004-05 Monitor Resource Self contained

Dewey Elementary School

Changes in Least Restrictive Environment

25 20 15 10 5 0 50 45 40 35 30

27 60% 16

2003-04 students in SPED < 21% of day

45 78% 5

2004-05 students in SPED 21-60% of day 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% ISAT scores

EE Data

(continued) • First step is accessing the data • Next is discussing with range of stakeholders and determining accuracy or how to make it accurate • Possible tools/procedures to make a difference

Getting Started with Data-Based Decision-Making with EE Data

Similar to how we got started with ODR data • clean up data (e.g. ODR form) • review data trends and ask questions;

Getting Started

(continued) • decide what it means by those who “live” the data; • decide what data points to focus on • design actions that seem likely to effect change; • monitor/revise action plan

Why We Need MH Partnerships

• One in 5 youth have a MH “condition” • About 70% of those get no treatment • School is “defacto” MH provider • JJ system is next level of system default • 1-2% identified by schools as EBD • Those identified have poor outcomes • Suicide is 4 th young adults leading cause of death among

Why Do We Need to Go Beyond Use of ODRs?

• Use of “alternative” discipline responses; often w/o documentation • Over use of “Special Education” placement w/o adequate dosage of interventions

Why Do We Need to Go Beyond Use of ODRs?

(

continued

) • High rate of unidentified MH problems • Youth get identified only after “crisis” which makes it harder and more “costly” to intervene.

The Systematic Screening for Behavior Disorders (SSBD)

(Walker and Severson, 1992) • Developed as a school-wide ( Universal ) screening tool for children in grades 1-6 – Similar to annual vision/hearing screenings

Background

• Identifies behaviors that may impede academic and social functioning • Leads to earlier intervention • May reduce need for formalized, lengthy “requests for assistance” by using data to identify youth

Implementation

• Between early September-first of November, completed screenings districts and 18 schools in 6 • Initial results indicate that approximately 5%-10% of students enrolled in grades 1 6 were identified by the SSBD

Implementation

• A Middle school case example: – Approximately sixth grade were screened using the SSBD 320 students enrolled in – 38 six graders or 11% passed gate two

Implementation

• Currently, school-based secondary teams are using SSBD data to implement out) low-intensity interventions (e.g., check-in/check-

Ensuring Capacity at All 3 Tiers

• Begin assessment and development of secondary and tertiary tiers at start-up of universal – Assess resources and current practices (specialized services) – Review current outcomes of students with higher level needs – Position personnel to guide changes in practice – Begin planning and training with select personnel • All 3 tiers addressed at all district meetings and at every training

Requirements for IL Tertiary Demos

• District Commitment • Designated Buildings/District Staff • External Tertiary Coach/Coordinator • Continuum of Skill Sets (training, guided learning, practice, coaching, consultation) • Commitment to use of Data System – Going beyond ODR’s (i.e. SSBD) – Self assessment/fidelity – SIMEO-Student Outcomes

District-wide Secondary/Tertiary Implementation Process

• District meeting quarterly – District outcomes – Capacity/sustainability – Other schools/staff • Building meeting monthly – Check on all levels – Cross-planning with all levels – Effectiveness of practices (CICO/BIP/Wrap, etc) • Secondary/Tertiary Coaching Capacity • Wraparound Facilitators

System Data to Consider

• LRE – Building and District Level – By disability group • Other “places” kids are “parked” – Alternative settings – Rooms w/in the building kids are sent • Sub-aggregate groups – Sp. Ed.

– Ethnicity

Ongoing Self–Assessment of Secondary/Tertiary Implementation

Building Level:

• IL Phases of Implementation (PoI) Tool • IL Secondary/Tertiary Intervention Tracking Tool • Sp. Ed Referral Data • Suspensions/Expulsions/Placements (ongoing) • Aggregate Individual Student Data (IL SIMEO data) • LRE Data trends • Subgroup data (academic, discipline, Sp. Ed. Referral, LRE, etc)

District Level:

• Referral to Sp.Ed. Data • LRE Data (aggregate and by building) • IL Out-of-Home-School-Tracking Tool (multiple sorts) • Aggregate SIMEO data • Aggregate PoI Data