Transcript USD #305 PBIS Evaluation
USD #305 PBIS Evaluation
M A R Y B E T H G E O R G E , U S D 3 0 5 P B I S D I S T R I C T C O O R D I N A T O R
District-wide Planning
Create vision for district Foundation and leadership Use of resources Internal expertise Ensure sustainability
Training
SWPBS Organizational Logic
٭ Funding Visibility Political Support
Leadership Team
Active Coordination Coaching Evaluation Local School Teams/Demonstrations
SALINA SWPBS DISTRICT ORGANIZATION District Coordinator Secondary School Teams External Coaches External Coaches External Coaches School Coaches School Team School Coaches School Team School Coaches School Team
Current District PBIS Team District coordinator District and building administrators Professionals representing curriculum and instruction Safe and drug free representation Professional development Bus system leadership Mental health professionals Family members Students Special education cooperative Interagency partners (court services, mental health, developmental disability services, children and family services, etc.)
What Progress has the District Made in Implementing PBIS?
Training for all schools to use the School-wide Information System (SWIS) 3 schools at tier 2 9 schools at tier 1 District Coordinator meetings occur regularly Linda Benjamin has been shadowing Project Stay professionals to learn more about how the SIT training systems at the district level may benefit Individuals within the district are participating in the Kansas Institute for PBS Tier 3 individualize planning across home, school and community Tier 3 Interagency event in the works for spring, 2010
District-level Progress (Continued)
Three PBIS District Newsletters Board presentations on PBIS Visits from outside school districts Interagency meetings started to begin building Tier 3 capacity District 3 year action plan for PBIS is in place
District 3 Year PBIS Plan
Document in MTSS Notebook
Fidelity of Implementation: District Team Checklist
District Team Checklist
Document in MTSS District Notebook
Are Schools Implementing PBIS?
Self-Assessment Data EBS Survey Results Oregon School Safety Survey, Action Plans, Office Discipline Referral Data Effective Behavior Support Survey Team Implementation Checklist Data Annual action plans for each school School-wide Information System (SWIS) Data Average referrals per day per month By time By location By student By problem behavior School-wide Evaluation Tool (SET) Outcome Data
Fidelity of Implementation: Team Implementation Checklist
Team Implementation Checklist
Document in MTSS District Team Notebook
EBS Survey Data Results for Schools
Self Assessment Summary
Team Implementation Checklist results show 8 out of the 12 school teams are at 80% overall implementation, There is a 68% increase in the number of surveys completed by staff, students, parents and community members between 2008-2009 and 2009-2010 school years.
There was a 16.6% decrease in the perceived risk factors across the USD 305 schools between 2008-2009 and 2009-2010 school years.
There was a 3.3% increase in perceived protective factors across the USD 305 Schools between the 2008-2009 and 2009-2010 school years.
Training and Support
All 12 buildings are receiving implementation support Coaches meetings occur monthly to support schools External coaches are now completing the SET in schools they are not supporting directly All 12 buildings are implementing school-wide PBS in varying degrees Targeted and intensive individual positive behavior support is being implemented at Heusner and Stewart
Training and Technical Assistance
Tier 1 Training All buildings district wide 2008-09 Advanced Tier 1 Training All buildings district wide 2008-09 (Except CHS) Tier 2 Training Central High School, Heusner Elementary School, Stewart Elementary School 2008-09 Tier 3 On site Support Central High School Fall 2009
Why Create Inter-rater Agreement for the SET?
Avoid “we’re doing MTSS for behavior” Provide consistent responses to school teams Build capacity of district personnel to guide PBIS/MTSS process Link district data to state PBS Kansas Districts participating are working on plan to unify data Consistency between all state district coordinators and SET implementers Districts understand they are implementing PBIS---there is evidence via inter-rater agreement
Inter-rater Agreement on the School-wide Evaluation Tool (SET)
1 District Coordinator + 3 External Coaches Reliable with 1 Benchmark Person;
Proportion of Agreement: Formula = [Number of Agreements/ (Agreements + Disagreements)] x 100% Inter-Rater Agreement on SET for Reliable Raters
# of Reliable Raters in District 4 SETs Completed Inter-rater Process 3 SETs in USD305 Average Agreement Across All SETs 97.63% Reliability Range Across Features 66.7% - 100% Across Features
Inter-rater agreement is achieved if the person scores 3 SETS with 80% or higher agreement with the benchmark inter-rater reliability agreement person.
Internal vs. External Evaluation of the School-wide Evaluation Tool (SET): Example of First Steps Inter-rater Agreement System at the Salina School District
District PBS Coordinator and Lead Inter-rater (Shonda) completes the SET and compares scores SET inter-rater agreement with External Coaches (complete) Starting together with Shonda for SET Continuing inter-rater agreement with Mary Beth (25% of all SETs completed) External coaches obtain agreement with district coordinator Data on inter-rater agreement is reported at the district level
External Evaluation: Show SET data are consistent with People Outside the District
Invite an expert to review PBIS data and training systems and provide guidance Establish inter-rater agreement with others outside the district Other PBS District Coordinators Shonda
SW-PBS Impact on Student Behavior
School #1 School #2 School #3
SW-PBS Impact on Student Behavior
School #5 School #6 School #7
Examples of Decreases in ODRs Reported by School Teams
School #7 implemented a teacher mentor program last spring and data shows that last years ODR’s were 593 to date and this year’s are 438 which is a 26% decrease in ODR’s.
School #6 implemented a program to reinforce their expectations taught and their ODR’s dropped from 96 to 58 to date which is a 39% decrease in ODR’s.
Faculty, Staff, Students, Families and Community Stakeholders
There is a 68% increase in the number of surveys completed by Staff, students, parents and community members between 2008-2009 and 2009-2010 school years.
There was a 16.6% decrease in the perceived risk factors across the USD305 schools between 2008-2009 and 2009 2010 school years.
There was a 3.3% increase in perceived protective factors across the USD305 Schools between the 2008-2009 and 2009-2010 school years.
Policy Impact
Integrating district team for academics and behavior Website created to share information formally with community Board meeting presentations Evaluation plan for MTSS for Behavior Draft document for review
District Capacity Building
2009-10 the district is working on identifying trainers who will continue providing Tier 1 training A District tertiary team planning process is now in progress to Identify how to work smarter not harder with existing school personnel who support students who engage in serious problem behavior Identify trainers for Tiers 2 and 3 who can support Student Improvement Teams onsite Discuss how to collaborate with Project Stay by partnering when individual plans are being facilitated Set the stage for internal district curriculum to support PBIS training on an ongoing basis