A School Improvement Framework for Promoting Evidence-Based Academic and Behavior Supports May 21, 2007 Closing the Achievement Gap Conference University of Connecticut, Storrs.
Download ReportTranscript A School Improvement Framework for Promoting Evidence-Based Academic and Behavior Supports May 21, 2007 Closing the Achievement Gap Conference University of Connecticut, Storrs.
A School Improvement Framework for Promoting Evidence-Based Academic and Behavior Supports May 21, 2007 Closing the Achievement Gap Conference University of Connecticut, Storrs Agenda • George – “Warm-up: CBER & Why Academic & Behavior Framework?” • Mike C. – “Early Literacy” • Mike F.L. – “Adolescent Literacy” • Brandi – “Behavior” • Sandy – “Data Driven Decision Making Session Outcomes 1. Description of CBER 2. Descriptions of features of school-wide framework for improving academic & behavioral outcomes 3. Sample of current CBER research & dissemination efforts 4. Importance of integrating academic & social behavior efforts within data driven, decision making approach What is CBER? Center for Behavioral Education & Research PURPOSE Work group organized to conduct & disseminate rigorous applied research that promotes academic & social supports for all children & youth in schools. • Formed in Spring 2005 • Approved by UConn Board of Trustees in Fall 2006 CBER Research Scientists Sandy Chafouleas Assoc. Prof. School Psych. Mike Faggella-Luby Assist. Prof. Spec. Educ. Mike Coyne Assoc. Prof. Spec. Educ. Brandi Simonsen Assist. Prof. Spec. Educ. George Sugai Prof. Spec. Educ. CBER Goals • Conduct, translate, & disseminate academic & social behavior research • Prepare personnel in application of evidence based practices & systems • Prepare leaders to conduct, translate & disseminate research • Establish & evaluate demonstrations & exemplars • Collaborate w/ researchers & practitioners • Establish & sustain Neag School of Education research priority CBER Guiding Principles Positive & Preventive Instructional Orientation Continuum of Support Academic & Social Behavior Evidence-based Socially Important Practices & Applied CognitiveBehavioral Theory Systems Perspective Typical & Real Implementers Cultural & Contextual Designing School-Wide Systems for Student Success Academic Systems Intensive, Individual Interventions •Individual Students •Assessment-based •High Intensity 1-5% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response Universal Interventions •All students •Preventive, proactive Behavioral Systems 5-10% 80-90% 1-5% Intensive, Individual Interventions •Individual Students •Assessment-based •Intense, durable procedures 5-10% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response 80-90% Universal Interventions •All settings, all students •Preventive, proactive Supporting Social Competence & Academic Achievement OUTCOMES Supporting Decision Making Supporting Staff Behavior PRACTICES Basics Supporting Student Behavior What Matters! • Contexts for teaching & learning environments • Interaction between academic & social behavior instruction • Data for informed decision making • Effective, efficient, durable, & relevant practices & systems EARLY LITERACY Mike Coyne Assoc. Prof. Spec. Educ. The Achievement Gap: Reading K 1st 2nd A School-wide Approach: Reading CONTINUUM OF SCHOOL-WIDE BEGINNING READING SUPPORT ~5% ~15% Universal Supports: Instruction with Core Reading Program For All Students ~80% of Students Individual Supports: Specialized, Individualized Intervention for Students at High Risk Targeted Supports: Supplemental Intervention for Students at Some Risk Data to Inform Instruction 18 16 14 12 10 8 6 4 2 0 + 75 10 9 -1 15 4 -1 20 9 -2 25 4 -2 30 9 -3 35 4 -3 40 9 -4 45 4 -4 50 9 -5 55 4 -5 60 9 -6 65 4 -6 70 9 -7 4 5 -4 Phoneme Segmentation Fluency 0 Frequency CBM Reading Correct Words 38% Meeting grade level benchmark 40% Not meeting grade level benchmarks 22% Significantly below grade level benchmarks Data to Inform Instruction CONTINUUM OF SCHOOL-WIDE BEGINNING READING SUPPORT Universal Supports: Instruction with Core Reading Program For All Students Individual Supports: Specialized, Individualized Intervention for Students at High Risk ~22% ~40% ~38% of Students Target Supports: Supplemental Intervention for Students at Some Risk Universal Supports: Core Instruction – Consistent “core” reading instruction of validated efficacy implemented school-wide – Core instruction focuses on “big ideas” in beginning reading (i.e., phonemic awareness, phonics, fluency, vocabulary, comprehension) – Core instruction implemented with fidelity – Consistent, prioritized, and protected time allocated to core reading instruction Targeted Supports: Intervention Examples of ways to intensify instruction Increase instructional time Provide instruction in smaller groups or one-on-one Preteach important concepts Modify program to make more explicit and supportive Monitor progress more frequently Provide program specific training or coaching to instructor Meet frequently to monitor instruction, coordinate and plan Give students more opportunities to practice skills Targeted Supports: Intervention Research Project ERI: Early Reading Intervention (2006-2010) Comparison of 3 Kindergarten Interventions 30 minutes of ERI (Early Reading Intervention) 15 minutes ERI 30 minutes of basal program Results: 30 ERI > 30 Basal & 15 ERI 15 ERI = 30 Basal Targeted Supports: Intervention Research Project IVI: Intensifying Vocabulary Intervention for Kindergarten Students at Risk of Learning Disabilities (2006-2009) All students received classroom vocabulary instruction Some at-risk students received targeted vocabulary intervention Results: Intervention > No Intervention At-risk students receiving intervention = Typical Students receiving class instruction A School-wide Approach: Reading 5.2 With targeted Reading grade level 5 4.9 intervention 4 3 Low Risk on Early Screening 2 1 At Risk on Early Screening 1 2 3 4 Grade level corresponding to age 3.2 2.5 With researchbased core instruction ADOLESCENT LITERACY Mike Faggella-Luby Assist. Prof. Spec. Educ. Critical RtI Planning Elements 1. Universal Screening 2. Evidence-based Instruction 1. Content 2. Pedagogy 3. Progress Monitoring 1. Word Recognition not enough for older students 2. Comprehension Measures 3. Process and Knowledge 4. Fidelity of Implementation Continuum of Literacy Instruction Content Literacy Continuum -- CLC Level 1: Enhance content instruction (mastery of critical content for all regardless of literacy levels) Level 2: Embedded strategy instruction (routinely weave strategies within and across classes using large group instructional methods) Level 3: Intensive strategy instruction (mastery of specific strategies using intensive-explicit instructional sequences) Level 4: Intensive basic skill instruction (mastery of entry level literacy skills at the 4th grade level) Level 5: Therapeutic intervention (mastery of language underpinnings of curriculum content and learning strategies) Content of Instruction that builds Comprehension 1. Vocabulary knowledge 2. Knowledge of text/discourse structures 3. Domain/Background knowledge 4. Cognitive Strategies 5. Increase motivation/engagement e.g., Gersten et al., 2001; Biancarosa & Snow, 2004; Torgeson et al., 2007 Intense-Explicit Instruction LEVEL 3/4/5 LEVEL 1 • Cue • Pretest • Do • Describe – Commitment (student & teacher) • Review – Goals – High expectations LEVEL 2 • “I do it!” (Learn by watching) • “We do it!” (Learn by sharing) • “Ya’ll do it!” (Learn by sharing) • “You do it! (Learn by practicing) • Model • Practice and quality feedback – Controlled and advanced • Posttest & reflect • Generalize, transfer, apply Embedded Strategy Study Design • Control-group Design with Random Assignment • 79 students (including 14 SWD) • Six 9th-grade summer school literature classes (3 Experimental and 3 Control) • Researcher conducted all instruction • Embedded Short Story (ESS) vs. Comprehension Skills Instruction (CSI) • Time: 120 minutes per day (9 days total) • Measures of Knowledge, Use and Comprehension • Material: The InterActive Reader (McDougal Little, 2001) Results: Formative Mastery Use Test Strategy-use Test 25 M ax imum S c ore 16.59 16.00 15 14.69 10 6.23 5 6.23 6.58 Control Experimental 6.02 6.08 0 nc e st M ai n Po s te na tte s Pr og re s te s t Time*Condition Pr e Raw Score 20 Wilks’ = .33, F(3,73) = 49.5, p<.001 2 = .670 Results: Summative Knowledge Test 30 22.69 Raw Score 25 20 Control Experimental 15 10 5 4.55 4.62 6.18 Time*Condition 0 Pretest Posttest Wilks’ = .38, F(1,75)=124.4, p<.001 2 =.624 Results: Summative Raw Score Unit Reading Comprehen sion Test 40 35 30 25 20 15 10 5 0 23.97 19.23 Control Experimental Time*Condition 1.63 2.08 Pretest Posttest Wilks’ = .91, F(1,75)= 7.61, p=.007 2 =.092 SOCIAL BEHAVIOR Brandi Simonsen Assist. Prof. Spec. Educ. Supporting Social Competence & Academic Achievement 4 Critical Dimensions of Support OUTCOMES Supporting Decision Making Supporting Staff Behavior PRACTICES Supporting Student Behavior “supports that are needed to enable the accurate and durable implementation of the practices” (OSEP Center, 2004, p. 14) GENERAL IMPLEMENTATION PROCESS Team Agreements Data-based Action Plan Evaluation Implementation “SW-PBS emphasizes research-validated practices, interventions, strategies, curriculum, etc. to achieve goals and outcomes” (OSEP Center, 2004, p. 11) PRACTICES Examples of PRACTICES in School-Wide Positive Behavior Support • Individualized positive behavior support strategies, based on Functional Behavioral • Efficient intervention focused on increasing Assessment (FBA), and documented in structure, self-management, opportunities for Behavior Support Plan (BSP; e.g., Crone & feedback, opportunities for reinforcement, Horner, 2003; O’Neill et al., 1997) and home-school connection [e.g., Behavior • Wrap-around Process and comprehensive Education Program (Crone, Horner, & Centered Planning (e.g., Eber, Hawken, 2004)] posting, • Person Establishing, and explicitly Sugai, Smith, & Scott, 2002; Scott & Eber, teaching a small number of positively 2003) Individual Targeted stated school-wide expectations • A school-wide system for acknowledging appropriate behavior (e.g., behavior tickets, positive office referrals, etc.) Universal “Data are used to describe, choose, and evaluate goals/outcomes” (OSEP Center on PBIS, 2004, p. 10) “Data are [also] used to guide which practices should be selected and/or adapted to achieve goals/outcomes” (p. 11) Examples of DATA in School-Wide Positive Behavior Support Individual Targeted • Total number of office discipline referrals Universal FRMS Total Office Discipline Referrals Sustained Impact 3000 Pre Total ODRs 2500 2000 Post 1500 1000 500 0 94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 05-06 Academic Years Examples of DATA in School-Wide Positive Behavior Support • Records review • Ratings of the students’ pro-socialinterviews behavior Functional behavioral assessment Individual •• Number of office discipline referrals for Direct observation data students in this group Targeted • Total number of office discipline referrals • Number of students who are suspended or expelled • Number students referred or found eligible for special education • Number of students requiring intensive mental health supports • Number of students referred for an evaluation for emotional disturbance Universal % Intervals w/ P.B. for Bryce % Intervals w/ P.B. Baseline 100 90 80 70 60 ContraIndicated Indicated ContraIndicated Indicated 50 40 30 20 10 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 Sessions* *Data points with arrows indicate no medication Data from Kimberly Ingram’s dissertation (subsequently published in JPBI) OUTCOMES “Specified academic and social behavior outcome indicators are linked to annual school improvement objectives, local and state initiative priorities, and individual academic goals and objectives” (OSEP Center on PBIS, 2004, p. 10) PBS associated w/ academic & behavior improvements: School Example -55% -85% Lovejoy Elem in Alton, IL Putting it Together Academic Support Behavioral Support Individual Supports: Targeted Supports: Individual Supports: ~5 % Targeted Supports: ~15% Universal Supports: Universal Supports: Supporting Social Competence & Academic Achievement OUTCOMES ~80% of Students Supporting Decision Making Supporting Staff Behavior PRACTICES Supporting Student Behavior DATA DRIVEN DECISION MAKING Sandy Chafouleas Assoc. Prof. School Psych. Te rtiary Le ve l Assessment focused on the individual student exhibiting the most severe levels of problem behavior Assessment efforts are highly individualized and intensive Assessment efforts must include monitoring of progress tow ard goals ~5% ~15% Se condary Le ve l Assessment focused on a select group of students deemed at-risk f or behavioral problems (i.e., academic and/or social) Assessment efforts must ef ficiently identify and monitor those problems across settings Prim ary Le ve l Assessment focused broadly on all students and settings Assessment efforts are preventative and proactive indicators of behavioral perf ormance (i.e., academic and/or social) Approximately 80% of Students Tertiary Le ve l Very Intense Efforts: Assessment focused on the individual student exhibiting the most severe levels of problem behavior (FBA) Assessment efforts are highly individualized and intensive Assessment efforts must include monitoring of progress tow ard goals Direct Observation Behavior Rating Scales Prim ary Le ve l Minimal Efforts: ~5% ~15% MoreSecondary Intense Le veEfforts: l Assessment focused on a select Extant Data group of students deemed at-risk for behavioral problems (i.e., academic Direct Behavior Ratings and/or social) Assessment efforts must efficiently Direct Observation identify and monitor those problems across settings Rating Scales Behavior Assessment focused broadly on all students and settings Assessment efforts are Whole and School preventative proactiveData indicators of behavioral performance (i.e., academic (e.g., ODRs, and/or social) State/District Assmts) Approximately 80% of Students Why do I need data? At what level should the problem be solved? Primary Secondary Tertiary What is the purpose of assessment? Evaluative Diagnostic Progress Monitoring Screening Which data do I need? Which tools are best matched to assess the behavior of interest? Frequency Flexibility What decisions will be made using these data? High Stakes Low Stakes What resources are available to collect the data? Training Time Intrusiveness Cost Which tools can answer these questions? Exam ining the Categoriesf oBehaviorAssessment Tools with Regard to Progress Monitoring. Guiding Qu estion P ermanent Behavior Direct Direct Behavior P roducts Rating Scales Observation Ratings At what level do I P rimary P robably Maybe Maybe Maybe need progress Secondary P robably Maybe P robably P robably monitoring data? Tertiary P robably Maybe P robably P robably made using these High Stakes Maybe P robably P robably Maybe progress mo n itoring Low Stakes P robably Maybe P robably P robably matched to monitor Frequency High Low High High the progress of Flexibility Medium Low High High W hat decisions will be data? W hich tools are best behavior ofinterest? Case Example • While reviewing ODR data from the past 3 months, the principal at Pine Grove School notices that Bus #7 has a disproportionate number of ODRs relating to compliance with adult request. The driver reports difficulty maintaining an acceptable level of noise on the bus. Because “all” the students on his bus are much too loud and do not listen to him when asked to lower their voices, he has been handling the problem by writing students ODRs. The principal and bus driver agree this problem should be addressed before it escalates. The student service team is asked to develop a primary intervention plan. Selection of appropriate data collection tools • Although ODR data will continue to be used, ODRs do not reflect the behavior that is less intense (i.e., annoying to the driver) • The team asks the bus driver to use direct observation by counting the number of reprimands given over each bus ride when students have excessively loud voices. (SDO in this case allows for high flexibility and frequency) • A golf counter is used to make data collection easier for the driver to manage • When intervention plan is determined to be successful, SDO is discontinued. ODR data continue to be reviewed periodically to ensure the problem has been resolved. CONCLUDING COMMENTS George Sugai Prof. Spec. Educ. Supporting Social Competence & Academic Achievement OUTCOMES Supporting Decision Making Supporting Staff Behavior PRACTICES Basics Supporting Student Behavior Designing School-Wide Systems for Student Success Academic Systems Intensive, Individual Interventions •Individual Students •Assessment-based •High Intensity 1-5% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response Universal Interventions •All students •Preventive, proactive Behavioral Systems 5-10% 80-90% 1-5% Intensive, Individual Interventions •Individual Students •Assessment-based •Intense, durable procedures 5-10% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response 80-90% Universal Interventions •All settings, all students •Preventive, proactive Designing School-Wide Systems for Student Success Academic Systems Intensive, Individual Interventions •Individual Students •Assessment-based •High Intensity 1-5% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response Universal Interventions •All students •Preventive, proactive Behavioral Systems 5-10% 80-90% 1-5% Intensive, Individual Interventions •Individual Students •Assessment-based •Intense, durable procedures 5-10% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response 80-90% Universal Interventions •All settings, all students •Preventive, proactive Designing School-Wide Systems for Student Success Academic Systems Intensive, Individual Interventions •Individual Students •Assessment-based •High Intensity 1-5% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response Universal Interventions •All students •Preventive, proactive Behavioral Systems 5-10% 80-90% 1-5% Intensive, Individual Interventions •Individual Students •Assessment-based •Intense, durable procedures 5-10% Targeted Group Interventions •Some students (at-risk) •High efficiency •Rapid response 80-90% Universal Interventions •All settings, all students •Preventive, proactive CONTINUUM OF SCHOOL-WIDE INSTRUCTIONAL & POSITIVE BEHAVIOR SUPPORT Tertiary Prevention: Specialized Individualized Systems for Students with High-Risk Behavior Secondary Prevention: Specialized Group Systems for Students with At-Risk Behavior Primary Prevention: School-/ClassroomWide Systems for All Students, Staff, & Settings Mean ODRs per 100 students per school day Illinois and Hawaii Elementary Schools 2003-04 (No Minors) Schools using SW-PBS lower rate of ODRs Mean ODR/100/Day report a 25% 1 0.8 .85 .64 0.6 0.4 0.2 0 N = 87 N = 53 Met SET 80/80 Did Not Meet SET Illinois 02-03 Mean Proportion of Students Meeting ISAT Reading Mean Percentage of 3rd graders meeting ISAT Reading Standard Standard t test (df 119) p < .0001 70% 62.19% 60% 50% 46.60% 40% 30% 20% 10% 0% PBIS NOT in place N = 69 PBIS IN place N = 52 Proportion of Students Meeting Reading Standards Proportion of 3rd Graders who meet or exceed state reading standards (ISAT) in Illinois schools 02-03 t = 9.20; df = 27 p < .0001 1 0.8 0.6 0.4 0.2 N = 23 N =23 NN= =8 8 0 Not Meeting SET Meeting SET www.education.uconn.edu www.pbis.org www.cber.org School Improvement Framework for Promoting EvidenceBased Academic & Behavior Supports [email protected] [email protected] [email protected] [email protected] [email protected]