Transcript Leadership Launch Year Two Insert Presenters Names Here>

Fall Data Review
<Insert Presenters Names Here>
Cohort 7 Schools
Fall 2012
Acknowledgements
The material for this training day was developed with the
efforts of…
– Melissa Nantais
– Steve Goodman
- Anna Harms
Content was based on the work of…
– Roland Good, University of Oregon
– Rob Horner, University of Oregon
– George Sugai, University of Connecticut
– Joe Torgesen, Florida Center for Reading Research
– Dawn Miller, Shawnee Mission School District, Kansas
Setting Group Expectations
To make this day the best possible, we need your
assistance and participation
• Be Responsible
– Attend to the “Come back together” signal
– Active participation…Please ask questions
• Be Respectful
– Please allow others to listen
• Please turn off cell phones and pagers
• Please limit sidebar conversations
– Share “air time”
– Please refrain from email and Internet browsing
• Be Safe
– Take care of your own needs
Agenda
• Gather
• Project Data
• Gathering Your Data this Year
• Study
• Analyzing Your Fall Data in Relation to Your
School Improvement Plan/Action Items from
Spring Data Review
• Plan
• Using the Problem-Solving Process to Build
Implementation Supports
• Do
• Next Steps
•
•
•
•
Green Assessment Binder
PET-R or SWEPT
Spring Data Review Participant Workbook with action items recorded
DIBELS 6th Edition & Next:
– Summary Report for 2012-2013
– Distribution Report by Subgroups for Fall 2013
– Cross Year Box Plots
• AIMSweb:
– Tier Transition Reports
– Tier Transition Reports by Subgroups
– Multi-Year Norms Chart
• SWIS:
– School Summary Reports
• PBIS Self-Assessment Survey and Team Implementation Checklist
Do you have what you need?
Today’s Materials
• Participant Workbook (1 per person)
• Team & Facilitator Guide (1 per team)
• Green Assessment Binder (1 per team)
Who will do what?
Facilitation of Today’s Work
✔ Notetaker
✔ Follow-up activity worksheet
✔ Facilitator for discussions
✔ Timekeeper and Monitor of Team Time Tasks
ALL: What are the big ideas we need to
bring back to our whole staff?
Fall Data Review
1.0 Gather
Cohort 7 Schools
Fall 2012
Remember the Purpose of MiBLSi:
MiBSLi creates capacity for an integrated
Behavior and Reading RtI(MTSS) system that
can be implemented with fidelity, is sustainable
over time and utilizes data-based decision
making at all levels of implementation support.
• MIBLSI ASKS SCHOOLS AND DISTRICTS TO USE
DATA FOR:
– Data-based decision making as part of a continuous
school improvement process to improve student
outcomes through effective/efficient implementation
of research-based practices.
• MIBLSI COLLECTS DATA FROM SCHOOLS AND
USES IT FOR:
– Project-level data-based decision making to inform
allocation of resources and effective programming
support
– Accountability to our grant funding sources
• Did you submit your special education referral
tracking form last year (2012-13)?
– Yes: Great! Don’t forget to keep on top of it this
year and to turn it in at the end of the year.
– No: If you needed to update information before
submitting, make sure that task is accomplished
(OK to use the old form that you started). If you
never started the form last year, please complete
the new form with last year’s information and
submit as soon as possible.
Gather
Getting Ready
Collect School Data
Build School Profile
Do
Implement Plan
Monitor Plan
Evaluate Plan
Student
Achievement
RtI
Plan
Develop Action Plan
One Common Voice – One Plan
Study
Analyze Data
Set Goals
Set Measurable Objectives
Research Best Practice
Gather
We want to gather information that tells
us:
– How well we are implementing/doing
something: Program Quality/Fidelity
Data
AND
– Whether what we’re doing is working:
Student Outcome Data
Assessment Binder Review
• Data Summary
• Monthly Schedules
http://miblsi.cenmi.org
Assessment Binder*
2012-2013
(What, When, Where)
• Blank Copies of
Cohort 1-7 Elementary Schools
School ______________________________
District ______________________________
Measures
• Additional Copies to
*This binder is intended for use by schools participating with MiBLSi.
Turn in (on pink paper)
Big Ideas for Gathering Data in an
Efficient, Effective Way
• Stay organized (schedule, forms readily
available for use)
• Provide training to ensure consistent, accurate
data collection.
• Get the data back in front of the people who
are doing the work. Actively use the data for
decision-making.
• Keep the results in a central location.
Universal Screening
Systems/Process Measures
SWIS Decision Flowchart
Team Time
During Spring Data Review your team reviewed the
flowcharts for data collection.
• Locate your Follow-Up Activity items related to
these flowcharts.
• Have you accomplished the action items
identified last spring?
• Are there any NEW concerns that should be
addressed between now and winter data review?
• Flag any items that have not been accomplished
and any new concerns. You will come back to
these during the PLAN section of today.
Integration with School Improvement:
Two Tracks
Track 1
Action Items from
MiBLSi/Spring Data
Review are embedded
into your School
Improvement Plan
Track 2
Action Items from
MiBLSi/Spring Data
Review have not been
included in your current
School Improvement Plan
Team Time
Use this time to be sure everyone
knows/can recall the STRATEGIES &
ACTIVITIES related to READING &
BEHAVIOR.
If these strategies and activities are
not in your School Improvement
Plan, please identify where they
could fit into next year’s plan.
Fall Data Review
2.0 Study
Cohort 7 Schools
Fall 2012
Last Year’s Data:
• Helped us to develop goals for this year.
• Will serve as a comparison to this year’s
data.
• Will let us know whether what we are
doing this year is making a difference.
This Year’s Data:
• Will help inform what we need to do
differently this year and in the future.
Study:
Data to Study – Reading
• DIBELS Next
– Summary Report
– Cross Year Box Plot
• AIMSweb
– Tier Transition Reports
– Multi-Year Norms Charts
Two DIBELS Data Systems
Center for Teaching and
Learning –
University of Oregon
(CTL-U of O)
Dynamic Measurement
Group (DMG)
Box Plot
DIBELSnet
DIBELS Data
System
Histogram
DIBELSnet
DIBELS Data
System
Summary Report
DIBELSnet: District Overview
DIBELS Data System:
Summary Report
Dynamic Indicators of Basic Early Literacy Skills
Kindergarten Summary Report
District: Example
Medford District
549C
School: Example
AbrahamSchool
Lincoln
Date: 2007-2008
Beginning
Middle
ISF
Goal: 8 initial sounds
Goal: 25 initial sounds
Students Tested
58
57
Mean (SD)
11.2 (9.1)
34 (14.4)
64% Low Risk
74% Established
16% Some Risk
23% Emerging
21% At Risk
4% Deficit
LNF
Goal: 8 letter names
Goal: 27 letter names
Goal: 40 letter names
Students Tested
58
57
55
Mean (SD)
17.2 (13.5)
42.6 (16.1)
54 (14.8)
74% Low risk
81% Low risk
84% Low risk
14% Some risk
18% Some risk
13% Some risk
12% At risk
2% At risk
4% At risk
PSF
Goal: 18 phonemes
Goal: 35 phonemes
Students Tested
57
55
Mean (SD)
42.3 (15.5)
52.2 (12.6)
91% Low Risk
91% Established
4% Some Risk
7% Emerging
5% At Risk
2% Deficit
NWF-CLS
Goal: 13 letter sounds
Goal: 25 letter sounds
Students Tested
57
55
Mean (SD)
37.8 (18.3)
56 (24.9)
96% Low Risk
96% Low Risk
2% Some Risk
2% Some Risk
2% At Risk
2% At Risk
Goal:
Goal:
57
8.2 (6.7)
3
6
55
10.2 (8.7)
4
8
WUF
Students Tested
Mean (SD)
20th Percentile
40th Percentile
Goal:
58
6.2 (4.7)
2
4
End
Kindergarten Summary Report, 06/11/2008, 1
Effectiveness
DIBELSnet: Effectiveness
of Instructional Levels
DIBELS Data System:
Summary of Effectiveness
Classroom Report/Class List
DIBELSnet:
Classroom Report
DIBELS Data
System:
Class List
Progress Monitoring
DIBELSnet
DIBELS Data System
Summary Report:
DIBELS 6th Edition / DIBELS Next
Box Plot
(with whiskers)
DIBELS 6th Edition / DIBELS Next
95th %ile
80th percentile (20% of
the students scored
above this score.)
Target Zone
Median Score
50th percentile
(The score of
the middle
student.)
5th %ile
20th percentile (20% of
the students scored
below this score)
Example Cross Year Box Plot
Tier Transition Report AIMSweb
Box Plot
(with whiskers)
AIMSweb
90th percentile
75th percentile (25% of
the students scored
above this score.)
Target Line
Median Score
50th percentile
(The score of
the middle
student.)
10th percentile
25th percentile (25% of
the students scored
below this score)
Example Multi-Year Norms Chart
Data to Study - Behavior
• PBIS Self-Assessment Survey (SAS)
• SWIS Big Five Reports
– Average Referrals Per Day Per Month Multi-Year
Graph
– Referrals by Time of Day
– Referrals by Location
– Referrals by Problem Behavior
– Referrals by Student
New website, same data!
www.pbisassessment.org
Data from PBIS
Surveys has been
transferred to the
new website (SAS,
TIC, SET, BoQ)
Same 6-Digit Login Code
Generate SAS reports
Click on “View
Report” under
the Self
Assessment
Survey
SAS Reports
Three reports:
Total Score
Subscale
Items
SAS Total Score Report
SAS Subscale Report
SAS Items Report
Quick Big 5 blends all
of the Big 5 Reports
together into a single
report.
Average Referrals Per Day Per Month
(National data lines)
SWIS summary 2010-11 (Majors Only)
4,634 schools; 2,394,591 students; 1,802,178 ODRs
25th
Percentile
ODR/100/
school day
75th
Percentile
ODR/100/
school day
.32 (.41) .21
.11
.39
626
.64 (.81) .46
.25
.79
390
818
.86 (.89) .62
.34
1.08
PreK-8
254
438
.50 (.49) .32
.19
.65
PreK-12
50
455
1.1 (3.0) .37
.18
.71
Grade
Range
Number of Mean
Schools
Enrollmen
t per
school
Mean
ODRs per
100 stud/
school day
K-6
2979
456
6-9
889
9-12
Median
ODRs per
100 per
school day
(Frank, McIntosh, & May, 2010)
Cumulative Mean ODRs Per Month
for 325+ Elementary Schools 08-09
12
Cumulative Mean ODRs
10
8
0-1
6
2-5
6+
4
2
0
Aug
Sept
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Team & Facilitator Guide for
Fall Data Review
Fall 2012
White Pages: Team
Grey Pages: Facilitator
Purpose of this Activity
• To provide a structure to the study phase of
today’s data review.
• To provide teams with a tool that can be used
in subsequent years for annual review of
student outcome and systems/process/fidelity
data.
• To link your MiBLSi work with your School
Improvement Work to raise achievement for
all students and close achievement gaps.
How to Use the Team and
Facilitator Guide
• The pages of the guide break down the steps
of studying the student outcomes and
systems/process/fidelity data collected this
fall in relation to the action plan created last
spring.
• Special notes for the meeting facilitator are
provided on pages with a grey background.
• All steps are important, but it is not necessary
to spend equal time on each step.
How to Use the Team and
Facilitator Guide
• The pages with the white background are for
the rest of the team to use.
• Most of these pages contain a table similar to
the one below:
Question
Team is looking at
Outcome
This column tells the
team the question(s)
that are answered in
this step of the study
phase.
This column tells the
team what they need
to examine for this
step.
This column tells the
team the outcome of
this step.
Critical Features of Team
Conversations During the Study Phase:
• Have conversation that revolves around the data.
• Have data available and organized in a central location.
• Ensure all team members are actively engaged in the data review
process and conversation.
• Discusses data in relation to established goals.
• Discuss strengths and weaknesses of the data.
• Discuss whether reports show that there has been growth or
sustained implementation since the last time data were collected.
• Note items for celebration to share with staff.
• Note areas of weakness for use during the planning process.
Team Time
Facilitator: Use the Facilitator
Guide to facilitate the Study of your
school’s data
Team Members: Follow the
prompts provided in your
Participant Workbook and those
provided by your Facilitator
Fall Data Review
3.0 Plan
Cohort 7 Schools
Fall 2012
Plan:
Implementation: The Missing Link
Between Research and Practice
“Once models and best practices are identified,
practitioners are faced with the challenge of
implementing programs properly. A poorly
implemented program can lead to failure as
easily as a poorly designed one.”
(Mihalic, Irwin, Fagan, Ballard, & Elliot, 2004)
59
Implementation: The Missing Link
Between Research and Practice
60
Why?
• It is easier to maintain the status quo rather
than deal with the resistance associated with
change
• Implementation of the change requires
systematic planning (often is overlooked)
• Practices/programs (innovations) are left
behind because they were poorly
implemented and in the end, people never
had to change their practice
61
Implementation Science:
Innovation Fluency
Innovation Fluency: The degree to which the
team knows the innovation (RtI/MTSS)
•
What is the research base or supporting evidence for
the RtI/MTSS?
•
What are the program and practice features? (What are
the core principles of RtI/MTSS?)
•
Intervention readiness for replication (What do we have
in place, how ready are we, and can we replicate what is
being done in one school in another?)
•
Requirements for implementation capacity (What do we
need to have in place to support RtI/MTSS?)
62
Implementation Science:
Implementation Fluency
Implementation Fluency: The degree to which
the team understands and pays attention to:
• Usability of MTSS/RtI (How does this benefit us
and students?)
• Intentionality around implementation
• Need for changing processes
• Closing the gap between research and practice
63
The Challenge…
MTSS
Framework
Evidence Based Instructional
PL
Practices
PLC’s, Grade levelCs
• Paragraph Shrinking
meetings, problem solving
• Explicit vocabulary
process
instruction
Assessments
Student
•SIMS
engagement
Evidence Based
strategies
Behavioral Supports
Interventions
• Reading Recovery • Schoolwide & Classroom
PBIS
• Readers
• Check-in Check-out
Workshop
Research Based
• Reading
Core Program
Apprenticeship
• Reading Street
• 6 Minute Solution
• Prentice Hall
“Research and field implementation
efforts tell us that RtI can work, but do not
ensure that it will work in schools…RtI is
vulnerable to the same misuse and
subsequent abandonment that has
plagued generations of educational
innovations”
(VanDerHeyden & Tilly, 2010)
Stages of Implementation
Focus
Should we
do it? Why
are we
doing it?
Stage
Description
Exploration/Ado Decision regarding commitment to
ption
adopting the program/practices and
supporting successful implementation.
Installation
Set up infrastructure so that successful
implementation can take place and be
supported. Establish team and data
systems, conduct audit, develop plan.
Initial
Implementation
Try out the practices, work out details,
learn and improve before expanding to
other contexts.
Elaboration
Expand the program/practices to other
locations, individuals, times- adjust from
learning in initial implementation.
Continuous
Improvement/R
egeneration
Make it easier, more efficient. Embed
within current practices.
Work to do
it right!
Work to do
it better!
66
The Work of Implementation
• Changing the behavior of adult human
service professionals is hard work
• It requires a systematic approach to support
behavior change of practitioners, volunteers,
administrators, and policy makers
• It requires the application of Implementation
Drivers
© Karen A. Blase & Dean L. Fixsen, 2004
67
Implementation is Loosely Defined
• Signs of this error:
– Results across buildings/grade levels are
inconsistent
– There are long delays between decisions
– There are times when action plans do not seem
to come to an “end”
(VanDerHeyden & Tilly, 2010)
How to Prevent and Repair a Loosely
Defined Model
• Agree on a plan:
– Addresses the what, when, where, who details
– Communication plans for parents, administrators,
teachers, and students are created and carried out
• Dedicate a coach to each site:
– When there isn’t a single person following
implementation details, miscommunications and
lapses of communication can occur
– Cost savings will occur because training alone has
little to no impact on implementation in the
classroom
(VanDerHeyden & Tilly, 2010)
Common RtI Implementation Errors
• The results are poor or are not known
• It takes too long to reach decisions
RECALL
• Implementers feel stuck at a phase of
implementation.
• Scaling has outpaced capacity for effective
implementation
(VanDerHeyden & Tilly, 2010)
70
Common RtI Implementation Errors
• Implementation model is loosely defined
• Interventions are not managed well
RECALL
• The school tries to run too many Tier 3
interventions
• The implementation has lost momentum after
some time period of initial enthusiasm
(VanDerHeyden & Tilly, 2010)
71
Signs that We’ve Lost our
Momentum…
• There are absences from leadership team
meetings
• There are little or no data, little or no followthrough
• Integrity problems begin to creep up – we don’t
do things the way we said we would
• Someone (inevitably) suggests a new model or
path
(VanDerHeyden & Tilly, 2010)
72
“Kisses of Death” for
Implementation
1. There is a lack of active administrative support.
2. The team is not meeting regularly or is
dysfunctional.
3. The school staff is not kept actively engaged in
the implementation.
4. No systematic planning is occurring to assure a
smooth transition from year to year.
5. More Kisses of Death
73
How to Prevent and Repair a Loosely
Defined Model
• Follow the data:
– Specify decision rules for each stage of
assessment (Tiers 1, 2, & 3)
– Follow the data to know when your
implementation is off-track
• Maintain focus on the RtI work. “Resist
chasing shiny objects.”
(VanDerHeyden & Tilly, 2010)
Inadequate Use
of Data
Poor Use of
Innovation
Reduced BuyIn
Weak
Implementation
Effects
“Unstable results are
bound to weaken the
implementation effort
and cause users to
begin to abandon
implementation.”
(VanDerHeyden & Tilly, 2010; p. 9)
Tightening Implementation
(adapted from Sweeny, 2003)
Implementation Plan
Implementation Plan
Identify the Practice/Innovation Being Implemented and Provide a Rationale for
Why the Practice/Innovation is Needed:
Implementation Plan
Implementation Plan
Who will be responsible?
When and how often will the data be reviewed?
Training:
When and how often will the implementation supports be provided?
What resources are needed?
When & how will the training take place (date, time, location, format)?
What resources are needed?
Who will be trained (teachers, paraprofessionals, itinerant staff, volunteers)?
What resources are needed (time, materials, money, space)?
Evaluating:
What follow-up, if any, will happen?
What are the intended outcomes of implementing the practice/innovation?
Implementing:
What are the implementation supports (i.e., coaching; prompting; implementation
partners)?
How will the implementation supports be established and maintained (i.e., observation
& feedback; modeling of the practice; planned co-implementation of the practice;
shared use of fidelity tools with discussion)?
How will we evaluate the outcomes? (i.e. what data sources will we use?)
How will we know if we are successful?
Who will be reviewing the data?
7/7/11
7/7/11
7/7/11
Interventions are not well managed
“Overemphasis of intervention selection
and under emphasis of intervention
management is probably the most
common error in RtI that we see.”
(VanDerHeyden & Tilly, 2010, p. 20)
Our goal is to optimize the
amount of time that students are
successfully engaged in
academic tasks.
(Archer, 2012)
81
Team Time
Facilitator: Use the Facilitator
Guide to facilitate the development
of your PLAN.
Team Members: Follow the
prompts provided in your
Participant Workbook and those
provided by your Facilitator
Fall Data Review
4.0 Do
Cohort 7 Schools
Fall 2012
Practice Profile
• In order to gauge the
levels of use for the
practice, the critical
components of the
practice need to be
identified along with
acceptable and
unacceptable variations so
implementation and
challenges can be
supported.
• A Practice Profile provides
this information.
Building Leadership Team RtI
Practice Profile
Practice: Integrated Behavior and Reading Response to
Intervention (RtI) Model of School-wide Supports.
Outcome: School-wide Implementation of RtI Practices
with Fidelity Resulting in Successful Student Outcomes
Critical Components
• Planning and Coordination of Implementation
Efforts
• Communication of Implementation Efforts to
Building Staff, School Community, and District
Administration
• Providing for Professional Development and
Technical Assistance for Building Staff
• Development of Materials, Tools, etc. for
Implementation Purposes
Team Time
Review your Implementation Plan(s) with
the Building Leadership Team RtI Practice
Profile in the participant workbook.
Given the descriptions of the “Gold
Standard” and “Acceptable Variation” of
each critical component, is there anything
that needs to be updated or added to your
Implementation Plan(s)?
What are the primary questions and interests of
our stakeholders?
Teachers
Is my teaching making a difference for my students?
Students
Am I a successful student?
District Administrators
Are we successfully supporting fidelity of
implementation within our district schools?
Business/Financial
Officers
Are we successfully providing schools with the
funding and resources they need to benefit
students?
Parents/Families
Are we working collaboratively with schools to help
our children be successful?
Community Members
Are our community schools developing successful
students and productive community members?
Communication Matrix
How are our
students
currently
doing?
Celebrations?
Needs?
Teachers
Students
District
Administrators
Business/Finan
cial Officers
Parents/Familie
s
Community
Members
Why do we
think RtI
and SWPBS
will benefit
our
students?
What exactly
are MiBLSi/
RtI/SWPBS?
This is our
proposed
plan for the
future.
Opportunities
for feedback
and joint
problem
solving
How can we
engage in this
work
together?
Effective Parent-Teacher Communication
Two Types of Communication…
One-way and Two-way
One-way communication is linear and
limited because it occurs in a straight line from
sender to receiver and serves to inform,
persuade, or command.
Two-way communication always
includes feedback from the receiver to the
sender that lets the sender know the
message has been received accurately.
In two-way communication both sender
and receiver listen to each other and work
toward a clear understanding.
(Adapted from Harcourt Assessment, 2004)
Examples
COMMUNICATION
EXAMPLES:
One-way:
Two-way:
Newsletters
Surveys
Focus groups
Informal conversations
Progress Reports (data
reports) with request
for parent response
Bulletin boards
School handbooks
Progress notes
Report cards
Problem solving and
goal-setting
Tools to Help You….
• Home/School Two-Way Communication,
goal-setting & action planning templates &
tools at:
http://miblsi.cenmi.org/MiBLSiModel/Support
/FamilySupport/FamilyWelcome/TeacherTo
olsforFamilySupport.aspx
Team Time
On a continuum of 0-5, discuss as a team
how your School Improvement Plan is
developed.
0 = Plan is developed by one individual with no
feedback or input
5 = Plan is completed at least 1 month before the due
date and opportunities for review and feedback from
staff, parents, and community members are
scheduled with time for revising the plan prior to
submission
Discuss the steps that can be
accomplished this year to move along the
continuum.
Team Time
• Effective communication & feedback loops-Complete
Communication Matrix
• Continuous evaluation of the implementation plan
and effectiveness
• Grade level meetings
• Monthly leadership team meetings
• Winter & Spring Data Review Trainings
• Strengthening and sustaining school-wide Universal
Supports
• State Implementers’ Conference (March –Lansing)
Preview
• At the end of the day we will be asking you to:
– Identify at least one aspect of your implementation
plan that our group can hold your team
accountable to by the time we meet again in
January.
– Explain how it is essential for moving your school
forward.
• Select your spokesperson and identify what that
person will share with the whole group.
• Each team will have approximately 1 minute to
share, so be succinct.
Preview (con’t)
• At the Winter Data Review we will be asking your
team to “formally” share your outcomes &
celebrations related to the one aspect of your
implementation plan you’ve identified.
• Your coach(es) and TAP will provide you with
reminders of this Assignment.
Group Share
• Identify at least one aspect of your
implementation plan that our group
can hold your team accountable to by
the time we meet again in January.
• Explain how it is essential for moving
your school forward.
The work you are doing is so
important. Thank you for being a
part of our learning community and
for all that you do for students!
Safe travels!