Transcript www.illinoisrti.org
I-RtI Network
Problem Solving Process and Aligning Policies & Procedures to RTI/MTSS
February, 2014
Facilitated/Presented by:
Insert name(s) here
The Illinois RtI Network is a State Personnel Development Grant (SPDG) project of the Illinois State Board of Education. All funding (100%) is from federal sources.
The contents of this presentation were developed under a grant from the U.S. Department of Education, #H325A100005-12. However, those contents do not necessarily represent the policy of the U.S. Department of Education, and you should not assume endorsement by the Federal Government. (OSEP Project Officer: Grace Zamora Durán)
What
Check-in
Making connections Applying
Review of January
One of the best ways to remember something is to test yourself.
January’s Fidelity Checklist Action Plans
Outcomes for Today
Participant Agenda for February 2014 EC Meeting Name:_________________________________ Date:________________ Meeting Content EC Fidelity Checklist Items Extension Activity
1.
2.
3.
Problem Analysis Deciding What to Teach Deciding How to Teach 4.
Plan Development 6. Plan Evaluation 5.
Problem Solving Process at Tier 2 Checklist 6.
7.
Review of Jan EC meeting content & field work Problem Identification Aligning Policies and Procedures with RTI/MTSS Communicating Policies and Procedures 1-13 (1,2,3,5,& 8 REQUIRED) 14. District RTI/MTSS Manual: District approach to problem-solving including overview of process, application at district/school/grade/individu al student levels, and procedures and policy checklist (REQUIRED) District RtI/MTSS manual District RtI/MTSS manual manual manual District RtI/MTSS District RtI/MTSS District RtI/MTSS manual District RtI/MTSS manual District RtI/MTSS manual District RtI/MTSS manual 8.
Other Needs
I plan to communicate my coaching and technical assistance needs to my Lead Coach by: Coaching Needs
1.
Additional training 2.
3.
4.
5.
Additional coaching to apply this content On-Site modeling Assistance with problem solving barriers Additional resources
Review Pre Meeting Survey Results
Prioritizing Activities
1. Learn about Problem Analysis for groups of students 2. Examine/evaluate a tool for assessing district/school implementation of the problem solving process at Tier 2 3. Review/work time for problem solving process section of RTI Manual 4. Learn about aligning district/school policies and procedures to RTI/MTSS
Prioritizing Activities
4. Learn tips for communicating policies and procedures to district/school staff, parents, and community 5. Create a checklist for communicating policies and procedures 6. Review each other’s RTI Manuals
I-RtI Network
THE PROBLEM SOLVING PROCESS AT TIER 2
EC Fidelity Checklist/RTI Manual
• 14. District RTI/MTSS Manual: District approach to problem-solving including overview of process, application at district/school/grade/individual student levels, and procedures and policy checklist (REQUIRED)
Problem Solving Process
Problem Identification Is there a problem? What is it?
Plan Evaluation Did the plan work?
Problem Analysis Why is it happening?
Plan Development What should be done about it?
Illinois ASPIRE
I-RtI Network
Beginning the Problem Solving Process
PROBLEM IDENTIFICATION
Problem Identification Activity
• Read p. 69-71 of Hosp & Ardoin’s “Assessment for Instructional Planning” http://www.sagepub.com/eis2study/articles/Hosp%20and%20Ardoin.pdf
• What are the important ideas in this section of the article?
• How does it relate to your understanding of identifying problems at Tier 1 & Tier 2?
• Review your district’s decision rules for problem identification at Tier 2. (Who gets Tier 2?)
Problem Identification
Is there a problem? What is it?
What’s Expected: (standard or peer median/average) What’s Occurring: (performance of this group of students) Data Source Examples:
•
Early Warning System
•
Curriculum-based measures (CBM)
•
ISAT , MAP or other group achievement tests
Risk Factors: 1. Disengagement
•20% absenteeism
2. Behind in Credits
•Particularly Core Course Failures
3. GPA less than 2.0
4. Failed FCAT Extreme Off Track
2-3 Years Behind No chance for graduation in a traditional school setting Disengagement
Example of Tier 2 Problem High Off Track
3 or more risk factors
Identification Off Track
2 of 4 risk factors indicated
Students entering with 20% absenteeism and/or 2 or mor F’s in 8 th Grade
Early Warning System
At Risk for Off Track
1 of 4 risk factors indicated
On Track
No risk factors indicated Hendry County Schools
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 15% 25%
Example of Tier 2 Problem Identification
CBM & ISAT
60% RCBM (WRC) Tier 1 Tier 2 Tier 3
Tier 2 considered when students fall in range on both
10% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 20% 60% 10% Exceeds ISAT Meets Below Acad War
Example of Convergent Data & Decision Rules Student Name Hermione Eloise Belinda Chris Darren Isabella Issac Gerald Danielle Emile Jeff Jennifer Frederica Bart Felicity Carla Griffin Annie Helen Alex Grade Teacher 6 B.Pitt
6 A.Jolie
6 B.Pitt
6 B.Pitt
6 A.Jolie
6 B.Pitt
6 A.Jolie
6 A.Jolie
6 B.Pitt
6 B.Pitt
6 B.Pitt
6 A.Jolie
6 B.Pitt
6 A.Jolie
6 A.Jolie
6 A.Jolie
6 B.Pitt
6 A.Jolie
6 A.Jolie
6 B.Pitt
Cauc AA Asian Asian Cauc Cauc Cauc AA Ethnicity Hisp AA Asian Hisp Cauc Hisp Hisp AA AA Asian Cauc Hisp FR FR Free/Red LEP FR LEP FR IEP IEP IEP IEP FR LEP LEP LEP IEP RCBM 132 135 136 140 140 147 155 165 70 82 86 98 102 108 109 117 119 120 120 120 MAP 50 62 72 75 75 88 65 86 2 10 7 10 25 15 22 33 60 38 55 36 ISAT
AW M AW Below Below Below Below M M M M M M M M Ex M M Ex M
Attend ODRs 84% 92% 95% 88% 78% 92% 84% 100% 78% 95% 95% 88% 100% 97% 98% 95% 93% 91% 94% 98%
0 0 0 0 0 0 0 0 2 1 2 1 2 0 0 0 0 1 1 1
2 0 2 2 1 0 1 0 3 4 10 6 3 3 1 3 2 7 3 5 Fail E/M
Problem Identification at Tier 2 Practice What’s Expected
= Students will score at least an 80% or better • • •
What’s Occurring
= 15 total students 11/15 students scored 80% or above 4/15 students scored below 80% • 11/15 = 73% of students are meeting • standard/ expectation 4/15 = 27% of students are not meeting standard/ expectation Is there a problem? = YES … for a small group of students in the class (Tier 2 need) J.A.
R.B.
L.C.
E.F.
D.J.
W.K.
J.M.
J.N.
D.Q.
L.Q.
K.R.
S.R.
S.S.
J.T.
T.T.
Student Score on Unit Test (out of 100%)
92% 88% 64% 52% 99% 82% 92% 82% 80% 80% 89% 90% 93% 62% 68%
I-RtI Network
PROBLEM ANALYSIS
Introduction to
PROBLEM ANALYSIS
Problem Analysis Activity
• • • Read p. 71-75 of Hosp & Ardoin’s “Assessment for Instructional Planning” While reading, note the following: – What are the key ideas in assessing for instructional planning?
– When determining performance versus skill deficits, what are the three general reasons for poor student performance? – when choosing assessments to plan instruction, what 4 things do we need to keep in mind? How do these ideas relate to your understanding of analyzing problems at Tier 2?
Problem Analysis Why is it happening?
Question: Why is the problem happening?
A.
Review data to… Differentiate between skill vs. performance problem Determine situations in which the problem is most likely and least likely to occur Identify the factors that are contributing to the problem B. Generate plausible hypotheses C.
Collect additional data as needed to validate or refute hypotheses D.
Select most validated and alterable hypotheses
What it looks like at Tier 2
• • Deciding what to teach – Determining skill deficits versus performance deficits – Strand/skill analysis Deciding how to teach – Analyzing instruction, curriculum, and environment factors
RIOT and ICEL:
Data can be collected from a number of sources:
• R = review • I = interview • O = observe • T = test
In a number of domains:
• I = instruction • C = curriculum • E = environment • L = learner
Important Ideas
• • When choosing measures to inform instruction, make sure the assessment provides information detailed enough to enable instructional planning the instrument should include ample opportunities for the student to demonstrate whether he or she can perform the skill
Collect & analyze only as much data as needed to determine what & how to teach
Problem Analysis
DECIDING WHAT TO TEACH
Can’t Do versus Won’t Do
ASSESSING SKILL & PERFORMANCE DEFICITS
3 General Reasons for Poor Student Performance
• • • Student does not have sufficient motivation to complete the work Student has not mastered the needed prerequisite skills to complete the work Student does not have sufficient motivation and has not mastered the needed prerequisite skills to complete the work Skill Deficit
(Can’t Do)
Performance Deficit
(Won’t Do/ Doesn’t Do)
Problem Analysis Flowchart
Problem Identified Do they ever successfully perform the skill(s)?
NO = Assess skill & prerequisites YES = Continue to next step NO = Assess skill & prerequisites; build plan around skill deficits YES = Assess performance factors; build plan around performance deficits Score increased to criteria?
Complete Can’t Do/Won’t Do assessment
Can’t Do vs. Won’t Do
1. Obtain previously completed assignments or universal screener. The assignment/assessment should be one the students have performed much below expectations.
2. Have students choose reinforcers (teacher approved) that each would like to work for in the future.
3. Present the assignment/assessment (answers removed) with incentive. Give reinforcer to any student who improves. If increases score by at least 20% or scores 80% or above, then move to next step.
4. Test by presenting another assignment with a reinforcer to the students. 5. Evaluate outcomes. If students markedly increase performance when offered incentives, likely WON’T DO.
6. Create incentive plan.
Joe Witt & Ray Beck, 1999
Let’s Practice Tier 2 Problem Analysis…
• 4/15 = 27% of students are not meeting standard/ expectation
Why?
Student Initial Score on Unit Test (out of 100%) Score when presented as Can’t Do/Won’t Do
L.C.
E.F.
J.T.
T.T.
64% 52% 62% 68% 70% 74% 68% 74%
Meets Performance Deficit Criteria
No Yes, but . . .
No No
Should instruction for this group focus on skills or performance/motivation? = Skills
ASSESSING PREREQUISITE SKILLS
Important Ideas
• • • • Task analysis to determine subcomponents Determine what skills are included and which are most emphasized Some curricula materials provide scope & sequence Do not need to assess all subcomponents, generally proficiency in a component higher on the scope and sequence chart indicates student(s) can adequately perform skills lower on the chart/task analysis
Assessing Prerequisites at a Group Level
SKILL ANALYSIS – STRAND, ITEM, & STUDENT WORK
Strand Analysis for Tier 2 Groups
• • • • Determine what strands are assessed & if there are any cross-cutting themes (e.g., application & problem solving) Find the standards or learning outcomes that are assessed in each strand (testing website/ manual; state standards documents; district curriculum) Find how many test items per strand Determine types of test items (multiple choice, short response, extended response) Love, et al. (2008). The Data Coaches Guide.
Data Display for Strand Analysis
Bar Graph Vertical Plot
Love, et al. (2008). The Data Coaches Guide.
Stoplight
Sample Questions for Strand Analysis
Predict
• Which strand do you expect to be the lowest? • Why? • What do you expect the trends over time to show?
Infer/Question
• What do the data show in terms of what students do and don’t understand? • How well does the test assess these strands (# of items)? • What about the instruction, curriculum, and/or environment may contribute to understanding or lack of understanding? • What policies (e.g., grouping/tracking) may contribute to poor performance? Love, et al. (2008). The Data Coaches Guide.
IIRC Example of Tier 1 Strand Analysis
Let’s Practice Tier 2 Problem Analysis… Strand
Number Sense Patterns, relations, functions Measurement Geometry Data Analysis Algebra Knowledge/skills Conceptual understanding Application/problem solving
ISAT % proficient
50% 25% 75% 75% 50% 25% 75% 50% 25%
MAP % proficient
50% 0% 50% 75% 50% 25% 50% 50% 0%
Semester Pre-Test % proficient
75% 25% 75% 75% 25% 0% 75% 75% 25%
Vertical Plot
• Create a vertical plot for the first three strands of ISAT data 100 Measurement 75% 75 Number Sense 50% 50 25 Patterns, relations, functions 25%
Item Analysis
• • Determine performance on individual items within a low strand Determine distractor patterns, rubric scores, and/or specific content and skills contained in frequently missed items Love, et al. (2008). The Data Coaches Guide.
IIRC Example of Tier 1 Item Analysis
Sample Template
Test Part Strand Standard Item # Correct Answer A B C D Blank % of Group % of Bldg
Love, et al. (2008). The Data Coaches Guide.
Task Deconstruction
• Analyzing the knowledge, skills, and big ideas/concepts required to answer a test item • Consider all of the skills and prerequisites needed to answer correctly.
Student Work Analysis
• • How does student work compare to this analysis?
What additional information does it provide for instruction?
Love, et al. (2008). The Data Coaches Guide.
Let’s Practice Tier 2 Problem Analysis… Strand Item # & skill
P,R,F P,R,F P,R,F P,R,F 1 Select the next number that fits the pattern.
2 Select the next number that fits the pattern.
3 Select the next number that fits the pattern.
4 Use the graph to select which number represents how many hamburgers were sold?
Correct Answer
B C A B
A B C D Blank % of Group
1 0 2 1 0 0% 0 3 1 0 0 1 1 1 1 0 0 2 2 0 0 25% 25% 50%
What skills should instruction focus on for this group? = % of Bldg
50% 72% 65% 84%
Problem Analysis
DECIDING HOW TO TEACH
Important Ideas When assessing how to teach. . .
• • consider the stages of learning. Could use Haring & Eaton’s model as in the Hosp article, Bloom’s taxonomy, etc.
it is primarily through formative assessment as instruction occurs (what type of instruction do they need next? – accuracy/acquisition?, fluency?
Tools for Use
ASSESSING CURRICULUM, INSTRUCTION, & ENVIRONMENT
ICE Activity
• • Review ICE Checklist How might you use this?
I-RtI Network
Problem Solving Process
PLAN DEVELOPMENT
Plan Development Activity
• • • Read p. 75-76 (“Developing Interventions” section) of Hosp & Ardoin’s “Assessment for Instructional Planning” How does it relate to your understanding of plan development at Tier 2?
How might this information impact your teams?
Important Ideas
• In addition to skills and good teaching practices, consider instructional hierarchies (e.g., Bloom’s taxonomy; Haring & Eaton) when developing an intervention
I-RtI Network
Problem Solving Process
PLAN EVALUATION
Plan Evaluation Activity
• • Read p. 76 (“Evaluating Interventions” section) of Hosp & Ardoin’s “Assessment for Instructional Planning” How does it relate to your understanding of plan evaluation at Tier 2?
Important Ideas
• • • Assess same skills as in problem identification/ screening, “since this was a skill that was determined to be a source of difficulty in the first place” Analyzing high-stakes assessments in addition provides a measure of generalization If the intervention group is not making expected progress, assess fidelity
Example of Tier 2 Plan Evaluation
5/6 students have a Trendline at/ above their Aimline = 83% of the group is making progress!!! Intervention Effective? YES!!!!
I-RtI Network
Problem Solving Process
PROBLEM SOLVING CHECKLIST – TIER 2
Problem Solving Checklist
Identify: - 1 item team is good at - 1 item team could improve on Discuss with a partner Share out whole group
RTI Manual: Problem Solving Process
• o o Item #14 on the Fidelity Checklist -- District RTI/MTSS Manual: District approach to problem-solving including overview of process, application at district/school/grade/ individual student levels, . . .
Review/reflect on your manual. o How well does it document your district’s problem solving approach?
o What might you want to change based on today’s information?
Share manuals with each other
Time to Reflect
One thing I learned during this section… One thing I would like to have clarified is… One way I could apply this learning is…
Questions/Comments
I-RtI Network
ALIGNING POLICIES & PROCEDURES WITH RTI/MTSS
Do our district’s current policies & procedures support or hinder implementation of RTI/MTSS?
Supports Hinders
Aligning Policies & Procedures Activity
• Read “Coherent alignment of policies and procedures to support implementation” article https://education.state.mn.us/mdeprod/idcplg?IdcService=GET_FILE&dDocName=049 088&RevisionSelectionMethod=latestReleased&Rendition=primary . • Discuss why alignment is important.
Important Ideas
• Each
policy, funding
and
regulatory
decision must be evaluated through an aligned set of lenses that operates from the individual student level, to the teacher or classroom level, school level and district level.
• Key Elements
Shared vision, values, & outcomes Collaboration and efficient use of resources Leadership & implementation teams at multiple levels Ongoing feedback & communication loops Data systems & supports Professional development Community engagement
Important ideas
• • • • • • Key Questions: How will this decision impact student outcomes? How will this decision support teacher confidence and competence? How will this decision support school building administrators in developing teacher confidence and competence? How will this decision impact the effective use of the Core Implementation components? How does this decision align with the current stage of implementation? How is this decision constrained by or supported by the system?
Essential Elements of RTI/MTSS to Align
• • • • Multi-tiered systems of teaching & support High quality curriculum Ongoing assessment and continuous progress monitoring Collaborative problem-solving process
Does your District currently have a Process for Policy/Procedure Alignment
• • • • When How – what will it look like – Indicators?
– Rubric? / Key Questions?
Who How – will it be communicated
Do our district’s current policies & procedures support or hinder implementation of RTI/MTSS?
Supports Hinders
I-RtI Network
COMMUNICATING POLICIES & PROCEDURES
Potential Considerations for Communicating/Disseminating Policies & Procedures
• What are our goals/outcomes for disseminating the information? • • • Will stakeholder input be sought when determining communication/dissemination plan?
What does evidence suggest is the most effective way to disseminate to the target audience? How will we ensure that the policy/procedure is disseminated in an effective and timely manner?
Communicating Policies & Procedures Activity
• • 2 teams Read ehow articles: – Group 1. How to communicate policies and procedures http://www.ehow.com/how_5869384_communicate-policies procedures.html#ixzz2qJVc4owS – Group 2. How to announce new policies and procedures http://www.ehow.com/how_5860507_announce-new-policies procedures.html#ixzz2qJWdM11X • Create a poster in pictures to convey the big ideas
Communicating Policies & Procedures Checklist
• Create a brief checklist as a group
Time to Reflect
One thing I learned during this section… One thing I would like to have clarified is… One way I could apply this learning is…
Questions/Comments
RTI Manual: Problem Solving Process
• Item #14 on the Fidelity Checklist -- District RTI/MTSS Manual: District approach to problem-solving including . . . & procedures and policy checklist o Review/reflect on your manual. o How well does it document your district’s policies and procedures approach?
o What might you want to change based on today’s information?
o Share manuals with each other
Key Ideas from Today
Ask participants to summarize.