Curriculum-Based Measurement and Its Use in a Problem

Transcript Curriculum-Based Measurement and Its Use in a Problem

OVERVIEW of the PROBLEM-SOLVING MODEL :

Implementing Science- and Practice-Based Solutions to Long-Standing Problems Including Response to Intervention (RtI) --------------------------------------------------------------------------- Illinois ASPIRE A

lliance for

chool-based

roblem-solving &

ntervention

esources in

ducation Illinois ASPIRE is a State Personnel Development Grant-funded initiative of the Illinois State Board of Education. All funding is from federal sources.

Illinois ASPIRE

lliance for

chool-based

roblem-solving &

ntervention

esources in

ducation

Project Goal :

Establish and implement a coordinated, regionalized system of personnel development that will increase school systems ’ capacity to provide early intervening services [with an emphasis on reading], aligned with the general education curriculum, to at-risk students and students with disabilities, as measured by improved student progress and performance.

Illinois ASPIRE

lliance for

chool-based

roblem-solving &

ntervention

esources in

ducation

Objectives:

Deliver research-based professional development and technical assistance in Problem-Solving Service Delivery Systems, Response-to-Intervention (RTI), scientifically based reading instruction, and Standards Aligned Classrooms (SAC).

Increase the participation of parents in decision-making across district sites.

Incorporate professional development content into higher education general and special education preservice & graduate level curricula.

Evaluate the effectiveness of project activities.

Objective 1: Research-Based Professional Development & T.A.

• 4 Regional Illinois ASPIRE Centers – Chicago: Chicago Public Schools – North: Northern Suburban Spec. Ed. Dist.

– Central: Peoria ROE #48 – South: Southern Illinois University • Collaboratives of LEAs, IHEs, regional providers and parent entities • Responsible for: – Training to districts and parents in region – General technical assistance (T.A.) – On-site T.A. to school data collection/demonstration sites

For More Information

www.illinoisaspire.net

SBE: Kathryn Cox 217-782-5589 [email protected]

Illinois ASPIRE – North Dr. Mark Shinn 847-275-7200 [email protected]

Illinois ASPIRE – Central Christy Culen 309-657-9337 [email protected]

Illinois ASPIRE – South Dr. Melissa Bergstrom 618-650-3182 [email protected]

Illinois ASPIRE – Chicago Amy Dahlstrom Klainer 773-553-2209 [email protected]

Presentation Intended Outcomes

Acquire model.

necessary background information

on Problem-Solving, RtI, and Scientifically Based Intervention and Progress Monitoring in a 3-tier Identify and use

key vocabulary

and

concepts

Understand the

problem solving steps

identify

key tools

. and See

examples of problem-solving implementation

at the level of the school, targeted group, and individual students.

Identify

activities and targets for year 1 implementation, including the role of the team

The Big Ideas

• It’s about “Every Ed” and school improvement • It’s about early intervention and a systems approach to meeting the needs of all students using scientific, research-based instruction and interventions • We need more options for meeting student needs than just special education • Data-based decision making is key • Problem solving is a process that can be used at every level of the system to make individual, group and system-level decisions

Bigger Picture:

Why do we want to implement?

Previous Policy:

Process-Driven; Outcomes Lacking

Current Policy: Outcomes-Driven; All Students Future Experience:

Full Implementation

Problem Solving including RtI Problem Solving Model Data-Based Decision Making 3-Tier Model Scientifically-Based Data Systems Scientifically-Based Interventions

Traditional Approach to Service Delivery

Special Education

Amount of Resources Needed to Benefit

General Education

Severity of Educational Need or Problem Sea of Ineligibility

Old System of Problem Solving…

Referral Driven 3 Tier System and Referral Driven

Still Wait to Fail Highly teacher dependent, some under refer, some over-refer Prevention-driven through Universal Screening (Benchmarking) and individual referrals Not dependent on referral; students not benefiting automatically referred Often teams changed names (TAT to PAT) Roles and functions change to Tools, but roles remained same (child focused) Training, Support Still seen as a ‘ hoop ’ to SE eligibility (2 Tiers) Focus on effective interventions in a 3-Tier model Solutions were 1-at-a-time, weak, Process Solutions come first to groups, and from time-consuming and cumbersome better Tools, Training, and Support

The VISION: To Provide Effective Interventions to Meet the Needs of ALL Students through Early and Scientifically Based Interventions through Careful Systems Planning

The Vision:

Building a System of Substantial Instructional Interventions to Reduce the Gap

5 5.2

4.9

With substantial instructional intervention 4 3 Low Risk on Early Screening 3.2

With research based core but without extra 2.5

instructional intervention 2 1 At Risk on Early Screening 1 2 3 4 Grade level corresponding to age

Big Ideas About a Problem-Solving Model

Is a

3-tiered service

(typically)

delivery system

designed to redress

practical and scientific concerns regarding general and special education

Is based on

foundational concepts of prevention

and

effective and intensive, needs-based intervention

Uses

scientifically based intervention progress monitoring practices

and 4.

Has been

field-tested

with a

history of more than 25 years of implementation

in schools, communities, and states across the country

No Child Left Behind (NCLB)

• •

Increase Accountability for Student Performance

: – States, districts and schools that improve achievement will be rewarded.

– Failure will be sanctioned. – Parents will know how well their child is learning, and that schools are held accountable for their effectiveness with annual state reading and math assessments in grades 3 8.

Focus on What Works

: – Federal dollars will be spent on effective, research based programs and practices. – Funds will be targeted to improve schools and enhance teacher quality.

No Child Left Behind (NCLB)

•

Reduce Bureaucracy and Increase Flexibility

:

– Additional flexibility will be provided to states and school districts, and flexible funding will be increased at the local level.

•

Empower Parents

:

– Parents will have more information about the quality of their child ’s school. – Students in persistently low-performing schools will be given choice.

IDEA 2004 Regulations LD Eligibility Determination

34 CFR §300.306

(b)

Special rule for eligibility determination.

child must not be determined to be a child with a disability under this part A (1) If the determinant factor for that determination is — (i) Lack of appropriate instruction in reading,

including in the essential components of reading instruction

(as defined in section 1208(3) of the ESEA of 1965); (ii) Lack of appropriate instruction in math; or (iii) Limited English proficiency.

IDEA 2004 Regulations LD Eligibility Determination (cont.)

34 CFR § 300.307

(a)

General

. A state must adopt…criteria for determining whether a child has a specific learning disability…. In addition, the criteria adopted by the State — (1) Must

not

require the use of a severe discrepancy between intellectual ability and achievement…; (2)

Must

permit the use a process based on the child ’s response to scientific, research based intervention;…

IDEA 2004 Regulations LD Eligibility Determination (cont.)

34 CFR § 300.309

(b) To ensure that underachievement in a child suspected of having a specific learning disability is not due to lack of appropriate instruction in reading or math, the group must consider — 1) Data that demonstrate that prior to, or as part of, the referral process the child received appropriate instruction in regular education settings from qualified personnel 2) Data-based documentation of repeated assessments of achievement at reasonable intervals, reflecting formal assessment of student progress during instruction, which was provided to the child ’s parent

• •

IDEA 2004 Regulations Use of Funds

34 CFR

300.224

LEAs may use up to 15% of their IDEA funds to develop and implement early intervening services

for K-12 students not identified as needing special education or related services but who need additional academic and behavioral support to succeed in the general education environment.

34 CFR

300.646 Disproportionality

LEAs with identified significant disproportionality based on race or ethnicity must use the maximum (15%) amount of their IDEA Part B funds to provide early intervening services

Illinois State Regulations: 23 IL Admin Code 226.130 Additional Procedures for Students Suspected of or Having SLD

b) Provided that the requirements of this subsection (b) are met, each district shall , no later than the beginning of the 2010-11 school year, implement the use of a process that determines how the child responds to scientific, research-based interventions as part of the evaluation procedures described in 34 CFR 300.304. When a district implements the use of a process of this type, the district shall not use any child ’s participation in the process as the basis for denying a parent ’s request for an evaluation.

Built to Resolve Known Problems in Service Delivery

Known Problem Problem-Solving Solution The

“

Gap

”

between successful readers and At Risk students grows over time, leaving some students behind Early Identification, Intervention using Scientifically Based Core Programs with intervention supports General Education has seen Special Education as the primary support option for learning and behavior problems Provide a better range of Tools, Training, and Support for all students, teachers, educators Using Special Education as the primary support has created an enabling system, decreasing the likelihood that we provide General Education teachers the best Tools, Training, and Support Universal Screening and Progress Monitoring in all 3 Tiers to identify what works; Using student outcomes for deciding who receives what, when; Providing better Tools, Training, and Support Special Education is not as effective as it could be, increasing risk of drop out and low exit rates Ensuring that when a student receives SE, they get the best of the best of Tools, Training, and Support and a Data System that ensures accountability

Known Problem Problem-Solving Solution Referral is teacher dependent; Some under-refer, some over-refer Universal Screening to ensure students with needs are addressed early and effectively Wait to Fail eligibility process Continuous Universal Screening, Progress Monitoring, and Early Intervention Assessment focus on eligibility and identifying the disability Consuming focus- effective interventions Time consuming assessment (lots of Efficient and direct assessment targeted data and reports) but not related to to identifying

“

What Works

”

intervention

Known Problem Problem-Solving Solution Highly educated personnel used primarily for eligibility Long, time-consuming meetings targeted to

“

round robin

”

jargony presentations or focussed on unalterable variables Disproportional and/or restrictive placements Valuing compliance Highly educated personnel used for supporting intervention or direct service Targeted discussions focussed on decision making, task delegation, and intervention outcomesß Building and supporting effective early intervention in general and remedial programs Scientifically based IEP goals and progress monitoring focussed on results

The Problem-Solving Model Has 4 Key Beliefs 1.

General education

, even with quality tools, training, and support, cannot meet the educational and behavioral needs of

all students .

Some

students:

--will not attain what we ’ d like --unless substantial changes are made in their educational programs

It is a

value judgment

as to when performance discrepancies are so severe that

additional services

beyond general education

are warranted

Because

effective interventions cannot be predicted

with certainty, we

must try out our programs and monitor progress

to see if they work.

Foundational Concepts

Foundational Concepts, Vocabulary, and Tools

Problem Solving is about:

Thinking Differently

About problems, causes, and solutions (Concepts)

Talking Differently

About problems, causes, and solutions (Vocabulary)

Doing Some Things Differently

(Tools)

Let ’s Start with a Name

• Problem Solving • Flexible Service Delivery System (FLEX or FSDS) • Response to Intervention (RtI) • Needs-Based Service Delivery

Are these the same or different?

Needs-Based Service Delivery System

A broad family of approaches that attempt to match the needs of students to interventions.

Problem-Solving (PS) Model

A type of needs-based service delivery system with a core of data-based decision making designed to improve educational and behavioral outcomes for all students.

FLEX

Illinois ’ version of a PS model begun in the mid-late 1990s.

RtI

Identifying and providing high quality instruction and research-based interventions matched to all students ’ needs and measuring progress and rate of improvement (ROI) over time to make important educational decisions

So...WHAT is RtI?

A data-based system for addressing

all students ’ needs

with an emphasis on

improved outcomes.

A process that can be used to

determine special education eligibility.

opportunity to redress years of dissatisfaction

with both special education and general education?

We See IT as ALL of these.

How We See It Needs-Based Service Delivery Systems Problem Solving Service Delivery System Flexible Service Delivery System RtI

Problem-Solving Solutions are Not “New:” More than 27 Years of School-Based Implementation Problem Solving Experience Translating Research into Practice

New Problem Solving System…

General Education with Intensive

(may include special

Support General Education

education services)

with Support General Education

Amount of Resources Needed To Benefit Severity of Educational Need or Problem

Problem Solving is Tied to 4 Major Questions

What is the problem?

Did it work?

Why is it happening?

What should be done about it?

Batsche, G. M., Elliott, J., Graden, J., Grimes, J., Kovaleski, J. F., Prasse, D., et al. (2005).

Response to intervention: Policy considerations and implementation

. Alexandria, VA: National Association of State Directors of Special Education, Inc.

The Questions Lead to Decisions

Problem Identification Progress Monitoring Problem Analysis Intervention Planning Batsche, G. M., Elliott, J., Graden, J., Grimes, J., Kovaleski, J. F., Prasse, D., et al. (2005).

Response to intervention: Policy considerations and implementation

. Alexandria, VA: National Association of State Directors of Special Education, Inc.

Problem Solving is a Way of Thinking Ecologically

A Problem is Defined as…

Discrepancy

between what is

EXPECTED

and What is

OCCURRING IN A GIVEN SITUATION

IS THERE A PERFORMANCE DISCREPANCY?

In Any Instance We Need to Know

What is Expected..

What is Occurring..

The Situation…

Problem = E-O/S

Then we can Measure the PERFORMANCE DISCREPANCY or “Educational Need”

Problem Identification:

What ’s the Problem?

Is there a discrepancy in performance between what is expected and what is occurring in a given situation?

Example of a Problem

Definitional Component

What Is Expected

Example Pass the ISAT

What Is Occurring The Situation

Not Passing the ISAT End of Grade 3 in Illinois

Example of a Problem

Definitional Component

What Is Expected

Example Reading 85 WRC at the beginning of Grade 3

What Is Occurring The Situation

Reading 35 WRC (10th Percentile) at the beginning of Grade 3 Beginning of Grade 3 in Anywhere, IL

Problem Definition also Applies to Behavior

Definitional Component Example

What Is Expected

All students are engaged and successful 90% of the time

What Is Occurring The Situation

John is engaged only 20% of the time and is not successful in his work Ms. Jones ’ Group Grade 3 Math

Problem Analysis

Requires an Ecological Perspective....

-Not Unalterable Variables -Not Just the Student

Learning problems results from a complex interaction between several factors: curriculum, instruction, the environment and learner characteristics.

”

(Howell, 1993)

Intervention Planning:

What do we do?

What is the intervention plan and how are we going to monitor progress?

Intervention Planning: What are We Going to Do About It?

Student Name___At-Risk 1st Graders________ Teacher Name________________ School Year __04-05____ Goal ______100% of 1st Graders will be reading at least 40-60 WRC in 1min.by Spring Benchmark..__________

Instructional Strategies Materials Arrangement Time Motivational Strategies Assessment Procedures Skill Decoding/Encoding (Working with Words) Vocabulary, Fluency (Teacher Read-Aloud Silent Reading) Comprehension (Guided Reading) Fluency & Comprehension (Guided Reading Groups) All 5 Nat ’ l Reading Panel Skills Teaching Strategy Teacher-Led Instruction Independent Teacher-Led Independent Teacher-Led Instruction Independent Teacher-Led Instruction Teacher-Led Instruction Word Walls Word Cards Teacher Selected Stories/Books on Theme Leveled Books Big Books Trade Books Leveled Books Read Well 19:1 Independent 19:1 Independent 19:1 Independent 3-6:1 3-6:1 At-Risk Students Only 20 mins. daily 30 mins. daily 20 mins. daily for At Risk Students 20 mins. daily 20 mins. 3x wk 20 mins. 4x wk Verbal Praise Verbal Praise Leveled Benchmarking Spelling Test Leveled Benchmarking Class Discussion Verbal Praise Leveled Benchmarking Verbal Praise Class Discussion Leveled Benchmarking Class Discussion Verbal Praise CBM 1x monthly 10/03 Adapted from the U of Oregon

Progress Monitoring: Did It Work?

3-Tier Model

Tier 3 Intensive Tier 2 Supplemental Tier 1 Core

New Problem Solving System…

Plan Development Education General Education Supplemental With Support Tier 1 Core General Problem Problem Identification

Amount of Resources Needed To Benefit Severity of Educational Need or Problem

Foundational Tools

• • Scientifically Based

Data

• • •

Systems

CBM and Members of the CBM “Family” like DIBELS Universal Screening/ Benchmarking Progress Monitoring Scientifically Based Academic and Behavior

Interventions

• • • • • • Standard Problem Solving Tasks and Decisions

Problem-Solving Self Study Instruments

Academic Problem Identification Interview Behavior Problem Identification Interview

Fidelity of Implementation

Process

Systematic Observation

Tools: Scientifically Based Data Systems

www.studentprogress.org

http://www.pbis.org/main.htm

Standards for Scientifically Based Progress Monitoring Have Been Established Reliability Quality of Good Test Validity Quality of Good Test Sufficient Number of Alternate Forms and of Equal Difficulty Evidence of Sensitivity to Improvement or to Effects of intervention Benchmarks of Adequate Progress and Goal Setting Essential for Progress Monitoring Critical for Progress Monitoring Critical for Progress Monitoring Rates of Improvement are Specified Critical for Progress Monitoring Evidence of Impact on Teacher Decision Making instruction or Student Achievement; Evidence of Improved Instruction and Student Achievement; Critical for Formative Evaluation Gold Standard for Progress Monitoring Logistically Feasible--Low Cost, Efficient, Accurate, Critical for IMPLEMENTATION

Most Tools that Meet Standards are Members of the Curriculum-Based Measurement (CBM)

“

Family

”

DIBELS or AIMSweb Are Members of the CBM “Family”

Meet National Student Progress Monitoring Center Technical Standards http://www.studentprogress.org/chart/chart.asp

Meet Reading First Assessment Standards Executive Summary of Final Report on Reading First Reading Assessment Analysis http://idea.uoregon.edu/assessment/index.html

Are Used in Universal Screening and Progress Monitoring in a 3-Tier Model

St udent Scores: Correct W ords per m in.

180 169 160 160 157 150 146 140 140 120 130 125 120 115 100 100 94 90 80 80 70 65 60 40 20 43 0 Su e Jo hn Bi ll Ja m ie Sa ra h Jo hn Tr av is Ka te ly n An na Ge or ge El le n Ad ria n Am el ia He id i Lin ds ay La ur a Ch uc k

Student Scores Range from 169 to 43

52 Words per min.

Student Scores- Correct Words per Minute

BOXPLOT 90%ile 75%ile 50%ile 25%ile 10%ile 169 160 157 150 146 140 130 125 120 115 100 94 90 80 70 65 43 Above 90%ile

Box Plot draws a box around the range of student scores: 169-43

Below 10%ile 53

Benchmark Assessment in General Education Allows Universal Screening and Progress Monitoring and Meets Federal Regulations Educational Need Reduced with Clear Educational Benefit Identified as At Risk and Early Intervention

(2)

Data-based documentation of repeated assessments of achievement at reasonable intervals

, reflecting formal assessment of student progress during instruction,

was provided to the child's parents.

SAME

Accountability System at Tier 2 But

More Frequent Monitoring

Risk of Students At

For students with IEPs receiving Tier 3 interventions, frequent monitoring can be used as a core component of IEP Annual Goal Review

What Assessment Systems Does Your District/School/ Grade Level Use for Each Purpose?

Essential components Screening (Problem Identification) Diagnostic (Problem Analysis) Progress M onitoring (Plan Development & Implementation) Outcome/ Accountability Reading Behavior

Taken from Heartland AEA 11

• • • •

Purposes of Assessment

Who has problems? (Problem Identification) Why is the problem is occurring? (Problem Analysis) Is our instruction working to fix the problem? (Plan Development & Implementation) How well are we doing overall? (Plan Evaluation)

Assessment Plan Grid Activity

• Put each of your district assessments into the appropriate column(s).

• Use the “Assessment Plan Grid” worksheet packet.

• • Note gaps and overlaps Consider recommendations and plan next steps

Essential components Phonemic Awareness Screening Phonics Diagnostic Progress M onitoring Outcome/ Accountability Fluency

Taken from Heartland AEA 11

Vocabulary Comprehension

In Your Group… • What process is used in your school to determine who needs extra support and/or special education?

• What about the process is going well?

• What can be improved?

Tools: Scientifically Based Interventions

Examples of TIER 1 : Benchmark/Core Reading Programs That Meet NRP Standards *

: ~5% ~15% Trophies (Harcourt School Publishers, 2003) The Nation ’ s Choice (Houghton Mifflin, 2003) Macmillan/McGraw-Hill Reading (2003) Open Court (SRA/McGraw-Hill, 2002) Reading Mastery Plus (SRA/ McGraw-Hill, 2002) Scott Foresman Reading (2004) Success For All (1998-2003) Wright Group Literacy (2002)

Reviewed by

Oregon Reading First Review of Comprehensive Reading Programs

: Addressed all 5 areas and included at least Grades K-3 http://oregonreadingfirst.uoregon.edu/curriculum_review.php

~80% of Students

~5% ~15% ~80% of Students

Examples of TIER 2 Strategic/Supplemental Reading Programs *

Early (Soar to) Success (Houghton Mifflin) Read Well (Sopris West) Reading Mastery (SRA) Early Reading Intervention (Scott Foresman) Great Leaps (Diamuid, Inc.) REWARDS (Sopris West) Ladders to Literacy (Brookes) Read Naturally Peer Assisted Learning Strategies (PALS)

Reviewed by

Oregon Reading First Review of Comprehensive Reading Programs

: Addressed all 5 areas and included at least Grades K-3 http://oregonreadingfirst.uoregon.edu/curriculum_review.php

~5% ~15% ~80% of Students

Examples of TIER 3 Reading Programs *

Corrective Reading (SRA) Language! (Sopris West) Wilson Reading System Reading Mastery Earobics (phonics/phonemic awareness; Cognitive Concepts) Great Leaps/ Read Naturally (Fluency) REWARDS (Fluency, Comp. and Vocab. in Plus Program) Soar to Success (comp.)

Reviewed by

Oregon Reading First Review of Comprehensive Reading Programs

: Addressed all 5 areas and included at least Grades K-3 http://oregonreadingfirst.uoregon.edu/curriculum_review.php

In Your Group… • What interventions do you have in place for everyone (core instruction), for students who are having some difficulty (strategic instruction), and for students who are having a lot of difficulty (intensive instruction)?

• What data do you have to indicate who needs strategic and intensive instruction?

Tool: Problem-Solving Self Study Instruments

Some Universal Standard

Initial Planning (Baseline) Tasks

Collect Referral Information Notify Parent Review Records Problem Identification Interview with Parent Problem Identification Interview with Teacher Obtain Description of Current Intervention Determine Educational Needs and the Performance Discrepancy 8. Observe Student and Instruction 9. Develop Progress Monitoring Plan 10. Collect Progress Monitoring Data 11. Observe Fidelity of Implementation of Current Intervention 12. Decide If Educational Need and Benefit Warrants a Modified Intervention Plan Districts adopt, adapt, drop, or add the tasks to fit their Problem Solving model .

**Some Universal Standard Intervention Tasks**

Assess for Intervention Planning (CBE) Develop Intervention Obtain Resources/Provide Training for Intervention Develop Progress Monitoring Plan Collect Progress Monitoring Data Support and Implement Intervention Observe Fidelity of Implementation of Current Intervention Decide If Educational Need and Benefit Warrants a Modified Intervention Plan or Potential Need for an IEP Districts adopt, adapt, drop, or add the tasks to fit their Problem-Solving model.

Academic Problem Identification Interview

Instructional Planning Form

Student Name_______________________ Teacher Name________________ School Year ____________ Goal ___________________________________________________________________________________ Instructional Strategies Materials Arrangement Time Motivational Strategies Assessment Procedures Skill Teaching Strategy

Kindergarten Instructional Planning Form

Student Name_______________________ Teacher Name________________ School Year ____________ Goal ___________________________________________________________________________________ Instructional Strategies Skill Oral Language Teaching Strategy Teacher-Led Activities Materials Books Letter Names Concepts about Print Vocabulary & Oral Comprehension Teacher-Led Activities & Independent work Teacher-Led Activities & Independent work Teacher Reads Aloud Books, Manipulatives, Worksheets Books Teacher Selected Stories/Books Arrangement 18:1 18:1 18:1 18:1 Time 50 mins. week 30 mins. week 20 mins. week Motivational Strategies Verbal Praise Verbal Praise Verbal Praise 10-20 mins. daily Verbal Praise Assessment Procedures Observation Written work accuracy Observation Class Discussion

K Instructional Planning Form Student Name_________K Students_____ Teacher Name________________ School Year ____04-05____ Goal __All K students meet DIBELS benchmark criteria.___________________________________________

Instructional Strategies Materials Arrangement Time Motivational Strategies Skill Teaching Strategy Assessment Procedures Listening Teacher-Led Activities *Teacher-made materials Taped Sounds Musical Instruments Bingo Cards 18:1 4/5:1 10 mins. daily Verbal Praise Observation of accuracy during activities DIBELS Syllabication Auditory Blending Phoneme Segmentation Teacher-Led Activities Teacher-Led Activities Teacher-Led Activities *Teacher-made materials Student name& picture cards Unifix cubes *Teacher-made materials 2-3 syllable & 2-4 phoneme picture cards Bingo cards *Teacher-made materials Unifix cubes Paper dollars 18:1 4/5:1 18:1 4/5:1 18:1 4/5: 1 10 mins. daily 10 mins. daily 10 mins. daily Verbal Praise Verbal Praise Verbal Praise Observation of accuracy during activities DIBELS Observation of accuracy during activities DIBELS Observation of accuracy during activities Phoneme Manipulation Teacher-Led Activities *Teacher-made materials Paper squares 18:1 4/5:1 10 mins. daily Verbal Praise DIBELS Observation of accuracy during activities 72 DIBELS *Teacher-Made Material Activities were taken from programs such as Phonemic Awareness in Young Children by Marilyn Jager Adams, et. al.

TREATMENT INTEGRITY/FIDELITY CHECKLIST EXAMPLE

I ntervention I ntegrity Checklist Check the boxes considered in developing your intervention integr ity. Sign and date on the bottom of the form. 1. 1 . I nt ervention is focused on area(s) of concern 1. 2 . H igh Probability I nterventions a. E mpirically supported b. I nterventions are easy c. I nterventions are positive (constructive/educative approaches) d. A re at child’s instructional level

3. T reatment I ntegrity checks or intervention monitoring systems are employed:

P articipant reports

Outside sources

E v a luation of permanent products

I nt ervention script

Guided Practice/M odeling

4. C onsult and Support

q q

W ho W hen

F requency development of an intervention integrity plan. Definitions of Checklist Guidelines The following are definitions of the four areas above. These can be used to help in the 1. The intervention chosen is tied to the area of concer n listed in your problem-solving. I t is targeted to create behavior change and/or enhance the child’s educational development. 2. This refers to choosing interventions that have a high probability of success. Successful interventions are usually supported by resear ch and have been utilized with positive results in an applied setting (such as a school.) I nterventions should be free from complicated steps, positive rather than punitive in nature, and designed at the child’s instructional level to lead to improved performance and skills. 2. These are ways to support the interventionist(s). Suppor t can include: having the interventionist report back as to how the intervention is going, having an outside person come and observe the intervention, evaluating the permanent products of the intervention (graphs, charts, etc), writing a script or outline for the inter vention, or practicing/modeling the inter vention before starting it. Please see team leader for examples of checklists/scripts.

Systematic Observation

Naturalistic Observation •ABC •Frequency/Event Recording •Duration Recording •Latency Recording •Time-Sampling Interval Recording •Whole Interval •Partial Interval •Momentary Time-Sampling Systems •BOSS •STIC 74

**Specify Initial Planning Standard Decisions**

No significant educational need

Maintain current intervention and evaluate rate of progress using benchmark assessment

Some significant concerns but current intervention may be effective

Maintain current intervention and continue frequent progress monitoring

Some significant concerns and intervention needs modification

Change current intervention and evaluate rate of progress with frequent progress monitoring

Significant educational needs that team believed could not be remedied alone with scientifically-based, resource-intensive general education intervention

Requires immediate consideration of special education needs and IEP

Districts adopt, adapt, drop, or add the decisions to fit their Problem-Solving model.

**Specify Intervention Standard Decisions**

Some significant concerns but current intervention may be effective

Maintain current intervention and continue frequent progress monitoring

Some significant concerns and intervention needs modification

Change current intervention and evaluate rate of progress with frequent progress monitoring

Significant Educational Needs that Team Believed Could Not Be Remedied with Scientifically Based, Resource Intensive General Education Intervention

Requires immediate consideration of special education needs and IEP

Significant Educational Needs and Current Intervention May Be Effective, But Is Not Sustainable with Respect to General Education Resource Time and Intensity

Requires immediate consideration of special education needs and IEP

Districts adopt, adapt, drop, or add the decisions to fit 76 their Problem-Solving model.

Tools: Specify and Standardize Problem-Solving Tasks and Decisions

To do problem solving, you

have to identify what you “do”

There ’s a general

implicit universality of tasks and decisions

A district must make the

implicit explicit

Once explicit,

they can be taught

Once taught,

they can be used

Once used, they can be

evaluated for fidelity

use and

supported

77 of

Year 1 Plan: Groundwork

•

Understand the change process and don ’t worry about the wrong things

•

Ensure staff understand big ideas and over-arching focus

Access to key readings Motivation to read, discuss, use and get feedback 78

Year 1 Plan: Self-Study

•

Do a Self Study of:

Existing systems of support, including roles and functions Existing scientifically based academic and behavior interventions Existing scientifically based progress monitoring •

Identify tools, approaches, roles that can be abandoned

Year 1 Plan: Accomplishments

• •

Define and teach your standard problem-solving process Build:

Your leadership and staff development plan More effective teams with teachers in mind Your scientifically based data system for progress monitoring (Fix IEPs now! And ensure reading benchmark) 80

Doing It Well (Year 2)

• •

Use your data system to problem solve about problem solving Adjust your staff development plan based on your data, self-study and

• •

needs Expand the range of scientifically based interventions Do a Self Study to

ensure

old tools and processes are abandoned

Bigger Picture:

Why do we want to implement?

Previous Policy:

Process-Driven; Outcomes Lacking

Current Policy: Outcomes-Driven; All Students Future Experience:

Full Implementation

Problem Solving including RtI Problem Solving Model Data-Based Decision Making 3-Tier Model Scientifically-Based Data Systems Scientifically-Based Interventions

**Case Studies *feel free to use your own examples**

Case Example: Karen

• 6th grade student • Referred by her math teacher to the problem solving team.

Step 1: Problem Identification

Question: What is the discrepancy between what is expected and what is occurring?

A. List problem behavior(s) and prioritize.

B. Collect baseline data on primary area of concern (target student and peer). • Record Review • Interview • Observation • Testing C. State discrepancy between target student performance and peer performance.

A. List problem behaviors and prioritize:

• Karen’s grades are dropping.

• Karen is not turning in her homework on a consistent basis. When she does, it is often incomplete.

• Karen is talking to her peers during class.

• Karen

may

have ADHD.

State the primary area of concern:

– Karen is not turning in completed homework on a consistent basis.

Collect baseline data on primary area of concern:

Record Review (R) Karen ’ s teachers reviewed their record books and examined Karen ’ s weekly percentage of homework turned in compared to the class average over the last three weeks.

Homework Done

Science English Math Spanish Social Studies Karen 72% 78% 27% 60% 92% Peers 78% 84% 80% 70% 89%

Record Review (R):

Karen ’s math teacher reviewed her record books and examined the weekly number of completed homework assignments Karen turned in compared to the total number of completed assignments given over the last three weeks.

Date

1/13 1/20 1/27

Weekly Homework Assigned

5 5 5

Weekly Homework Turned In

2 1 1

Homework “Made-Up”

0 0 0 88

Why use Cumulative Rates of Homework Completion?

C. State the discrepancy between what is expected and what is occurring:

Data collected over a 3-week period indicates Karen has turned in 4 out of 15 assignments, which is 3.8x discrepant from what is expected.

Step 2: Problem Analysis

Question: Why is the problem occurring?

Review RIOT data, think about why the problem is occurring, and determine appropriate additional RIOT data you need to collect to: A. Differentiate between skill problem and performance problem (e.g., can ’t do vs. won’t do).

B. Determine situations in which the problem behavior is most likely and least likely to occur.

C. Examine hypotheses for why a problem is occurring.

D. Narrow down to the most

validated and alterable hypothesis

Possible Hypotheses She leaves class not having the skills to independently complete the work.

eview •

Old report cards

•

Look trends in homework assignments/ week

nterview •

Teacher

•

Karen

Karen is socializing during class. •

Teacher

Something is happening when she gets home that is interfering with her doing her homework.

•

Parent

•

Teacher

•

Karen

bserve

est •

Observe during math class

•

Review (R):

Teacher records and student ’s permanent products.

Weekly Date

1/13 1/20 1/27

Homework Assigned

5 5 5 Cumulative Number of Assignments: 15

Weekly Homework Turned In Homework “Made-Up”

2 1 1 0 0 0 Cumulative Number of Assignments Turned In: 4

Math In Class Work

75% accurate

Out of Class Work

52% accurate

Test Score Average

50%

Differentiate between skill problem and performance problem (e.g., can ’t do vs. won ’t do).

• Do the data suggest that Math homework is a problem for Karen because she does not have the requisite skills or because she is choosing not to do the homework? •

If you are not sure, treat problems as skill deficits first!

Hypotheses She leaves class not having the skills to independently complete the work.

Karen is socializing during class.

• HW completion rate low.

• In class more accurate than out of class work.

Something is happening when she gets home that is interfering with doing her homework.

eview

nterview • Karen enjoys working with peers.

• Is more likely to complete work that is started in class.

• Involved with after school sports.

• Mom reports that she is no longer able to help with math HW.

• Karen enjoys socializing with her friends on the computer.

bserv e • Obser vations indicate her on task behavior is compa rable to peers

est

Narrow down to the most validated alterable hypothesis.

Karen has turned in 4 out of 15 assignments (3.8x discrepant) in the last 3 weeks

because she leaves class not having the skills to independently complete the work.

Step 3a: Plan Development

Question: What is the goal?

A. Write the goal, a measurable statement of expected outcomes.

Question: What is the intervention plan to address the goal?

B. Define logistics (e.g., what strategies/procedures will be used, when and how often the intervention will occur, who will implement the intervention and where it will be implemented, and when it will begin).

Question: How will progress be monitored?

C. Define logistics (e.g., what materials are used, when and how often data will be collected, where data will be collected, and who is responsible).

D. Decide on decision-making rules for plan evaluation.

A. Write the goal, a measurable statement of expected outcomes.

In 8 weeks, Karen ’s total number of completed assignments will increase to 35.

Define the intervention plan logistics:

What?

When?

Where?

Who?

Set the Stage: Paired with hard working student.

Teach: Karen will be given time in class to complete assignment with assistance or will schedule a time to work with the teacher later in the day. Motivate: Time spent with peers. Praise from teacher and parent.

Class or other scheduled time by Karen. Intervention will occur daily.

Classroom or library Peer group, teacher, and/or teacher tutor

Start Date?

2/03

How?

What?

Where?

When?

Who?

Karen will keep homework chart signed by teacher.

Materials needed are the grade books and Karen ’ s completed homework chart.

In math classroom Collected daily, reviewed weekly Karen ’ s math teacher and Karen 100

Week_____

Monday Tuesday Wednesday Thursday Friday

Assignment Due Daily Assignment Completed

Yes No Yes No Yes No Yes No Yes No Weekly # Turned In: _____ Student Signature ___________ Date_____ Teacher Signature___________ Date _____

Number of Assignments

“

Made Up

”

Step 3b: Plan Implementation

Question: How will implementation integrity be ensured?

A. Provide support to those implementing interventions.

B. Observe intervention in action.

C. Make adjustments to intervention plan if needed.

D. Collect and graph data on intervention goal.

102

30 25 20 15 10 5 0 1/ 13 /2 00 4

D. Collect and graph data on intervention goal

Baseline 1/ 20 /2 00 4 Intervention 1/ 27 /2 00 4 2/ 3/ 20 04

Instructional Week

2/ 10 /2 00 4 2/ 17 /2 00 4 Cumulative Assignments Assigned Karen's Cumulative Number of Assignments Turned In

The Individual Student

Matt is in the 3

grade group of students receiving additional phonics instruction in place of silent reading, and his teacher (Ms. Armstrong) has indicated concerns that he is not making adequate progress toward his goal despite this intervention.

104

Problem Identification

Question: What is the discrepancy between what is expected and what is occurring?

Examine Matt ’s progress monitoring graph to determine his growth rate as compared to the expected growth rate for students participating in the intervention program 105

Matt ’s Problem Identification

Data Drives Decisions

Matt is making less than 1 word per week increase in WRC on 3 rd grade probes, while his expected growth is 1.3 words per week.

Problem Analysis

Question: Why is the problem occurring?

• Conduct a teacher interview to clarify the problem and define current program.

• Examine CBM Survey-Level Assessment data.

• Develop hypotheses

108

3rd Grade Instructional Planning Form Matt ’s Current Instructional Program Activity Materials

• • • Pre-teach story vocabulary Round Robin Choral Reading • Story Mapping • Grammar Workbook • • • 3rd grade basal • Blackboard • Practice Sentences • 3rd grade basal Visual organizer 3rd grade workbook • •

Arrangemen t

• Small Group (1:5 Ratio) • Small Group (1:5 Ratio) • •

Time

• 10 min/day 20 min/day • Small Group t • Independen • 15 min/day • 15 min/day • •

Motivational Strategy

• Praise for attention Reminding of rules Praise for answering • Reminding and peer • grading

Matt

’

s Survey-Level Assessment Grade 3 Reading Materials

The little meadow mouse was very hungry.

7 In the summer, when food was plentiful, she liked 16 to eat seeds, roots, and berries. Her favorite food 25 was sweet clover and tender plants of alfalfa. But 34 now the snow was deep and the food was hard to 45 find. Winter had come early this year. The mouse ’ s storehouse of seeds was already gone. 53 60 She stuck her nose into a crack in the tree. 70 She was looking for a bite of something - a weed 80 seed, or maybe a dry tuft of moss. But she found nothing. She often visited the old pine tree. She had long ago picked it clean of every seed, every nut, every dry blade of grass.

91 100 110 116

Matt

’

s Survey-Level Assessment Grade 2 Reading Materials

Books, books, and more books! That may 7 be how you think of a library. The word library comes from a very old word for “ book.

” 17 A library 27 is a collection of books. 32 A library may be little or big. You and your 42 family may have a little library at home. A home 52 library has books you like. It may have books you 62 use to find information, too. A school library is big. It was books to 67 76 help you with your schoolwork .

Hypotheses:  Matt is not making adequate rates of progress in 3rd grade materials BECAUSE his current instructional reading level is in 2 nd grade material (61 WRC/minute).

 Matt also displays difficulty with reading because he makes a high rate of errors that violate meaning in texts.

Question: What is the goal?

In 6 weeks, Matt will read 2nd grade material with a fluency rate of 70 WRC per minute.

Matt's Reading

80 70 60 50 40 30 20 ba seli 1/2 1 1/2 2/4 8 2/1 2/1 1 8 2/2 3/4 5 3/1 3/1 1 8 3/2 4/8 5 4/1 5 4/2 2 4/2 5/6 9 5/1 3 Series1 39 40 39 41 42 40 37 41 44 43 45 61 Series2

Date

Plan Development

Question: What is the intervention plan to address the goal?

Since problem analysis points to a problem in the curriculum instructional level, revisit the Instructional Planning Form to select curricular modifications.

114

•

3rd Grade Instructional Planning Form Matt ’s Current Instructional Program

Activity Materials

Pre-teach story vocabulary • Round Robin Choral Reading • Story Mapping • Grammar Workbook • Reading, Pencil Tap • 2nd grade basal • Blackboard • Practice Sentences • 2nd grade basal • Visual organizer • 3rd grade workbook • Book • • • •

Arrangemen t

• Small Group (1:5 Ratio)

Time

• 10 min/day Small Group (1:5 Ratio) Small Group Independent Individual • • 20 min/day • 15 min/day • 15 min/day 10 min/day • • •

Motivational Strategy

• Praise for attention Reminding of rules Praise for answering Reminding and peer grading • Praise for effort

Plan Development Question: How will progress be monitored?

• Continue collecting CBM reading data weekly with 2

grade probes • Collect CBM reading data with 3

grade probes at end of 6 weeks.

116

Plan Implementation

Question: How will implementation integrity be ensured?

A. Provide support to those implementing interventions.

Second grade teacher provided Ms. Armstrong with reading materials, and came in during one of her specials times to model the pencil tap method (that she had used before).

B. Observe intervention in action.

Fourth grade teacher came in to observe intervention so he would feel comfortable with doing it in the future 117

Plan Implementation (cont.)

C. Make adjustments to intervention plan if needed.

New second grade reading materials with more words & fewer pictures per page were substituted.

D. Collect and graph data on intervention goal.

Principal agreed to collect the 1 minute fluency sample & graph weekly.

118

Step 4: Plan Evaluation

Question: Is the intervention plan effective?

A.

Is the student making progress toward the goal?

B. Is the student decreasing the discrepancy between him/her and the general education peers?

C. Is the plan able to be maintained in the general education setting without more individualized, intense/specialized services?

119

Plan Evaluation Decision

Educational Progress

Discrepancy

Instructional Needs

Plan Evaluation Decision 120

A. Is the student making progress toward the goal?

Progress Conclusion Performance improved Goal not met Performance did not improve or got worse Begin to fade intervention Goal met or exceeded Increase goal Continue intervention Recycle through problem solving process Insufficient data Implement intervention and collect progress monitoring data

Is the student making progress toward the goal?

B. Is the student decreasing the discrepancy between him/her and the general education peers?

Less discrepant from acceptable level of performance Continue intervention until significant discrepancy is no longer present More discrepant from acceptable level of performance Recycle through Problem-Solving process Discrepancy Conclusion Same amount of discrepancy from acceptable level of performance Increase the intensity of the intervention Insufficient data Implement the intervention and collect progress monitoring data

C. Is the plan able to be maintained in the general education setting without individualized, intense/specialized services?

Instructional Conclusion Same materials, planning, and personnel as general education peers Similar materials, planning and personnel as general education peers.

Begin to fade intervention Continue intervention using general education resources

Significantly different materials, planning, and personnel than general education peers.

Recycle through Problem-Solving (consider special education entitlement)

References

The Vision:

Building a System of Substantial Instructional Interventions to Reduce the Gap Torgesen, J.K. ( 2001). The theory and practice of intervention: Comparing outcomes from prevention and remediation studies. In A.J. Fawcett and R.I. Nicolson (Eds.). Dyslexia: Theory and Good Practice. (pp. 185-201). London: David Fulton Publishers. Slide coursety of W. Alan Coulter http://www.monitoring

center.lsuhsc.edu

The Problem-Solving Model Has 4 Key Beliefs

Deno, S. L. (1989). Curriculum-based measurement and alternative special education services: A fundamental and direct relationship. In M. R. Shinn (Ed.),

Curriculum-based measurement: Assessing special children

(pp. 1-17). NY: Guilford.

Deno, S. L. (1995). School psychologist as problem solver. In A. Thomas & J. Grimes (Eds.),

Best practices in school psychology III

(pp. 471-484). Washington DC: National Association of School Psychologists.

Deno, S. L. (2002). Problem-solving as best practice. In A. Thomas & J. Grimes (Eds.),

Best practices in school psychology IV

(pp. 37-55). Bethesda, MD: National Association of School Psychologists.

Deno, S. L. (2005). Problem-solving assessment. In R. Brown-Chidsey (Ed.),

Assessment for Intervention: A problem-solving approach

(pp. 10-40). New York, NY: Guilford.

Problem-Solving Solutions are Not “New:” More than 27 Years of School-Based Implementation Problem-Solving Experience Translating Research into Practice Batsche, G. M., Elliott, J., Graden, J., Grimes, J., Kovaleski, J. F., Prasse, D.,Reschly, D, Shrag, J.. & Tilly, W.D. (2005).

Response to Intervention: Policy considerations and implementation

. Alexandria, VA: National Association of State Directors of Special Education, Inc.

Heartland Area Education Agency. (2004).

Heartland Area Education Agency Special Education Program Manual

. Johnston, IA: Author.

Reschly, D., Tilly III, W. D., & Grimes, J. (Eds.). (1999).

Special education in transition

. Longwood, CO: Sopris West.

These Changes Did NOT Come Out of the Blue Sample References for Changes

President ’ s Commission on Special Education Excellence (2002).

A NEW ERA: Revitalizing Special Education for Children and their Families.

Washington, DC: US Department of Education. http://www.ed.gov/inits/commissionsboards/whspecialeducation/reports/summ.htm

l Fordham Foundation Report-- Lyon, G. R., Fletcher, J.M., Shaywitz, S.E., Shaywitz, B.A., Torgesen, J.K., Wood, F.B., Schulte, A. & Olson, R. (2001). Rethinking Learning Disabilities. In C.E. Finn, A.J. Rotherham, and C.R. Hokanson (Eds).

Rethinking special education for a new century

(pp. 259-287). Washington DC: Thomas B. Fordham Foundation. http://www.edexcellence.net/doc/special_ed_final.pdf

National Research Council (2002).

Executive summary. Disproportionate representation of minority students in special education.

Washington, DC: Author. http://www.ecs.org/html/offsite.asp?document=http://www.nap.edu/catalog/10128 Learning Disabilities Summit-- Bradley, R., Danielson, L., & Hallahan, D. (Eds.) (2002). Identification of learning disabilities: Research to practice. Mahwah NJ: Erlbaum. www.air.org/ldsummit

The VISION: To Provide Effective Interventions to Meet the Needs of ALL Students Through Early and Scientifically Based Interventions Through Careful Systems Planning

Batsche, G. M., Elliott, J., Graden, J., Grimes, J., Kovaleski, J. F., Prasse, D., et al. (2005).

Response to intervention: Policy considerations and implementation

. Alexandria, VA: National Association of State Directors of Special Education, Inc .

References

Problem Solving is a Way of Thinking Ecologically

Deno, S. L. (1989). Curriculum-based measurement and alternative special education services: A fundamental and direct relationship. In M. R. Shinn (Ed.),

Curriculum-based measurement: Assessing special children

(pp. 1-17). NY: Guilford.

Deno, S. L. (1995). School psychologist as problem solver. In A. Thomas & J. Grimes (Eds.),

Best practices in school psychology III

(pp. 471-484). Washington DC: National Association of School Psychologists.

Deno, S. L. (2002). Problem-solving as best practice. In A. Thomas & J. Grimes (Eds.),

Best practices in school psychology IV

(pp. 37-55). Bethesda, MD: National Association of School Psychologists.

Deno, S. L. (2005). Problem-solving assessment. In R. Brown-Chidsey (Ed.),

Assessment for Intervention: A problem-solving approach

(pp. 10-40). New York, NY: Guilford.