Generalization of RtI Procedures to Written Language

Merilee McCurdy, Ph.D.

University of Nebraska - Lincoln

Description From Program

As schools begin to implement RtI procedures, it is important to realize that RtI procedures can be used with all academic areas, including writing and mathematics. This presentation will review writing assessment issues such as norming writing skills, universal screening procedures, and progress monitoring techniques. In addition, writing interventions that can be used within RtI frameworks will be identified for each writing skills area.

Research in Reading

To date, much of the literature and research in RtI has been done in the area of reading.

Why?

 Research has shown that those children who become adults with low levels of literacy are at a disadvantage in a society that has high demands for effective literacy skills in the work place.

Torgeson, J.K. (2000)  There has not been a significant change in the reading performance of students from 1992 to 2005 (National Center for Educational Statistics, 2003).

 A 1997 study documented that at least 20% of the population in the United States had reading difficulties (Lyon & Moats, 1997).

RtI and Reading

 Research has focused efforts on understanding reading deficits and identifying effective reading programs or interventions.  This concentration of RtI procedures in the area of reading has been appropriate as the field seeks to understand and develop initial RtI procedures.  Unfortunately, this intense focus has not been similar in other academic areas, especially for written language.

The Significance of Writing

 Writing has become the neglected element of school reform (National Commission on Writing, 2004)  The Commission asserts that students must “struggle with details, wrestle with facts, and reword raw information and dimly understood concepts into language they can communicate to someone else.”  In short, students must write if they are to learn.

The National Commission on Writing. (2003). The neglected “R”: The need for a writing revolution.

The Significance of Writing

 The National Commission on Writing for America’s Families, Schools, and Colleges (2004) described writing as a threshold skill for employment and promotion and indicated that people who cannot write well are less likely to be hired, retained, and/or promoted. National Commission on Writing for America’s Families, Schools, and Colleges. (2004). Writing: A ticket to work…or a ticket out. NY: The College Board  Writing skills are often needed for demonstrating learning (e.g., responding to exam items) and communicating across a variety of employment settings.

The Significance of Writing:

Percentage of students, by writing achievement level, grades 4, 8, and 12: 1998 and 2002

Grade 4

1998 2002

Grade 8 Grade 12

1998 2002 1998 2002 Below

Basic

At

Basic

At

Proficient

At

Advanced

At or above

Basic

At or above

Proficient

16 14 16 15 22 26

61 58 58 54 57 51

22 26 25 29 21 22 1 2 1 2 1 2 84 86 84 85 78 74 23 28 27 31 22 24

Nebraska Statewide Writing Assessment All Students 2005 - 2006 Grade Levels 4 8 11

Nebraska Statewide Writing Assessment Students in Special Education 2005 - 2006 Grade Levels 4 8 11

RtI and Writing

Schools appear to be having difficulty preparing students to meet society’s demands in the area of writing.

The RtI process can assist with:  Identifying students who are at-risk in the area of writing    Providing or developing intensive interventions in the area of writing Monitoring weekly progress of student progress Using data to make decisions regarding students’ needs following a predetermined length of time

RtI and Writing

 In many ways, RtI procedures developed for the area of reading can be easily generalized to writing  However, due to the lack of research on written language, many questions remain.

Questions

Questions regarding RtI and writing assessment: 1) 2) How much writing time is needed for a valid evaluation of writing? Is writing performance stable across assessments? 3) 4) 5) 6) 7) How can story organization and story development be evaluated using CBA techniques?

Which variables should be used for initial decision making?

Which variables should be used for ongoing decision making?

Are some assessment variables useful independently or in a combination with others? What type of writing prompts should be used?

Questions

Questions regarding RtI and writing interventions: 1) 2) How are Tier 1 services evaluated? What are empirically based interventions for writing? 3) 4) 5) 6) 7) Can interventions be delivered in small groups? How much intervention time is required to observe changes? Which variables are more reliable and valid for progress monitoring? What gains should be expected?

How can students be motivated to write?

RtI Writing Team: Lincoln Public Schools



What we have achieved so far….

 Established an RtI Writing Team (2004)  Collected normative data  Grades K-2 (2005-2006)  Grades 3-5 (2006-2007)  Initiated small pilot studies

RtI Writing Team: Lincoln Public Schools

 Future Plans….

 2007-08     Pilot K-5 writing procedures Small pilots at high school and middle schools to determine normative data collection procedures Evaluate research based interventions Train others to score writing probes (reliably)  2008-2009   Implement RTI writing district-wide at elementary schools Begin norming at middle schools  others will score if meet criteria for training  2009-2010  ?????

Core Components for Quality RtI Implementation

           School-wide buy-in and implementation plan Team Leadership Integration of Services Implementation Infrastructure Parent Involvement Universal Screening and Assessment ** Individual Progress Monitoring ** Planned Service Delivery Decision Rules Scientifically Supported Instruction ** Intervention Delivery SLD Verification

Assessment

 Curriculum-Based Measurement (CBM)   Content of the assessments are based on the instructional curriculum.

Measures are presented in a standardized format.

  Material for assessment is controlled for difficulty by grade levels.

Measures are generally brief.

Shapiro (2004)    Development of Norms Universal Screening Progress Monitoring

General Assessment Procedures for Writing

 The student is given a “story starter” in the form of a starting sentence or partial sentence  Provides the student with an idea to write about  The student is asked to think about their story for one minute and to write for three minutes  At the end of the writing period, the examiner collects the student’s writing

Differences between Reading and Writing CBM

Reading Writing  Takes one minute  Administered individually  Must be scored concurrently with performance  Takes four minutes  Can be administered in groups  Permanent products allow for later scoring

Other data to consider

 Classroom writing products  Classroom writing assessments/rubrics  Statewide writing assessments  Reading performance  Motivation!

Development of Norms



LPS Normative Procedures

     N=250 per grade; randomly selected by district One story starter administered at fall, winter, spring (different starter each period) Same starter used with each grade (1 st /2 nd ) Kindergarten  Letter writing (fall and winter)  Story starter in spring Used a “scoring team” to increase reliability

K – Winter K – Spring 1 – Fall 1 – Winter 1 – Spring 2 – Fall 2 – Winter 2 – Spring

NORMATIVE VARIABLES BY GRADE

TLW TWW CWS CMIWS %CWS X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X TLW CWS Total Letters Written Correct Writing Sequences %CWS TWW CMIWS Total Words Written Correct minus Incorrect Writing Sequences Percent Correct Writing Sequences

Universal Screening

The practice of assessing all students to identify those who are not making academic or behavioral progress at expected rates.

National Association of State Directors of Special Education, Inc. (2005) A classroom-wide, school-wide, or district-wide assessment used to identify students who are at risk for academic failure or behavioral difficulties and could potentially benefit from specific instruction or intervention.

A critical prerequisite to providing early school-based prevention and intervention services for students at risk for or with academic, behavioral, or emotional difficulties.

Glover & Albers (2007)

Universal Screening

“Assessments are in place that provide teachers with information about the progress of all students to determine which students need closer monitoring, additional intervention, and/or adjusted learning opportunities”  Norms are necessary to (1) identify students who need additional supports, (2) develop growth expectations, and (3) set entrance and exit criteria.  Develop norming procedures based on school resources

Universal Screening

 Scoring: It doesn’t take that long!

  Requires between 22 and 31 seconds to score TWW Requires between 25 and 37 seconds to score WSC  Requires between 57 and 151 seconds to score CWS  If all three scores are used, it takes between 1 ½ minutes and 2 ½ minutes to score one writing sample. Gansle, Noell, VanDerHeyden, Naquin, & Slider, 2002 Malecki & Jewell, 2003

Difficulties regarding RtI Writing and Universal Screening

  Several components of writing to consider:  Spelling   Grammatical usage Mechanics of writing Numerous variables to consider!!

 It has not yet been determined which variables:  are most sensitive to change over time  are most valid and reliable for a given grade level

Writing Variables

 Production-dependent    Correctly Written Words Total Words Written Correct Writing Sequences    Mature Words Letter Sequences Correct Total Words Spelled Correctly  Production independent    % Correct Writing Sequences % Correctly Spelled Words Mean Length of Correct Writing Sequences A measure of both fluency and accuracy = Correct Minus Incorrect Writing Sequences (CMIWS), an accurate-production measure



How to set growth expectations?

Low and variable levels of growth on many variables.  From AIMSweb norms:

TWW

2nd grade 3rd grade Fall to Winter +5 +6 Winter to Spring +7 +2

CWS

2nd grade 3rd grade +7 +7 Data represent growth across entire normative sample +5 +5

Progress Monitoring

The practice of assessing students to determine if academic or behavioral interventions are producing desired effects.

National Association of State Directors of Special Education, Inc. (2005) Provides critical information about student progress that is used to ensure the use of effective educational practices and to verify that students are progressing at an adequate rate.

“Assessments are in place that provide teachers with information about which students are benefiting from intervention and which need additional supports.”

Progress Monitoring Procedures

1. Based upon the norms you have decided to use and each student’s screening results, set a goal for each student.

 This goal should reflect an average gain per week as determined by the normative data.

2. Once the student’s intervention has begun, monitor the student’s progress at least once per week. For writing, may want to include more than 1 writing sample. 3. Graph the student’s scores on a chart.

Progress Monitoring Procedures

4. Review the chart weekly to determine whether progress is being made.

5. After the student has been in an intervention for a specified amount of time:    Look at the level and the rate of progress Determine whether the goal was attained Plan for next steps  These are general RtI procedures and should be the same for regardless of academic area!!

25 20 15 10 5 0 Bas eline 1 2 3

The Complexity of Writing: Sample Student Data

TWW CWS

Baseline Intervention 10 8 6 4 2 0 Intervention 4 5

Session

6 7 8 1 2 3 4 5

Session

6 7

WSC

12 10 8 6 4 2 0 Baseline 1 2 3 4 5

Session

Intervention 6 7 8 8

CMIWS

3.5

3 2.5

2 1.5

1 0.5

0 Baseline 1 2 3 4 5

Session

Intervention 6 7 8

Further Complexity of Writing

-2 -4 -6 -8 8 6 4 2 0 1 2 3

CMIWS

4 5

Session

6 7 8

Advantages of Writing Variables

 Although numerous variables add to the complexity of assessment, they aid in identifying specific difficulties.

 Awareness of specific difficulties further assists in developing/identifying interventions for the next phase of intervention (if necessary)

Scientifically Supported Instruction “Core instruction and interventions have been validated through scientific research” Must demonstrate that districts have a process for identifying interventions with scientific evidence of effectiveness.

Do Not Forget Tier 1

Tier 1 must be evaluated to examine effectiveness of curriculum for most students. This can be accomplished through universal screening. “When teachers use a scientifically supported curriculum and appropriate instructional practices, they are preventing many problems from occurring.” Nebraska Department of Education Technical Assistance Document (2006)

Interventions in RtI

 Basic Message: Intervention delivery for writing is very similar to reading procedures. However… MORE RESEARCH IS NEEDED ON INTERVENTIONS THAT CAN BE USED IN THE SCHOOLS!!

What makes a good Writer?

 Writing extends beyond mastering grammar, punctuation and the ability to identify parts of speech.

 Writing is best understood as a complex intellectual activity.

 It requires students to “stretch their minds, sharpen their analytical capabilities, and make accurate and valid distinctions.” (National Commission on Writing)  Writing is a way for students to demonstrate what they know AND a way to help them understand what they know .

Areas for Intervention

 Three primary writing stages:  Prewriting    Planning Story Development Idea Development  Writing     Story Organization Grammar Spelling Fluency  Post Writing  Editing/Revising

Challenges to improving students’ writing

 Time for writing in and out of the classroom  Assessment or measurement of progress and results  Integrating technology into the teaching and learning of writing  Support for teaching and other classroom issues The National Commission on Writing. (2003). The Neglected “R”: The need for a writing revolution.

        

Characteristics of Strong Writing Instruction

Writing Strategies - involves teaching students strategies for planning, revising, and editing their compositions Summarization - involves explicitly and systematically teaching students how to summarize texts. Collaborative Writing - uses instructional arrangements in which adolescents work together to plan, draft, revise, and edit their compositions. Specific Product Goals - assigns students specific, reachable goals for the writing they are to complete. Word Processing - uses computers and word processors as instructional supports for writing assignments. Sentence Combining - involves teaching students to construct more complex, sophisticated sentences. Prewriting - engages students in activities designed to help them generate or organize ideas for their composition. Inquiry Activities - engages students in analyzing immediate, concrete data to help them develop ideas and content for a particular writing task. Process Writing Approach - interweaves a number of writing instructional activities in a workshop environment that stresses extended writing opportunities, writing for authentic audiences, personalized instruction, and cycles of writing.



Tier 2 and 3 Instruction: Identifying Target Skills

Generate hypotheses based on accuracy or fluency concern:  Questions to ask when identifying target skill:   Does the student have letter identification and letter formation skills?

Can the student produce legible text?

     Did the student use the planning time?

Does the student produce an adequate amount of text?

Does the student produce a grammatically correct story? Does the student understand story organization?

Could motivation be impacting performance?

Classwide Fluency Strategies



Classwide and individual performance feedback strategies are very successful at increasing student writing production.



Students must practice writing to improve writing skills!

Building Effective Interventions

 Writing Program composed of:   Direct Instruction Goal Setting  Writing Practice   Feedback   Individual Feedback Charting Rewards McCurdy, M., Skinner, C. H., Watson, T. S., & Shriver, M. D. (accepted). Examining the effects of group contingencies on the writing performance of middle school students with learning disabilities in writing. School Psychology Quarterly.

Evaluating Published Programs



Examine for:

 Technical Adequacy  Research Support  Grade appropriateness/match to need  Relationship to 6 Traits of Writing  Price

Grade Name

K 1 2 3 4 5

Teaching Competence in Written Language Second Edition Description (from website) From the basic sentence to the formal essay,

Teaching Competence in Written Language

is the program to choose to teach writing at any level. This program works because it teaches both basic and advanced writing skills in an easy to learn, step-by-step format. Each lesson also includes social (pragmatic) uses of writing and proofreading and error recognition.

This program is different from other programs in the field because it can be used either:  As a step-by-step, structured hierarchy of writing skills, from the most basic to advanced, or  As a resource to plug into any level where an individual or group needs focused work. From the original 44 lessons, the program has been modified and expanded to a total of 80 lessons for individuals or groups at different levels of ability.

Writing Trait Ideas Organization Voice Word Choice Convention Other Publisher: Pro-ed Inc. www.proedinc

.com

Price: $62.00

Intervention Resources

 Strategic Instruction Model (SIM)  Center for Research on Learning  http://www.ku-crl.org/sim/strategies.shtml

 National Center on Accelerating Student Learning  http://kc.vanderbilt.edu/casl/index.html

 Mnemonics for Planning and Editing   Cognitive Strategy Instruction; Dr. Bob Reid – UNL http://www.unl.edu/csi

Intervention Delivery: Resources!

 Interventions can be delivered in small groups  Behavior management procedures will be necessary  Motivation and engagement is a concern Consider and evaluate “comprehensive” reading programs  These programs may also impact writing performance   May serve as the first supplementary intervention Must monitor for improvement or lack of improvement

LPS Goals of Pilot Year (2007-2008)

      Identify any modifications to district RtI procedures Identify appropriate assessment variables for progress monitoring and decision making Incorporate district assessment data into decision making structure Establish structured decision making criteria for measuring adequate progress or referral to special education Locally validate multiple interventions for use with writing concerns Prepare for the future    Norming at upper levels (3, 5, or 10 minute writing times?) Procedures at upper levels Interventions for writing skills in the upper grades

References

Daly, E., Glover, T., McCurdy, M. (2006). Response to intervention: Technical assistance document. Lincoln, Nebraska: Nebraska Department of Education.

Gansle, K. A., Noell, G. H., VanDerHeyden, A. M., Naquin, G. M., & Slider, N. J. (2002). Moving beyond total words written: The reliability, criterion validity, and time cost of alternative measures for curriculum-based measurement in writing. School Psychology Review , 31 , 477- 497. Glover, T. A., & Albers, C. A. (2007). Considerations for evaluating universal screening assessments.

Journal of School Psychology, 45 , 117-135. Graham, S., & Perin, D. (2007). writing of adolescents in middle and high schools - A report to Carnegie Corporation of New York. Writing next: Effective strategies to improve Washington, DC: Alliance for Excellent Education.

Lyon, G.R. & Moats, L.C. (1997). Critical conceptual and methodological considerations in reading intervention research. Journal of Learning Disabilities, 6 , 578-588.

References

Malecki, C. K., & Jewell, J. (2003). Developmental, gender, and practical considerations in scoring curriculum-based measurement writing probes. Psychology in the Schools, 40 , 379-390. McCurdy, M., Skinner, C. H., Watson, T. S., & Shriver, M. D. (accepted). Examining the effects of group contingencies on the writing performance of middle school students with learning disabilities in writing. School Psychology Quarterly.

National Association of State Directors of Special Education, Inc. (2005). Response to intervention: Policy considerations and implementation. NASDE. National Center for Educational Statistics. (2003). NAEP 2002 writing report card for the nation and the states (Report NCES 1999-462). Washington, DC: Office of Educational Research and Improvement. National Commission on Writing for America’s Families, Schools, and Colleges. (2004). Writing: A ticket to work…or a ticket out. Examination Board NY: The College Entrance

References

National Commission on Writing in America’s Schools and Colleges. (2003). The neglected “R:” The need for a writing revolution.

The College Entrance Examination Board. NY: Shapiro, E. S. (2004). intervention Academic skills problems: Direct assessment and (3rd ed.). NY: The Guilford Press.

Torgesen, J.K. (2000). Individual differences in response to early interventions in reading: The lingering problem of treatment resisters. Learning Disabilities Research and Practice, 15, 55-64.

Contact Information

Merilee McCurdy, Ph.D.

Assistant Professor of School Psychology Licensed Psychologist University of Nebraska - Lincoln 234 Teachers College Hall Lincoln, NE 68588-0345 [email protected]