Transcript Reading and Writing Collection of Evidence

Reading and Writing
Collection of Evidence
Lesley Klenk, ELA COE Specialist
September 25, 2014
Agenda

Results of 2014 Reading and Writing COE
submissions

Lessons learned

Inclusion bank updates

Instructional support suggestions

Security Protocols

Beginning the Transition to the ELA COE
7/18/2015
2
2014 Reading and Writing Submissions

Reading
 Winter:
1,382 submitted81.8% met standard
 Summer:

578 submitted 82.2% met standard
Writing
 Winter:
798 submitted 79.9% met standard
 Summer:
323 submitted 69.7% met standard
7/18/2015
3
Lessons Learned: Reading Scoring



Well done:

Inference—abstract and linked to the text

Vocabulary—defined and explained within the text’s context

Evaluation—clear student judgment regarding “issues” in the text
Needs work:

Summaries frequently scored a “2”; retelling rather than synthesizing

Analysis targets sometimes were lists without connections—(lacking “because”)
Copied text has no evidence of a student answer

Paragraphs or sentences from different parts of the text

Statements such as “I think” OR “the author says” OR “It does” imbedded in the text

Restates the question
7/18/2015
4
Rushing at the end: Making entry mistakes


Some students are drafting on paper and entering their answers to the
questions in a way that is not connected:

The answer is uploaded in the wrong task

The answers are uploaded in the wrong boxes within the task

A single answer is uploaded in all of the boxes within the task

The answer is from an older task and no longer matches the question
A few teachers entered their students’ answers into the system. This is a test
security violation and could result in invalidation of the student’s collection.
7/18/2015
5
Lessons Learned: Writing Scoring

Well done:
 Evidence to support their expository work samples shows
growth
 Student voice is more authentic

Needs work:
 Statements of claims and counter claims are not strong
 Decisions to simplify conventions of written work impacts
COS elements
7/18/2015
6
Rushing at the end: Making entry
mistakes
Errors:
Papers are missing sections or conclusions.
Papers break mid-sentence into another paragraph which indicated
formatting problems.
Papers are put into the wrong tasks so the prompt and the paper do not
match.
Papers will have sentences where capital letters and lower-case letters
are combined; this causes papers to score low in conventions
Papers include a majority of language other than English which causes a
paper to score low in conventions, organization, and style
Papers are plagiarized and it is not recognized by the school. If identified
at the scoring center, the entire collection is invalidated.
7/18/2015
7
Inclusion Bank: Reading




Tasks removed due to significant plagiarism:

“Silk Stockings”

“After Twenty Years”

(If your students have already begun work on these tasks, they can complete them and
submit them. It is a good idea to review your students’ work. )
Choice of passages may affect students’ interest and performance

High interest: A Friendship Like No Other

Low Interest: Boyhood Days
Fact-laden passages may impede students’ analysis

Mountain Beaver

Barbaro
There will be no more task development for the current Reading COE due to the
transition to the ELA COE in 2016
7/18/2015
8
Inclusion Bank: Writing




Successful prompts due to high interest, student knowledge and experience:

Being a Teen

Learning Outside School

Outstanding Leader

Personal Goal
Challenging prompts due to low interest and limited knowledge and experience:

Why Should We Study History

World Language Learning
Prompts most likely to be plagiarized: (the two above and the three below )

Electronic Devices

Is PE Necessary

Changing Places
There will be no more task development for the current Writing COE due to the
transition to the ELA COE in 2016
7/18/2015
9
Reading Instructional support:
Informational
7/18/2015
10
Reading Instructional Support: Literary
7/18/2015
11
Instructional Support: Writing
7/18/2015
12
Instructional support writing
7/18/2015
13
7/18/2015
14
7/18/2015
15
Security Protocols for Reading and
Writing
Assessment materials:

COE inclusion bank passages and tasks are secure state test materials.
These are never left unattended by building educators. As the COE
allows for multiple opportunities for students to review and revise
these materials, they must be kept in a secure location between
administration sessions. Passages and tasks may be printed for one
student at a time for marking passages, developing responses, and
reviewing their work. Passages and tasks cannot be duplicated for a
class, or held in a file for future COE work. All passages and tasks
must be destroyed in a secure environment after students have
submitted their reading COEs.
7/18/2015
16
Security Protocols for Reading and
Writing

Room preparation: Remove or cover any aids or prompts that might
potentially assist students in answering questions on an inclusion bank
task.

Materials preparation: Notes regarding instructional strategies are
allowed, but specific material surrounding the passage or the tasks
may not be used for support material.

Supervision: A student is always supervised by building educators for
on-demand and extended-time assessments. Students cannot be alone
in a computer lab, have access to the internet during the work on the
assessments, or save their work on a jump drive or shared network.
7/18/2015
17
Security Protocols for Reading and
Writing

Teacher assistance—the description below is intended for
before and after student work. Teachers cannot “assist”
while students are working directly with prompts and tasks
For the extended-time student responses, it is expected that
there is an appropriate amount of teacher assistance. The
assistance may include providing standards-based instruction
that includes the specific reading skills expected in the COE.
Appropriate teacher assistance may also include teaching
instructional strategies such as note taking, highlighting,
underlining, reading journals, using graphic organizers, and
reviewing how to understand the requirements of questions
7/18/2015
18
Testing Irregularities (Plagiarism)

A testing irregularity is evidence of possible inauthentic student work
discovered in a student’s COE during scoring. COE testing irregularities are
defined as a Level 1 or Level 2 Alerts.

A Level 1 alert is labeled as “likely inauthentic student work” due to the fact
that one or more student response(s) is a copy of another student’s work or a
retrievable copy of material from the internet.

A Level 2 alert is labeled as “suspected inauthentic student work” because
one or more work samples in question are quite different and unusually strong
as compared to the other work samples in the collection or contain elements
of copied material from the internet that has been changed slightly. Districts
are asked to conduct an investigation regarding the authenticity of the
student using OSPI guidelines.
7/18/2015
19
The “Current” and “New” ELA COE

Moving from separate collections to a single ELA collection

Targets for Claims 1, 2, and 4 will be “woven” together in the
collection

Item types will be: short response, brief writes and full writes

Short response will use task-specific rubrics

Brief writes and full writes will be scored using the Smarter Balanced
rubric for performance tasks

Lexile and Flesch-Kincaid measures and Text Complexity Qualitative
Criteria play a more significant role in selecting stimuli
7/18/2015
20
What Will Stay the Same Between the
Current COE and New ELA COE?

Extended time to work on tasks

An inclusion bank to hold all secure assessment materials

Individual student work is submitted

Collections will be scored by professional scorers

Passages and questions will look and “feel” familiar

Standards will be set at the same level on the ELA COE as
the ELA High School Exit Exam
7/18/2015
21
When is the new ELA COE coming?

Summer and Fall 2014 performance task development

Fall 2014 Bias and Sensitivity Review

Fall 2014 Content Review

Winter and Spring 2015 Pilot of performance tasks

Spring and Summer 2015 pilot rangefinding

Fall 2015 ELA COE opens for students submitting in June
2016 or beyond

June 2016 first scoring of ELA COE
7/18/2015
22
COE website: www.coe.k12.wa.us
Reading Guidelines: http://www.coe.k12.wa.us/domain/32
Writing Guidelines: http://www.coe.k12.wa.us/domain/33
7/18/2015
23
COE Teacher Training

http://www.coe.k12.wa.us/domain/14
COE Teacher Training

Checklist for Training

COE Guidelines

Test Security Acknowledgement
7/18/2015
24
Contacts

Lesley Klenk, ELA COE Specialist
[email protected]

Amanda Mount, COE Operations Specialist
[email protected]

Kim Anderson, Math COE Specialist
[email protected]

Scott Killough, Biology COE Specialist
[email protected]

COE Technical Support, [email protected]
7/18/2015
25