Transcript Developing work samples for the Collection of Evidence

CAA Options: Reading Collection
of Evidence
Lesley Klenk, CAA Options Administrator
Fall Workshop
October 2007
What is a Reading COE?


Reading is the process of making meaning
of text. Students must be able to
demonstrate their comprehension, their
ability to analyze, and their critical thinking
related to text in written responses.
The Reading COE is a set of classroom work
samples created by a student that show
what the student knows and can do in
reading. They must demonstrate the same
skills measured on the Reading WASL.
What is a Reading Collection of
Evidence?





It demonstrates a breadth and depth of the
WASL skills and knowledge in reading.
A collection must include 8 – 12 work samples that
address all six strands. Each work sample must address at
least two strands. Two work samples must be short
papers, one literary and one informational.
Work samples may be on-demand or extended
time.
On demand work is done in one sitting with teacher supervision.
Extended time work samples are student generated with limited
teacher assistance.
What Does Breadth and Depth
Look Like in a Reading COE?



The Reading COE demonstrates a
breadth and depth of the WASL skills
and knowledge in reading.
A collection is composed of written
responses to a wide variety of texts. Those
texts should be of high school rigor and
may come from various content areas i.e.
biology, U.S. History, business law,
language arts.
The work sample will clearly state the
strand being addressed (ie. literary
comprehension) and include text based
evidence that explains and supports the
demonstration of that skill.
Scoring the Reading COE





239 Registered; 18 submitted; 15 met standard
71 points out of 96 points was judged to be
proficient
65 points-70 points is the augmentation band
90 was the highest score and 54 was the lowest
score
Too few collections to “set standard”. Instead, a
Proficiency Committee studied Performance Level
Descriptors for WASL and judged which
collections were proficient.
Reading COE Sufficiency Issues



Work Samples that included the text the
student read were insufficient and were
not scored.
Work Samples that had grades on them
were insufficient and were not scored.
Work Samples that did not state the
text’s name in the assignment or the
student response were insufficient and
were not scored.
Reading COE Scoring Rules





Only sufficient collections are scored.
Each strand in each work sample is
scored.
Each collection (all work samples) are
scored twice.
The top two scores for each strand counts
towards the total score.
96 points possible and 71 points judged
sufficient.
How do the Reading COE scores add
up?
Collection # 12345
LC
LA
LT
IC
IA
IT
Total
Points
Points Possible
8
8
8
8
8
8
48
Points Earned
8
6
5
7
6
5
37
LC
LA
LT
IC
IA
IT
Total
Points
Points Possible
8
8
8
8
8
8
48
Points Earned
7
7
6
8
5
6
39
15
13
11
15
11
11
76
Scorer 1
Scorer 2
Total :
Scorer 1 and Scorer 2
Strong Reading COEs…





had excellent assignments that explicitly
asked students to demonstrate a specific
reading skill.
featured texts that were interesting and
timely to students.
demonstrated that students know how to
cite text evidence to show understanding.
gave students “meaty” assignments that
required more than just a few sentences
to answer.
showed confident readers who weren’t
perfect but showed definite strengths
Weak Reading COEs…





had hard to understand assignments with factoriented questions as opposed to skill-oriented
ones.
featured texts that were technical without
context; historical without background; or lean
and lacking details.
used little to no textual evidence for support of
claims about text.
had a preponderance of research papers where it
was almost impossible to find the text much less
the comprehension, analysis, or evaluation.
showed readers uncomfortable with manipulating
text.
Lessons Learned





If students don’t know the skill going into a work sample,
they don’t magically learn it in the process.
Students with weak COEs don’t know how to use textual
evidence for support. In fact, they don’t include much
support at all.
“Personal responses” did not lend themselves to target
questions.
Several work samples attached to one text “lost steam.”
Kids got tired of dredging up the same evidence.
Summer module assessments were moderately
successful. However, they were rarely 4-point answers
because the number of details limited their responses.
Scoring Notes



While scoring student work, it was easier to identify
3 or 4 from the rubric than 1 or 2 from the rubric.
Often, the Work Sample Documentation form was
incorrectly filled out; the student lost points because
the scorer had to score what was indicated.
Students did better when they turned in the
maximum number of work samples; only the top
two scores for a strand were reported.

Using OSPI generated tasks were easier to score—
just because the tasks used the language of the
targets.
Recommendations for February
‘08 and June ‘08 Scoring Windows
Schools and districts that were most successful
dedicated an elective class to working on COEs.
Schools and districts that met regularly as a
staff about the COE had more students meet
standard.
Schools and districts that created a calendar, a
list of deliverables, and a set of goals had more
students who met standard.
Teachers who used OSPI tasks as samples
created excellent tasks of their own.
Educators who believed that students could
meet standard saw it happen.