Transcript Item Review Process and Criteria for State Educators

PARCC State Educator
Item Review Meeting
March 2013
Note: all items included in this presentation are for illustrative training purposes
only. They are not representative of PARCC assessment items.
1
Overview of Training
1. Purpose and Features of PARCC Summative
Assessments
2. PARCC Summative Assessments (PBA, EOY)
3. Your Charge as a Committee Member
4. Item Review Process by Core Leadership Group
5. Item Review Criteria by Core Leadership Group
6. Item Review Process and Criteria by State
Educators
2
Purpose of PARCC Summative
Assessments
•
•
•
•
•
3
Determine whether students are college- and careerready (CCR) or on track to become CCR
Assess the full range of the Common Core State
Standards (CCSS) for reading, writing, and language
Measure the full range of student performance,
including the performance of high- and low-performing
students*
Provide data for accountability, including measures of
growth
Incorporate innovative approaches throughout the
system
Features of the PARCC Summative
Assessment System
The PARCC summative assessment system is composed of
two assessments:
4
•
Performance Based Assessment
•
End of Year Assessment
Performance-Based Assessment
• Literary Analysis Task (LAT)
•
Literary Analysis Task plays an important role in honing students’ ability to read complex text closely, a skill
that research reveals as the most significant factor differentiating college-ready from non-college-ready
readers. This task will ask students to carefully consider literature worthy of close study and compose an
analytic essay.
• Research Simulation Task (RST)
•
Research Simulation Task asks students to exercise the career- and college- readiness skills of observation,
deduction, and proper use and evaluation of evidence across text types. In this task, students will analyze
an informational topic presented through several articles or multimedia stimuli, the first text being an
anchor text that introduces the topic. Students will engage with the texts by answering a series of
questions and synthesizing information from multiple sources in order to write two analytic essays.
• Narrative Task (NT)
•
5
Narrative Task broadens the way in which students may use this type of writing. Narrative writing can be
used to convey experiences or events, real or imaginary. In this task, students may be asked to write a
story, detail a scientific process, write a historical account of important figures, or to describe an account of
events, scenes or objects, for example.
Performance-Based Assessment
Eligible Item Types for Performance-Based
Assessment (PBA):
•Evidence-Based Selected Response
(EBSR)
•Technology-Enhanced Constructed
Response (TECR)
•Prose-Constructed Response (PCR)
Pg 13 Item Guidelines
6
Sample Evidence-Based Selected
Response (EBSR) Item
Part A: What does the word “regal” mean as it is used in the passage?
A.
B.
C.
D.
generous
threatening
kingly*
uninterested
Part B: Which of the phrases from the passage best helps the reader
understand the meaning of “regal?”
A.
B.
C.
D.
“wagging their tails as they awoke”
“the wolves, who were shy”
‘their sounds and movements expressed goodwill”
“with his head high and his chest out”*
Note: all items included in this presentation are for illustrative training purposes only. They are not representative of
PARCC assessment items.
7
Sample Technology-Enhanced
Constructed Response (TECR) Item
Part A : Below are three claims that one could make based on the article
“Earhart’s Final Resting Place Believed Found.”
Earhart and Noonan lived as castaways on
Nikumaroro Island.
Claims
Earhart and Noonan’s plane crashed into the
Pacific Ocean.
People don’t really know where Earhart and
Noonan died.
Highlight the claim that is supported by the most relevant and sufficient
evidence within “Earhart’s Final Resting Place Believed Found.”
Part B : Click on two facts within the article that best provide evidence to
support the claim selected in Part A.
8
Note: all items included in this presentation are for illustrative training purposes only. They are not representative of
PARCC assessment items.
Sample Prose-Constructed
Response (PCR) Item
Use what you have learned from reading “Daedulus and Icarus,” by Ovid
and “To a Friend Whose Work Has Come to Triumph,” by Anne Sexton to
write an essay that analyzes how Icarus’s experience of flying is portrayed
differently in the two texts.
As a starting point, you may want to consider what is emphasized,
absent, or different in the two texts, but feel free to develop your own
focus for analysis.
Develop your essay by providing textual evidence from both texts. Be
sure to follow the conventions of standard English.
Note: all items included in this presentation are for illustrative training purposes only. They are not representative of
PARCC assessment items.
9
End-of-Year Assessment
The End of Year Assessment will be:
• Focused on supporting Reading Comprehension
Claims
• Machine scored
Eligible Item Types for End of Year Assessment (EOY):
• Evidence-Based Selected Response (EBSR)
• Technology-Enhanced Constructed Response
(TECR)
10
Item Review Committee Charge
• Your role is to provide expert CONTENT review of
items and tasks.
• You will use information provided in this item review
training to review the items.
• You should focus exclusively on PARCC’s item review
process.
11
Item Review Committee
Please also note the following:
• Passage review committees have already approved
the passages according to PARCC content guidelines.
• Bias/sensitivity item review committees will apply
bias/sensitivity guidelines to all items.
• Concerns beyond the scope of your charge as an
item reviewer will be placed in the “parking lot” for
consideration by PARCC leadership.
12
Item Review Criteria by
Core Leadership Group
1. Does the item allow for the student to demonstrate the
intended evidence statement(s) and to demonstrate the
standard(s) to be measured?
2. Is the wording of the item clear, concise, and appropriate for
the intended grade level?
3. Does the item provide sufficient information and direction
for the student to respond completely?
4. Is the item free from internal clueing and miscues?
5. Do the graphics and stimuli included as part of the item
accurately and appropriately represent the applicable
content knowledge?
(continued)
13
Item Review Criteria by
Core Leadership Group
6. Are any graphics included as part of the item clear and
appropriate for the intended grade level?
7. If the item has a technology-based stimulus or requires a
technology-based response, is the technology design
effective and grade appropriate?
8. Is the scoring guide/rubric clear, correct, and aligned with the
expectations for performance that are expressed in the item
or task?
9. If the item is part of a PBA task, does it contribute to the
focus and coherence of the task model?
14
Previous Steps in the Item
Development Process
A large group of experts in the area of English Language Arts
have previously viewed these items, including:
• English Language Arts teachers
• ELA assessment experts
• English higher education faculty
• State Department of Education personnel
• School and district administrators
The items have been previously reviewed and edited for content
area accuracy, alignment to the standards/evidence
statements, and grade level appropriateness. This ensures
that the items you will be viewing have been vetted.
15
Item Review Process and Criteria
for State Educators
Step 1: Each reviewer on your team will read the passage/text
and the items independently.
Step 2: For each item, check to make sure the item meets the
evidence statements/standards . If the item can be edited to
align with the evidence statements/standards, note in comments
how to align the item. If not, reject the item and move on.
Step 3: Determine if there are any fatal flaws in the item (i.e.
anything that makes the item unusable as written). Document
fatal flaw.
Step 4: Advise to accept, accept with edits, or reject the item.
16
Fatal Flaws
• Use of language in the item in a way that could have
unintended consequences
• Language that is convoluted and obscure
• Note that the flaw should be such that when one
raises the issue, the whole review group
immediately says, “Oh yes—this can’t go forward as
is.”
• If some see “the flaw” and others don’t see it as a
flaw, then it shouldn’t be seen as a fatal flaw.
17
Questions?
18