Transcript Online Training Nov 13 ELA

Welcome to the
ELA Smarter Balanced Item and
Task Types
November 13, 2014
West Virginia Department of Education
Office of Assessment and Research
While you wait for the webinar to begin, please be
sure to turn on your speakers.
You can try this link if your speakers aren’t working.
https://support.office.com/en-us/article/Set-up-andtest-Lync-audio-f4336d14-50ef-45a8-91df1074121a0033
ELA Smarter Balanced Item and Task
Types
Welcome
Stacey Murrell, Ed. D.
Interim Assessments Coordinator
Office of Assessment and Research
Balanced Assessment System
Smarter Balanced ELA Claims
ELA: Targets by Grade Level for each Claim
ELA Claims and Targets
•
•
•
•
Claim 1 has Targets 1-14.
Claim 2 has Targets 1-9.
Claim 3 has Targets 1-4.
Claim 4 has Targets 1-4.
Targets were
created by Smarter
Balanced. When
you read them, you
can see that content
standards are
embedded in them.
So a test question
can assess multiple
content standards
within one target.
Claim 1 and Its Targets
• Students can read closely and analytically to
comprehend a range of increasingly complex
literary and informational texts.
1. Targets 1–7 correspond with literary texts
2. Targets 8–14 correspond with informational texts
3. The assessment targets incorporate the content
clusters from the Common Core State Standards
•
Grade
5
Claim
1
Target
4
4. REASONING & EVIDENCE: : Make an inference or provide a conclusion and use
supporting evidence to justify/explain inferences (character development/actions/traits;
first- or third-person point of view; theme; author’s message).
• Standards: RL-3, RL-3, RL-6
• (DOK 3, DOK 4)
RL-3 Compare and contrast two or more characters, settings, or events in a story or drama,
drawing on specific details in the text (e.g., how characters interact).
RL-6 Describe how a narrator’s or speaker’s point of view influences how events are
described.
RL-9 Compare and contrast stories in the same genre (e.g., mysteries and adventure
stories) on their approaches to similar themes and topics.
Claim 2 and Its Targets
• Students can produce effective and well
grounded writing for a range of purpose and
audiences.
–
–
–
–
–
–
Targets 1a, 3a, & 6a: Write Brief Texts
Targets 1b, 3b, & 6b: Revise Brief Texts
Targets 2, 4, & 7: Compose Full Texts
Target 5: Use Text Features
Target 8: Language & Vocabulary Use
Target 9: Edit
Grade 7 Claim 2 Target 1
1a. WRITE BRIEF TEXTS: Apply narrative techniques (e.g., dialogue, description) and appropriate text structures
and transitional strategies for coherence when writing one or more paragraphs of narrative text (e.g., closure,
introduce narrator or use dialogue when describing an event).
1b. REVISE BRIEF TEXTS: Apply narrative techniques (e.g., dialogue, description) and appropriate text structures
and transitional strategies for coherence when revising one or more paragraphs of narrative text (e.g., closure,
introduce narrator, or use dialogue when describing an event).
Standards: W-3a, W-3b, W-3c, W-3d, and/or W-3e
(DOK 3)
W-3
a. Engage and orient the reader by establishing a context and point of view and introducing a narrator and/or
characters; organize an event sequence that unfolds naturally and logically.
b. Use narrative techniques, such as dialogue, pacing, and description to develop experiences, events, and/or
characters.
c. Use a variety of transition words, phrases, and clauses to convey sequence and signal shifts from one time
frame or setting to another.
d. Use precise words and phrases, relevant descriptive details, and sensory language to capture the action
and convey experiences and events.
e. Provide a conclusion that follows from and reflects on the narrated experiences or events.
Claim 3 and Its Targets
• Students can employ effective speaking
and listening skills for a range of purposes and
audiences.
1. Language & Vocabulary Use
2. Clarify Message
3. Plan/Speak/Present
4. Listen/Interpret
Grade 11 Claim 3 Target 3
4. LISTEN/INTERPRET: Analyze, interpret, and use information delivered orally.
Standards: SL-1, SL-2, SL-3
(DOK 1 DOK 3)
SL-1 Initiate and participate effectively in a range of collaborative discussions (one-on-one in groups and
teacher-led) with diverse partners on grades 11–12 topics texts and issues building on others’ ideas and
expressing their own clearly and persuasively.
SL-2 2 Integrate multiple sources of information presented in diverse formats and media in order to make
informed decisions and solve problems, evaluating the credibility and accuracy of each source and noting any
discrepancies among the data.
SL-3 Evaluate a speaker’s point of view, reasoning, and use of evidence and rhetoric, assessing the stance,
premises, links among ideas, word choice, points of emphasis, and tone used.
Claim 4 and Its Targets
• Students can engage in research / inquiry to
investigate topics, and to analyze, integrate,
and present information.
1. Plan/Research
2. Interpret & Integrate Information
3. Analyze Information/Sources
4. Use Evidence
Grade 4 Claim 4 Target 3
3. ANALYZE INFORMATION/SOURCES: Distinguish relevant-irrelevant information
Standards: RI-1, RI-7, RI-9, W-8, W-9
(PT: DOK 4) (CAT: DOK 2)
• RI-1 Refer to details and examples in a text when explaining what the text says
explicitly and when drawing inferences from the text.
•
RI-7 Interpret information presented visually, orally, or quantitatively (e.g., in
charts, graphs, diagrams, time lines, animations, or interactive elements on
Web pages) and explain how the information contributes to an understanding
of the text in which it appears.
• RI-9 Integrate information from two texts on the same topic in order to write
or speak about the subject knowledgeably.
• W-8 Recall relevant information from experiences or gather relevant
information from print and digital sources; take notes and categorize
information, and provide a list of sources.
• W-9 Draw evidence from literary or informational texts to support analysis,
reflection, and research.
ELA/Literacy
General
Item and
Task
Specification
Smarter BalancedELA Response Types
•
•
•
•
•
•
•
•
Multiple-choice, single correct response
Multiple-choice, multiple correct response
Two part multiple choice
Matching Tables
Select Text
Order Text
Short text- constructed response
Long essay- performance tasks
Ways to Score
• Automatic with a
single key
• Automatic with
multiple keys
• Automatic with
machine rubric
• Hand score with
rubric
• Multiple-choice, single
correct response
• Multiple-choice, multiple
correct response
• Two part multiple choice
• Matching Tables
• Select Text
• Order Text
• Short text- constructed
response
• Long essay- performance
tasks
Considerations
• Accessibility
– Ensure use of most accessible response types to
measure standards
– Include careful review of graphic and language use
in items
• Rubrics
– Include specific scoring rules to account for all
correct responses
– Carefully constructed for use by automated
scoring
Development Guidelines: SR, CR, and
PT
• Selected response and constructed response primarily
focus on one assessment target
• Performance tasks written to multiple claims and targets
• Items should not measure students’ feelings or values
• Items should not provide advantage or disadvantage to a
particular group of students
• Items designed using universal design
• Items coded to Smarter Balanced cognitive levels (pg. 68)
• Vocabulary should follow Common Core 3-tier and
academic vocabulary (pg. 69-70)
• http://www.wascd.org/images/SBAC/ELAGeneralItemandT
askSpecifications.pdf
Digging Deeper:
SR Stems, Options, Claims, and Scoring
• Stem presents a complete problem
– Grades 3-5 stems usually in form a question
– Grades 6-11 stems questions and also open-ended stems
• Options must be plausible and a correct answer should be
present
– Length and syntax should be similar
– Arrange according to a logical order
– Item writers must include a distractor analysis to provide
rationale common misunderstandings of a concept.
• All 4 Claims have SR items
• Scoring: Typically 1 point per item, unless more than one
answer, then can be 2 points
• Timing: Grade 3 allow for 3 minutes per item, Grade 4-11
allow 2 minutes per item
Digging Deeper:
CR Stems, Options, Claims, and Scoring
• Stems should provide the question and
scoring information
– 2-point and 3-point CRs require an explanation
with evidence that supports the explanation
– Most CR items relate to a stimulus
• All 4 Claims have CR items
• Scoring: 2-point and 3-point rubrics are used
that are unique to each item
• Timing: 5-10 minutes
Sample
Claim 1
CR
Rubrics
Sample Claim 4 CR Rubrics
Sample Claim 2 Writing CR Rubrics
Note: This is a brief write rubric. The CR typically asks students to
write a paragraph or two. This is not used to score essays.
Technology-Enhanced Items
• All 4 Claims have TE items
• Specialized interaction
• Same requirements as selected and
constructed response items
• Students manipulate information
• Defined responses, score points typically 1 or
2
Technology-Enhanced Example Item
Below is a poem, a sonnet, in which the speaker discusses her feelings about a relationship.
Read the poem and answer the question that follows.
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
*vestige: a mark, trace, or visible evidence of something that is no longer present or evident.
In the sonnet “Remember,” which two lines reveal a change in the speaker’s message to her subject?
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
The notes for a summary need to be arranged
correctly in the order in which the events occurred in
the passage. Click on each sentence and move it to
arrange the sentence into correct chronological order.
Summary of Events:
Maria laughs with the old women.
The guest and family eat dinner.
Maria’s mother asks the guests for a story.
Maria’s guests arrive.
Maria becomes sad.
The guests take turn telling stories.
INTERACTION
SPACE
Below is a poem, a sonnet, in which the speaker discusses her feelings
about a relationship.
Read the poem and answer the question that follows.
Remember
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
Remember me when no more day by day
You tell me of our future that you plann’d:
Only remember me; you understand
It will be late to counsel then or pray.
Yet if you should forget me for a while
And afterwards remember, do not grieve.
For if the darkness and corruption leave
A vestige* of the thoughts that once I had
Better by far you should forget and smile
Than that you should remember and be sad.
*vestige: a mark, trace, or visible evidence of something that is no longer
present or evident.
In the sonnet “Remember,” which two lines reveals a change in the
speaker’s message to her subject?
TYPE OF
INTERACTION
SPACE
Scoring
Rule
Scoring Rule: Logic used to score student response
Who is the protagonist in the story?
A. Joe
Scoring Rule:
If student response = C, then correct
B. Sue
Otherwise, incorrect
C. Maria
D. The father
Classify each word below based on
whether it is a verb or a noun.
Verbs
Nouns
Scoring Rule:
If object 1 = B, 2 = A, 3 = B, 4 = A,
5 = A, then correct
Otherwise, incorrect
Doll
Run
Dog
Swim
Eat
Technology-Enhanced Item Types
• Common English Language Arts TechnologyEnhanced item types
– Dropdowns
– Classification
– Reorder text
– Select and order
– Select text
Below is a poem, a sonnet, in which the speaker discusses her feelings
about a relationship.
Read the poem and answer the question that follows.
The notes for a summary need to be arranged
correctly in the order in which the events occurred in
Remember
Classify
each word
below
based
on and move it to
the passage.
Click
on
sentence
When you can
no more
holdeach
me by the
hand,
Nor I half turn toitgois
whether
aturning
verbstay.or ainto
noun.
arrange
theyetnosentence
correct chronological order.
Remember
me when
more day by day
You
tell
me
of
our
future
that
you
plann’d:
VerbsOnly remember me; you understand
Nouns
ItSummary
will be late to counsel
then
or
pray.
of Events:
Yet if you should forget me for a while
MariaAnd
laughs
with
thedoold
women.
afterwards
remember,
not grieve.
For if the darkness and corruption leave
The guest
and
eatonce
dinner.
A vestige*
of thefamily
thoughts that
I had
Better by far you should forget and smile
Maria’s
mother asks the guests for a story.
Than that you should remember and be sad.
Doll
Run
Dog
Swim Eat
Maria’s
guests
arrive.
*vestige: a mark, trace, or visible evidence of something that is no longer
Maria
becomes sad.
present
or evident.
The guests take turn telling stories.
In the sonnet “Remember,” which two lines reveals a change in the
speaker’s message to her subject?
Sample Technology-Enhanced ELA
• Highlight text
• http://www.cde.ca.gov/ta/tg/sa/question-types.asp
TE: Mark the box
TE: Drag and Drop
TE: Select Text
TE: Matching tables
ELA Passages/Stimuli
• Passages and Stimuli are found in all 4 Claim
test questions
Claim 1 Stimuli
• Reading comprehension
• Selected Response, Constructed Response, and
Technology-Enhanced Items
• Literary texts
– narratives, dramas, poems
• Informational texts
– literary nonfiction, historical, scientific, and technical
texts
• On grade level
• May include multiple texts
Claim 2 Stimuli
• Writing, revising, and editing
• Selected Response, Constructed Response,
and Technology-Enhanced Items
• Literary and informational texts, audio/video
presentations, texts needing editing/revising
• Shorter than other stimuli
• Should be one grade below grade level
Claim 3 Stimuli
• Speaking and listening
• Selected Response, Constructed Response,
and Technology-Enhanced Items
• Audio presentations, 1-2 minutes in length
Claim 4 Stimuli
• Research skills
• Selected Response, Constructed Response,
and Technology-Enhanced Items
• Includes literary text, informational text, audio
presentations, statistical data, simulated web
pages, and visual stimuli
• May use multiple stimuli
• Should be one grade below grade level
Performance Task Stimuli
• Includes informational text, audio
presentations, statistical data, simulated web
pages, visual stimuli
• Should be one grade below grade level
• Multiple stimuli
Selecting Appropriate Materials
• Interesting and grade
appropriate topics
• Free of bias and
sensitivity issues
• Note: speaking and
listening passages and
performance task
sources have different
length requirements.
• Follow reading passage
length guidelines:
Grade
Maximum Word Count
3
650
4
750
5
750
6
950
7
950
8
950
High School (9–11)
1,100
Text Complexity
• Quantitative and
qualitative measures
– The quantitative
measures determine the
grade band for a text
(e.g., 6–8), while the
qualitative measures
help determine the
appropriate placement
within the band (e.g., 8)
• Quantitative measures
–
–
–
–
Flesch-Kincaid index
Lexile
Word count
Range of difficulty
Common Core Lexile Grade Level
Ranges
Text Complexity Grade Lexile Ranges Aligned
Band in the Standards to CCR Expectations
K-1
N/A
2-3
450-790
4-5
770-980
6-8
955-1155
9-10
1080-1305
11-CCR
1215-1355
Shortcomings of Quantitative
Measures
• Measures difficult to obtain for
– poems and dramas
– highly complex ideas
Qualitative Measures of Text
Complexity
English Language Arts State Collaborative on
Assessment and Student Standards Rubrics
– Meaning/Purpose
– Text Structure
– Language Features
– Knowledge Demands
Common Core State Standards
Appendix A
• Quantitative Measures – Pages 4-7
Stimulus Metadata
•
•
•
•
•
•
•
•
•
Title
Author
Source
Copyright status
Genre
Topic
Length
Reading level
Level of complexity
URL Access
• Here is the URL that takes you to Smarter
Balanced Practice Tests and Training Tests. If you
are using a web browser, only these work: Quick
Access (any users): Any Internet-connected
computer using a current Web browser including:
–
–
–
–
Mozilla Firefox
Google Chrome
Microsoft Internet Explorer 10
Apple Safari
• http://www.smarterbalanced.org/practice-test/
Questions?
• Survey on this training session
• Go here:
• https://www.surveymonkey.com/r/VPFNJVN