Transcript Science Alliance

Science Alliance
November 29, 2007
Please submit your MC Question
with your next step suggestions.
Group Norms
•
•
•
•
•
•
•
Start and end on time
Put cell phones on silent
Be respectful of all comments
Everyone participates
Exercise the rule of “two feet”
Come prepared for the meeting
Keep side conversations to a minimum
Review from October
• Completed deconstruction
of standards addressed in
your unit
• Developed a performance
task
• Experienced a
vocabulary/engagement
strategy
• Examined an assessment
plan for congruency
between stages 1 and 2
• Deepened understanding
of FA
Agenda for Today
• Continue Stage 2
Development


–
–
–
–
–
Rubric Development
Peer Review
FA Strategy
Quality MC and ORQ’s
DoK Levels
Item Development
Self Assessment of
Alignment
• Vocabulary Strategy
• “In times of change, the learners will
inherit the world, while the knowers
remain well-prepared for a world that no
longer exists.”
– Eric Hoffer, writer
Performance Assessment
• There are two parts to a performance assessment:
– The Task
• Simple target – simple task
• Complex target – complex task
– The Performance Criteria
• We use performance assessments when the
learning targets require doing (e.g., design,
produce, make, create, make, write, draw,
represent, display, model, construct).
• Performance assessments are ways to allow
students to demonstrate how well they can do
science – not the drills, but the game!
Performance Assessment
• The task is any activity that we use as a context to
observe a skill or a product – a naturally occurring
event or a separate event, at the end of instruction
or during instruction. The only requirement is that
the task elicits the desired skill or product so that it
is capable of being judged. Pg. 194 CASL
• …performance criteria [is] the basis for judging
the quality of the performance on the task. Pg. 194 CASL
Benefits of Performance Criteria
• To help educators clarify the nature of complex
learning targets so that they feel comfortable
teaching to them
• To assess student progress and status in ways that
are consistent across students, assignments, and
time
• To improve student achievement by letting
students in on the secret of the nature of quality
• Through all these things, to integrate assessment
and instruction and grasp the essence of standardsbased instruction
– Scoring Rubrics in the Classroom by Arter and McTighe, pg. 16
Benefits of Performance Criteria
For Teachers:
1. Consistency in
Scoring
2. Improved Instruction
For Students:
1. Clear Target(s)
2. Quality Performance
3. Self Assess and
Adjust
Rubrics
• A rubric is a particular format for
criteria – it is the written down
version of the criteria, with all the
score points described and defined.
The best rubrics are worded in a way that
covers the essence of what we, as teachers,
look for when we’re judging quality, and they
reflect the best thinking in the field as to what
constitutes good performance. Rubrics are
frequently accompanied by examples
(anchors) of products or performances to
illustrate the various score points on the
scale.
– Scoring Rubrics in the Classroom by Arter and
McTighe pg. 8
Carousel Brainstorming –
Performance Criteria
• As a table group, brainstorm what performance
criteria should do for us in the classroom. Capture
your ideas on chart paper. 5 mins.
• Pass your brainstormed list to another table, add to
their list any other ideas not included. 3 mins.
• Exchange charts one more time and add ideas not
listed. 3 mins.
• What can you generalize about performance
criteria? What are the implications for your
performance task?
Carousel Brainstorming –
Rubric Features
• As a table group, brainstorm the features a rubric
should have to serve the performance criteria
purposes. Capture your ideas on chart paper. 5
mins.
• Pass your brainstormed list to another table, add to
their list any other ideas not included. 3 mins.
• Exchange charts one more time and add ideas not
listed. 3 mins.
• What can you generalize about features of rubrics?
What are the implications for your rubric?
What Kind of Rubric?
• Working with a partner, write your
definition for each type of rubric listed and
give an example of what type of assignment
would be appropriate for each.
• Jot down some disadvantages for each type.
• Sketch an example for each.
• Share with your tablemates and revise your
ideas as needed.
What Kind of Rubric?
• Identify the type of rubric that would best serve
your needs and your student’s needs.
–
–
–
–
Holistic
Analytic
Generic (General)
Task-specific
• Does it need to be unique to your task or is it one
that could be used across similar performances
(and thus best developed as a department or school
team)?
Common Problems with
Instructional Rubrics
• Counting – when quality is more important
than quantity
• Important details left out
• Irrelevant details in the rubric
• Student-developed rubrics where anything
goes
• Skimpy scoring guides
• Converting performance standards to grades
Developing Rubrics
•
•
•
•
•
Establish a knowledge base
Gather samples of student performance
Sort student work by level of quality
Cluster the reasons into traits
Identify sample performances that illustrate
each level
• Make it better
– Pg 211 CASL
Work Time
• Complete/revise your performance task
using the criteria on pg. 220 in CASL and
the GRASP format from UbD workbook pg.
172 as guides.
• Develop the rubric for your task. Use the
Metarubric Summary on pg. 203 in CASL
and pgs. 181-195 in the UbD workbook as
resources for development.
Peer Review of Performance
Task and Rubric
• Identify the PoS and/or CCA
standard(s) being addressed by
the performance task.
• List what you consider the most
obvious/important knowledge,
reasoning, and skills that will be
required by students to
successfully complete the task.
• Use the Task Rubric Summary
on pg. 220 in CASL to provide
feedback to the developer(s)
concerning the task.
Peer Review of Performance
Task and Rubric
• Use the Metarubric Summary on pg. 203 in
the CASL book as well as common
problems with instructional rubrics
discussed earlier (pgs. 205-210 in CASL) to
provide feedback to the developer(s)
concerning their draft rubric.
• Meet with the developer to discuss strengths
and weaknesses of the task and rubric.
Agenda for Today
• Continue Stage 2
Development
–
–

–
–
–
–
Rubric Development
Peer Review
FA Strategy
Quality MC and ORQ’s
DoK Levels
Item Development
Self Assessment of
Alignment
• Vocabulary Strategy
Research-based Strategies
• 4 Research-based strategies that
significantly improve student learning:
Sharing criteria (clear learning targets with
success criteria)
Questioning
– Feedback
– Peer and self-assessment
• “More effort has to be spent in framing
questions that are worth asking; that is,
questions which explore issues that are
critical to the development of students’
understanding.”
– Assessment for Learning: Putting It Into Practice
• Pg. 42
Effective Questioning
• “You understand it
only if you can teach
it, use it, prove it,
explain it, or read
between the lines.”
– Wiggins and McTighe
– Understanding by Design
• “Current classroom
practices generally
encourage superficial and
rote learning,
concentrating on recall of
isolated details, usually
items of ‘knowledge’
which pupils soon forget.”
– Black and Wiliam
– Inside the Black Box
Effective Questioning
• Questioning should be
used to:
– Cause thinking
– Provide data that
informs teaching
• Questioning should
NOT be used to:
– Direct the attention of
the class
– Keep students on task
• Dylan Wiliam, NSTA
Preconference on
Assessment, 2005
Thinking Hats
• Draw a hat from the baggie.
• Find 2 others in the room with the same hat.
• Consider the following question from the
perspective specified by your hat color:
Does human activity affect Earth’s ecosystems,
both locally and globally?
• Return to your original table. Have each person
share their “hat” perspective pertaining to this
question.
Six Thinking Hats
•
•
•
•
•
•
White Hat – facts, figures, information
Black Hat – caution, truth, judgment
Red Hat – emotions, feelings, hunches
Yellow Hat – advantages, benefits, good
Green Hat – create, explore, new ideas
Blue Hat – reflections, lacks decisions
Formative Assessment in Action, p. 66
• “If the only tool you have is a hammer, you
tend to see every problem as a nail.”
– Abraham Maslow
Agenda for Today
• Continue Stage 2
Development
–
–
–



–
Rubric Development
Peer Review
FA Strategy
Quality MC and ORQ’s
DoK Levels
Item Development
Self Assessment of
Alignment
• Vocabulary Strategy
Key Resources for Development
• Academic Expectations*
• Core Content for Assessment*
• Test Blueprint*
• Kentucky Performance Level Descriptions
• Content resources
* These three play a key role in item
development.
General Item-Writing Guidelines
Alignment to Standards
• The match to Core Content for
Assessment is essential to the validity of
the KCCT.
Content Relevancy
• Make sure the item assesses important
knowledge or skills identified by
standards in the Kentucky Core Content
for Assessment.
Sample Item
The Preamble and the Articles of
Confederation were written in which
year?
A. 1697
B. 1765
C. 1787
D. 1865
Sample Item
Which document describes the
purpose of the U.S. Constitution?
A. Preamble
B. Bill of Rights
C. Articles of Confederation
D. Declaration of Independence
General Item-Writing Guidelines
Grade-Level Appropriateness
• Make sure the item reflects the Core
Content for Assessment at the
appropriate grade level.
Basic vs. Technical Vocabulary
• Use simple, basic vocabulary instead of
technical vocabulary unless you are
assessing the students’ knowledge of
the meaning of the technical
word/phrase.
Sample Item
Science, Grade 4
When you plant a seed, the roots grow
downward. This is called geotropism.
Which factor is responsible for
geotropism?
vs.
When you plant a seed, the roots grow
downward, and the stem grows upward.
Which factor is responsible for the
roots growing downward?
General Item-Writing Guidelines
 Grade Appropriate Vocabulary
• Use grade-appropriate vocabulary as much as
possible. Check the vocabulary lists to
determine the grade level of words being used
in the item.
 Essential Information Only
• Make sure that the item only includes
information essential to understanding and
answering the item. Text that is not essential to
the item increases the reading burden on the
student.
General Item-Writing Guidelines
 Clear, Correct, and Understandable
Graphics
• Include clear, correct, easily
understood graphics as required by
the item. Do not use a graphic if it
is not really essential to answering
the item.
 Bias and Sensitivity
• Consider Bias and Sensitivity
Guidelines when drafting items.
Guidelines for Developing
Multiple-Choice Items
Advantages of Multiple-Choice Items
• Can be used to measure a wide
variety of learning outcomes
• Permit wide sampling and broad
coverage of a content domain
• Are reliable and efficient to score
• Can provide useful diagnostic
information about the learning of
individual students or groups of
students
Disadvantages of Multiple-Choice Items
• Multiple-choice items are difficult to write.
• Multiple-choice items cannot measure
certain types of skills (e.g., the ability to
organize and express ideas in writing).
• Performance on multiple-choice items can
be influenced by student characteristics
unrelated to the subject of measurement,
such as reading ability and “testwiseness.”
Guidelines for Developing
Multiple-Choice Items
Tips for writing response options…
• Make the incorrect response options
plausible.
• Make sure one response option (especially
the correct answer) does not stand out as
being different from the other options. One
option should not be much longer or shorter
than the others, should not be different
grammatically, etc.
Guidelines for Developing
Multiple-Choice Items
More tips…
• In science and math items, consider
using common misconceptions or errors
as response options.
• When appropriate, use other concepts or
terms from the grade-appropriate Core
Content for Assessment.
Guidelines for Developing
Multiple-Choice Items
More tips…
• Avoid humorous or nonsensical
response options.
• Avoid using the options “all of the
above” and “none of the above”.
Guidelines for Developing
Multiple-Choice Items
More tips…
• Logically order the response options that
include numbers, dates, etc. Numbers
should be listed in ascending or
descending order. Unless testing the
sequence of events from a passage, list
options in the order in which they appear
in the passage.
Training Examples
for
Multiple-Choice Items
Do all four response options make
sense? Each response option should
make sense both grammatically and
with respect to content.
In order to grow and flourish, pumpkins need
lots of
A. milk, eggs, and molasses.
B. sun, water, and space.*
C. gourds, melons, and cucumbers.
D. cream, eggs, and lots of sugar and
spices.
Is the vocabulary used in the stem
and response options grade-level
appropriate? It is very important
that the wording used in the stem and
response options be grade-level
appropriate, otherwise the item may
be assessing a student’s knowledge of
vocabulary rather than content
knowledge.
What is the one way heat can
move from one object to
another?
A. acceleration
B. insulation
C. adaptation
D. conduction*
Is there one – and only one – right
answer? Each item must have only
one right answer.
According to the passage, where
do most home accidents occur?
A. in the kitchen
B. on the cutting boards
C. on the stove-top burners*
D. in hot ovens
Is there any inadvertent cueing
going on in the item? Sometimes
items are written in such a way
that one of the response options
contains an important word or
phrase from the stem, which leads
the student to choose this option.
A style of architecture that uses
the Roman arch, thick walls, and
dimly lit interiors is called
A. Gothic.
B. modern.
C. Romanesque.
D. Byzantine.*
Does one response option stand out
from the others in any way? Some
items are written so that one of the
response options clearly stands out
from the other response options. There
are many ways that a response option
may stand out from the other options.
One way is if one of the options begins
with a different word than the other
three options.
In The Prince, Renaissance author,
Machiavelli, instructs the ruling monarchy
in methods of statesmanship and argues
A. for reconciliation of faith and reason.
B. that the ends justify the means.*
C. for the rise of the common man.
D. for allowing freedom of thought.
Another way that an option can stand
out is in length. If an option is much
longer or shorter than the other three
options, it stands out.
Some students want to build a pond near their
schoolyard.
This change will MOST LIKELY
A. increase the frog population and be harmful
to the grass in the area.*
B. be harmful to both frogs and grass.
C. have little effect on the plants in the area.
D. cause most animals to leave the area.
• An option can stand out because it is
the only option that contains a
negative or a negative connotation.
Response options need to be balanced
with respect to the use of negatives, so
that one option does not stand out.
Which is a reason Mike went to the park?
A.
B.
C.
D.
He wanted to play with his friends.
He did not want to go to school.
He liked playing on the swings.
He hoped to find his notebook.
Janice can best be described as
A.
B.
C.
D.
angry.
mean.
rude.
caring.
Is the item format an appropriate
one? Items should not be written in
certain formats. For example, “fill in
the blank format” should be avoided,
as should the analogy format.
Analogies in particular, are very
difficult for many students.
Color is to art as _______ is to music.
A. melody
B. rhythm
C. dynamics
D. timbre*
Is there any extraneous information
in the stem? The stem of an item
should only include information that
is essential to answering the item.
For example, in the item below, the
first sentence is not essential to the
item.
Corn is one of the largest crops
grown in the United States. There
are 2,500,000,000 bushels of
corn grown in the U.S. each year.
If corn production is decreased by
50% each year, what will be the
U.S. corn production in 4 years?
Does the item stem include all
information essential to answering
the item? Just as some items include
too much information, other items
include too little. It is very important
for an item stem to include all of the
information necessary to answer the
item or necessary for the answer to be
correct.
To conduct an experiment, Mike pulls marbles from a
bag that contains 100 marbles. Each time he pulls a
marble he records the color and then returns it to the
bag. He does this 10 times. His results are shown
below. Theoretically, how many red marbles and blue
marbles are in the bag?
Red / Blue
A. 60 red, 40 blue
B. 50 red, 50 blue
C. 70 red, 30 blue*
D. 30 red, 70 blue
In this article, passive smoke is defined as
A. only being found in public buildings.
B. affecting only smokers.
C. inhalation of environmental tobacco
smoke.
D. associated only with passive smokers.
In the article, the word “affluent” is used to
describe people with
A. homes in urban areas.
B. homes in rural areas.
C. the opportunity to gain and use resources.
D. live in specific locations around the world.
Charles Darwin
A. is the father of modern genetics.
B. described the theory of natural
selection.
C. was a famous geologist.
D. founded the genome project.
When drafting multiple-choice items...
 develop items based on need according to
the Test Blueprint.
 write the item based on the knowledge
and skill to be assessed by the Core
Content statement.
 write items on the multiple-choice item
development form and fill out the form
completely.
 prepare for the peer review of your items.
Guidelines for Developing
Open-Response Items
Advantages of Open-Response Items
• Open-response items allow for more
depth of knowledge to be
demonstrated than do multiple-choice
items.
• Open-response items allow students
to demonstrate more complex
cognitive behaviors, such as
comparing, relating, analyzing,
inferring, concluding, predicting,
generalizing, solving and/or applying.
Disadvantages of Open-Response Items
• Open-response items are more difficult
and more expensive to score.
• Because of their cost, the relative
number of open-response items is
significantly less than multiple-choice
items.
• Effectiveness of open-response items is
based on the scoring guide and answer
information provided.
Guidelines for Developing
Open-Response Items
•Achievable Tasks
Make sure that the task is actually
achievable. If the item asks students to
read a passage or examine a graphic and
then “give three ways” or “explain two
reasons” based on the material given, make
sure there are three ways or two reasons
found in the material provided.
Guidelines for Developing
Open-Response Items
• Clearly Formulated Prompt
Make sure that each item has an item
prompt (i.e., one or more introductory
sentences that set the stage for, or provide
context for, the item). The prompt
precedes the directions. Even if the item
has a graphic, there still must be a prompt
that describes or provides information
related to the graphic and/or item
directions.
Guidelines for Developing
Open-Response Items
•Clearly Formulated Directions
If you want the students to provide
examples, the directions to the students
should tell them to provide examples and
how many.
If you want the students to identify
information, use the word “identify,” not
“describe,” “explain,” etc. Choose your words
carefully.
Guidelines for Developing
Open-Response Items
• Clear Link Between Prompt and Directions
The link between the prompt and the
directions should be clear.
Do not ask students to do something in
part b of the directions that has no
relationship to part a of the directions.
Guidelines for Developing
Open-Response Items
Tips for Writing Open-Response Items…
As a general rule, if students are asked to
generate a certain number of ideas on
their own without benefit of a passage or
graphic, then there should be at least
twice that number of possible answers.
You will be asked to supply the possible
answers.
Guidelines for Developing
Open-Response Items
More tips…
• If there are numerous details in the prompt,
use bullets to emphasize the details.
• If the students are required to respond to
multiple parts of a question, label each part
separately (a,b,c).
Sample Item
Martin said, “I am thinking of a whole number
between 100 and 300.
 The number is divisible by three but not by 9.
 The ones digit is the sum of the hundreds digit
and the tens digit.”
A.Show why 153 cannot be Martin’s number.
B. Find all the numbers that match Martin’s clues.
Show all your work.
C.Write one more clue that would limit the
answer in part b to one and only one correct
number.
Developing Open-Response Items
• Establishing the Prompt
• Designing Directions
Developing a Prompt…
• What is the Core Content to be
assessed?
• What specific prompt is appropriate
to provide context for the students
to demonstrate their knowledge?
• Is the prompt age-appropriate?
• Does it include extraneous or
misleading information?
Designing Directions
• Do the directions clearly reflect knowledge
and/or skills from the Core Content?
• Do the directions specify what students must do?
• Do the directions provide students the
opportunity to demonstrate proficient
performance?
• Are the directions clearly formulated?
• Is there a clear link between the prompt and the
directions?
• Is the task achievable?
• Is it answerable within one page?
Types of Open-Response Items
• Scaffold
• Single Dimension/Component
• Student Choice: Topics/Options
Provided
• Response to Provided Information
Scaffold
• Sequences increasingly more
difficult/complex tasks
• Success on one part would likely mean
there was success on previous parts
• Multipart (simple to complex)
Mathematics Example
Corina was investigating information about
natural wonders of the world.
• She found that Mt. Everest is the highest
mountain in the world. It is 29,028 feet
ABOVE sea level.
• She found that the Marianas Trench in
the Pacific Ocean is the lowest point on
Earth. It is 35,840 feet BELOW sea level.
a. If Corina could throw a rock from the top of
Mt. Everest to the bottom of the Marianas
Trench, how many feet would the rock fall?
b. Draw a diagram and explain your answer for
part a.
Single Dimension/Component
• Straightforward question
• Draw a conclusion or take a
position, then support it
• Explain a phenomenon or describe
procedures
Social Studies Example
Many Kentucky cities are located near
large rivers.
a. Describe three important advantages that
the rivers provide these cities.
b. Explain why each advantage you
described in part a is important.
Student Choice: Topic/Options
Provided
• Choose from the provided options
• Complete directions as specified
Science Example
Some of Earth’s materials are listed below
• soil
• water
• gases of the atmosphere
A. Choose TWO materials from the list.
Explain how a PLANT uses each of these
materials to live.
B. Choose TWO materials from the list.
Explain how an ANIMAL uses each of these
materials to live.
Response to Provided Information
• Response to provided information such as
data, readings, graphics
Reading Example
In the story “First Light,” Matthew woke up
in another time period, the 1850s.
A. Describe FOUR things Matthew
discovered that were different from
what he was used to in his present life.
B. Explain how EACH of those differences
affected him. Use information from the
story to support your answer.
Training Examples for
Open-Response Items
Is the item specific enough?
Sometimes items are written that are
not specific enough. Such items are
usually difficult to score and unfair to
students. The following item is an
example of an item that lacks
specificity and would probably pose a
problem for students.
When Mike’s class was studying insects, the teacher
asked each student to catch an insect and keep it alive for
a few weeks in a plastic jar.
a. Name an insect that you think Mike should catch
and keep in the jar.
b. Draw a picture of the jar with all the features that
will be necessary for the insect to live in the jar for
a few weeks. Make sure that you labeled all these
features.
c. After Mike has placed his insect in the jar that you
you have drawn, what must he do to make sure
that the insect lives.
Are the item directions directly tied to
the content of the item stem? In an
open-response item, it is important that
the item directions be directly tied to the
content of the stem. When the stem’s
content is disjointed from the item
directions, it may cause confusion for
students who look to the stem for clues
as to how to answer the item.
Although this story is about the narrator’s
feelings, the thoughts and feelings of her
parents are also included.
a. Identify two character traits of either the
mother or the father in the story.
b. Describe how the character’s actions
reveal each of his or her character traits.
Does the item stem inadvertently cue
students to one or more answers?
Just as with multiple-choice items, it is
important to make sure that the stem
of an open-response item does not cue
students to the answer(s).
In the article, the author writes,
“Pumpkins are more than just symbols
of autumn holidays. They are
thousands of years old and the stars of
folklore, fairy tales, and famous feasts.”
a. Describe three ways people used or
are now using pumpkins.
b. Explain why each way was or is
important.
When drafting open-response items…
• Draft items on scratch paper.
• Write down examples of information that
would be included in an acceptable response.
If the information does not come easily, you
probably do not have a good item.
• Evaluate the information to be sure that it is
possible to distinguish four levels of response
(0 – 4, Blank).
• Exchange items with someone else and have
that person review the prompt, directions, and
answer information.
• Revise item as needed.
DOK
Activate Prior Knowledge
• Please work individually and mark if you
agree or disagree.
Depth of Knowledge
Focuses on the cognitive processing that
is required to complete an assessment
task
Descriptive, not a taxonomy
Not the same as difficulty
Why Depth of Knowledge?
Mechanism to ensure that the intent of the
standard and the level of student
demonstration required by that standard
matches the assessment items (required
under NCLB)
Provides cognitive processing ceiling for
item development
Depth of Knowledge
Four levels of depth of knowledge:
Recall and Reproduction - Level 1
Skills & Concepts/Basic Reasoning - Level 2
Strategic Thinking/Complex Reasoning - Level
3
Extended Thinking/Reasoning - Level 4
What is the DOK level?
• For the following slides, chose which DOK
level is appropriate.
• Justify your choice
• Share with a partner
Recall and Reproduction - Level 1
requires recall of information, such
as a fact, definition, term, or
performing a simple process or
procedure. A student either knows
the answer or doesn’t.
Answering a Level 1 item can involve
following a simple, well-known procedure
or formula. Simple skills and abilities or
rote responses characterize DOK 1.
Skills & Concepts/Basic Reasoning Level 2
includes the engagement of some
mental processing beyond recalling
or reproducing a response. Items
require students to make some
decisions as to how to approach the
question or problem.
These actions imply more than one
mental or cognitive process/step.
Strategic Thinking/Complex Reasoning Level 3
requires deep knowledge as exhibited
through planning, using evidence, and
more demanding cognitive reasoning.
The cognitive demands at Level 3 are
complex and abstract.
An activity that has more than one
possible answer and requires students
to justify the response they give would
most likely be a Level 3.
Extended Thinking/Reasoning - Level 4
requires high cognitive demand and is very
complex. Students are expected to make
several connections—relate ideas within the
content or among content areas—and have
to select or devise one approach among
many alternatives on how the situation can
be solved.
Due to the complexity of cognitive demand,
DOK 4 often requires an extended period of
time.
However, extended time alone is not the
distinguishing factor.
Task
DOK Level
Collecting data samples over several
months
DOK 1
Organizing the data in a chart
DOK 2
Using this chart to make and justify
predictions
DOK 3
Developing a generalized model from
this data and applying it to a new
situation
DOK 4
DOK is not verb-dependent…
DOK 1- List two animals that survive by eating
other animals. (simple recall)
DOK 2- List two reasons that desert
environments do not support large carnivores.
(requires conceptual understanding of how organisms interact with
their environments)
DOK 3- List the data you would need to collect
and the experimental steps you would take to
investigate the impact of precipitation amounts
on the bear population of an island. (requires deep
understanding of how a scientific investigation is conducted)
Same verb—three DOK levels
DOK levels can be cumulative
An item/standard written to DOK 3
often also contains DOK 1 and DOK 2
level demands
Remember…
Depth of Knowledge (DOK) is a scale of
cognitive demand.
DOK requires looking at the assessment
item/standard-not student work-in
order to determine the level. DOK is
about the item/standard-not the student.
The context of the assessment
item/standard must be considered to
determine the DOK not just a look at
what verb was chosen.
Agenda for Today
• Continue Stage 2
Development
–
–
–
–
–


Rubric Development
Peer Review
FA Strategy
Quality MC and ORQ’s
DoK Levels
Item Development
Self Assessment of
Alignment
• Vocabulary Strategy
Work Time
• Using the guidelines for multiple choice items and
your deconstructed standards, develop 5-10
multiple choice items.
– Consider which targets are best (efficient and accurate)
assessed using MC items.
– Try to develop some DoK 1’s and some DoK 2’s.
• Using the guidelines for open response items and
your deconstructed standards, develop 1-2 open
response items.
– Consider which targets are best assessed using ORQ’s.
– Let the CCA verbs help you design your ORQ’s.
– Try to develop a DoK 3 ORQ.
Self-Assessment of Alignment
Checking for congruency between Stages 1 and 2.
• Did the standard identified by your PT/Rubric
review group match the ones you had chosen?
Did the peer review group identify the same
knowledge, reasoning, and skill targets that you
did?
• Highlight or check the deconstructed standards for
which you have developed at least one assessment
item.
• What gaps do you have in your assessment plan?
• What else do you need to develop to complete
Stage 2?
Wrap-up and Review
• Continue Stage 2
Development
–
–
–
–
–
–
–
Rubric Development
Peer Review
FA Strategy
Quality MC and ORQ’s
DoK Levels
Item Development
Self Assessment of
Alignment
• Vocabulary Strategy
Preview for January 24th
• Begin Stage 3
• Special guest presenter:
Bobb Darnell
– High yield strategies that
work
• Have a joyous holiday
season!