Transcript Leveraging Evidence-Centered Design Within Scenario

How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
Variable
features of
tasks
Aspects of assessment situations that
can be varied to shift difficulty or
focus.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
Potential work
products
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
since
so
Warrant for
task design
Alternative
explanations
ATTRIBUTE
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
Other knowledge/skills/abilities that
is KSAs
organizedmay be required by tasks.
The design pattern
Data
Data
around
Focal KSAs.Characteristic
They will Aspects
be of assessment situations that
concerning
concerning
features of
are needed to evoke evidence about
situation
performance
involved in the Claim,tasks
although
the focal KSAs.
Variable
Aspects of assessment situations that
there may be
other KSAs
that are
Other
features of
can be varied to shift difficulty or
information
Warrant for included in
the
target
of
inference
concerning
tasks
focus.
since
since
scoring
student vis a vis
assessment
(e.g., Model
Formation—but
Potential workwhat
What students actually say, do, or
situation
products
make, to produce evidence.
models, what context?).
Student acting in
assessment situation
Potential
Aspects of work products we might
observations
Associatedidentify
withand evaluate, as evidence
about students’ KSAs.
Characteristic Features of
Potential
Ways of evaluating work products to
Tasks.
rubrics
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
The Rationale
Variable provides
Aspects of background
assessment situations that
features
of of
canthe
be varied
to shift
difficulty or
into the
nature
Focal
KSAs,
tasks
focus.
andPotential
the kinds
of things that people
work What students actually say, do, or
do in products
what kindsmake,
of to
situations
that
produce evidence.
Potential
Aspects of work products
evidence
it. It contributes
to thewe might
observations
identify and evaluate, as evidence
Warrant in the assessment
about students’argument.
KSAs.
Other
information
concerning
student vis a vis
assessment
situation
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
Variable
Aspects of assessment situations that
Additional
KSAs
play
multiple
features of
can be
varied to
shift difficulty
or
tasks
focus.need to think about
roles. You
Potential work What students actually say, do, or
which ones
you really DO want
products
make, to produce evidence.
to includeAspects
as targets
of inference
Potential
of work products
we might
observations
identify
evaluate,ones
as evidence
(validity)
andandwhich
you
about students’ KSAs.
really DON’T (invalidity).
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
Variable
Aspects of assessment situations that
The
Additional
you DO
features of
can be variedKSAs
to shift difficulty
or
tasks
want tofocus.
include as targets of
Potential work What students actually say, do, or
inferencemake,
aretopart
of the claim.
products
produce evidence.
E.g., knowing
Mendel’s
laws
as
Potential
Aspects of
work products
we might
observations
identifyable
and evaluate,
as evidence a
well as being
to formulate
about students’ KSAs.
model in
an investigation.
Connected
with Variable
Potential
rubrics
Ways of evaluating work products to
produce
valuesof
of observations.
Features
Tasks.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
The Additional KSAs you
Variable
Aspects of assessment situations that
DON’T
asor
features of
can want
be variedto
toinclude
shift difficulty
tasks
focus.
targets of
inference introduce
Potential work What students actually say, do, or
alternative
explanations for poor
products
make, to produce evidence.
performance.
(Especially
Potential
Aspects of work
products we might
observations
identify
evaluate, as evidence
important
forand
assessing
special
about students’ KSAs.
Connected
with &
Variable
populations
– UDL
Potential
Ways of evaluating work products to
Features of Tasks & Work
acommodations.)
rubrics
produce values of observations.
Products.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
Characteristic
features of
tasks
Aspects of assessment situations that
are needed to evoke evidence about
the focal KSAs.
Variable
features of
tasks
Aspects of assessment situations that
can be varied to shift difficulty or
focus.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
Features
PotentialThe
workCharacteristic
What students actually
say, do, or of
products
make, to produce evidence.
Tasks help you think about
Potential
Aspects of work products we might
critical
data
the
observations
identify
andconcerning
evaluate, as evidence
about
students’
KSAs.
situation
–what
you
need to get
Potential
Ways
of evaluating
work products
to
evidence
about
the Focal
KSAs.
rubrics
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
ATTRIBUTE
Focal
The primary knowledge / skills /
Variable Features
oftargeted
Tasksbyalso
Knowledge,
abilities (KSAs)
this design
Skills, Abilities pattern.
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
since
so
Data
concerning
situation
Warrant for
task design
since
Student acting in
assessment situation
help you think about data
Rationale
How/why this DP addresses evidence
concerning
the
– but
about
focalsituation
KSAs
now to influence
difficulty …that
Additional
Other knowledge/skills/abilities
KSAs
Characteristic
features of
tasks
Data
concerning
performance
Warrant for
since
scoring
DESCRIPTION
Other
information
concerning
student vis a vis
assessment
situation
Variable
features of
tasks
Potential work
products
may
by or
tasks.
orbetorequired
bring in
reduce
demand
for Additional
Aspects
of assessment
situationsKSAs
that
aretoneeded
evoke evidence about
avoidtoalternative
theexplanations.
focal KSAs.
Aspects of assessment situations that
can be varied to shift difficulty or
focus.
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
ATTRIBUTE
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
since
so
Data
concerning
situation
Warrant for
task design
since
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
DESCRIPTION
Focal
The primary knowledge / skills /
Some
Variable
Features of Tasks
Knowledge,
abilities (KSAs) targeted by this design
help
you match
Skills,
Abilities
pattern. features of tasks
Rationale
How/why this
DP addresses evidence
and background
/ knowledge
/
about focal KSAs
characteristics of students:
Additional
Other knowledge/skills/abilities that
Interests,
familiarity,
KSAs
may
be required byprevious
tasks.
instruction.
Characteristic Aspects
of assessment situations that
features of
tasks
are needed to evoke evidence about
the focal KSAs.
Variable
features of
tasks
Aspects of assessment situations that
can be varied to shift difficulty or
focus.
Potential work
products
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
ATTRIBUTE
Focal
The primary knowledge / skills /
Knowledge,Work
abilities
(KSAs) targeted
by you
this design
Potential
Products
help
Skills, Abilities pattern.
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
since
so
Data
concerning
situation
Warrant for
task design
since
Student acting in
assessment situation
think aboutHow/why
whatthis
you
want to
DP addresses evidence
aboutafocal
KSAs
capture from
performance
–
Additional
Other knowledge/skills/abilities
product, process,
constructed that
KSAs
may be required by tasks.
model, written explanation, etc.
Rationale
Characteristic Aspects of assessment situations that
of attention
are needed
Canfeatures
also call
to to evoke evidence about
tasks
the focal KSAs.
demand
for Additional
Data
concerning
performance
Warrant for
since
scoring
DESCRIPTION
Other
information
concerning
student vis a vis
assessment
situation
Variable
Aspects of assessment situations that
KSAs,
& avoid alternative
features of
can be varied to shift difficulty or
explanations
(e.g.,
Stella)
tasks
Potential work
products
focus.
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
Additional
KSAs
Other knowledge/skills/abilities that
may be required by tasks.
since
so
Data
concerning
situation
ATTRIBUTE
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other
information
concerning
student vis a vis
assessment
situation
Potential Observations are
Characteristic Aspects of assessment situations that
possibilities
the
qualities
features
of
are for
needed
to evoke
evidenceof
about
tasks Products
the focal KSAs.
Work
– i.e., the data
Variable
Aspects
ofperformance.
assessment situations that
concerning
the
features of
can be varied to shift difficulty or
tasks
Potential work
products
focus.
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.
How Design Patterns Support Thinking
about the Assessment Argument
Claim about student
unless
Warrant for
assessment
argument
Alternative
explanations
Warrant for
task design
since
DESCRIPTION
Focal
Knowledge,
Skills, Abilities
The primary knowledge / skills /
abilities (KSAs) targeted by this design
pattern.
Rationale
How/why this DP addresses evidence
about focal KSAs
since
Additional
And
KSAs
so
Data
concerning
situation
ATTRIBUTE
algorithms/rubrics/rules
for that
Characteristic
Aspects of assessment situations
features of
are needed to evoke evidence about
evaluating
Work Products to get
tasks
the focal KSAs.
Data
concerning
performance
Warrant for
since
scoring
Student acting in
assessment situation
Other knowledge/skills/abilities that
Potential
Rubrics
are
may be required
by tasks.
Other
information
concerning
student vis a vis
assessment
situation
the
Variable
features of
tasks
data
concerning
Aspects
of assessmentthe
situations that
can be varied to shift difficulty or
performance.
Potential work
products
focus.
What students actually say, do, or
make, to produce evidence.
Potential
observations
Aspects of work products we might
identify and evaluate, as evidence
about students’ KSAs.
Potential
rubrics
Ways of evaluating work products to
produce values of observations.