Transcript Designing Scoring Rubrics

Dr. Robert Mayes
University of Wyoming
Science and Mathematics
Teaching Center
[email protected]
Assessor – 3 basic questions
 What kind of evidence do we need to support the
attainment of goals?
 Tasks that reveal understanding, such as comparing and
contrasting or summarizing key concepts
 What specific characteristics in student responses,
products, or performances should we examine to determine
the extent to which the desired results were achieved?
 Criteria, rubrics, and exemplars are needed
 Does the proposed evidence enable us to infer a student’s
knowledge, skill, or understanding?
 Validity and reliability concerns
Stage 2: Evidence
Think like an assessor not an activity designer
Assessor
What should be sufficient
and revealing evidence
of understanding?
What performance tasks
must anchor the unit and
focus the instructional
work?
Against what criteria will I
distinguish work?
Activity Designer
What would be
interesting and engaging
activities on this topic?
What resources and
materials are available
on this topic?
How will I give students a
grade and justify it to
parents?
Stage 2: Evidence
Think like an assessor not an activity designer
Assessor
How will I be able to
distinguish between
those who really
understand and those
who don’t (though they
seem to)?
What misunderstandings
are likely? How will I
check for those?
Activity Designer
What will students be
doing in and out of
class? What
assignments will be
given?
Did the activities work?
Why or why not?
Continuum of Assessment Methods
 Vary in several characteristics
 Scope: from simple to complex
 Time Frame: short-term to long term
 Setting: decontextualized to authentic
 Structure: highly structured to ill-structured
 Move from snapshot to scrapbook
 Self-assessment of sources of evidence (HO)
Informal
checks
Observation/
Dialogue
Quiz/
Test
Academic
Prompt
Performance
Task
Collecting a Range of Evidence
 Activity: (HO) determine a range of assessment evidence
you may use related to the
 Enduring understanding
 Topics important to know and do
 Worth being familiar with
 Which assessment methods best fit the 3 categories?
Worth being familiar with
Important to know and do
Enduring Understanding
Academic Prompt
Assessments
 Open-ended question or problem that require student
to prepare a specific academic response
 Think critically and prepare response
 Require constructed response under exam conditions
 Divergent – no single best answer
 Subjective judgment based scoring using criteria or
rubric
 May or may not be secure
 Often ill-structured – require development of strategy
 Involve analysis, synthesis, and evaluation
Performance Task
Assessments
 Complex challenges that mirror the issues and
problems faced by adults
 Real or simulated settings, authentic
 Require student to address audience in non-exam




conditions
Divergent – no single best answer
Subjective judgment based scoring using criteria or
rubric,
Greater opportunity to personalize task
Not secure – students given criteria in advance
Performance Task – 6 Facets
 Activity: Use the 6 Facets of Understanding to
generate a performance task related to your enduring
understanding
 Questioning for Understanding (HO)
 Performance Verbs (HO)
 Performance Task creation (HO)
 Performance Task brainstorming (HO)
Performance Task -GRASPS
 Creating a performance task with context and roles
 Goal
 Role
 Audience
 Situation
 Product, Performance, and Purpose
 Standards and Criteria for Success
Performance Task -GRASPS
 Activity: Create a performance task using GRASPS
 GRASPS Performance Task Scenario (HO)
 Student roles and audiences (HO)
 Possible Products and Performances (HO)
Assessor Question 2:
Determine achievement
 What specific characteristics in student responses,
products, or performances should we examine to
determine the extent to which the desired results were
achieved?
 Criteria, rubrics, and exemplars are needed
Designing Scoring Rubrics
 Rubric: criterion-based scoring guide for evaluating a
product or performance along a continuum.
 Consists of:
 Evaluative Criteria – qualities that must be met for work
to measure up to a standard
 Fixed Measurement Scale – often 4 or 5 levels
 Indicators – descriptive terms for differentiating among
degrees of understanding, proficiency, or quality
Rubric Types
 Holistic – provide an overall impression of the
elements of quality and performance levels in a
student’s work
 Analytic – divides a student’s performance into two or
more distinct dimensions (criteria) and judges each
separately
 Recommend use of analytic with a minimum of:
 Criteria for understanding (HO)
 Criteria for performance
 Using Facet-Related Criteria (Figure 8.3, Pg 178)
Rubric Types
 Generic – general criteria in given performance area



Can be developed before specific task defined
Example: General Problem Solving Rubric
Example: Generic Rubric for Understanding (HO)
 Task-Specific – designed for use with particular
assessment activity

Task dependent so cannot be used to evaluate related
performance tasks
Rubric Types
 Longitudinal Rubric – progression from naïve to
sophisticated understanding



Increased understanding of complex functions and
interrelatedness of concepts
Greater awareness of how discipline operates
Greater personal control over and flexibility with knowledge
Effective Rubrics
 Relate specific task requirements to more general
performance goals
 Discriminate among different degrees of
understanding or proficiency according to
significant features
 Do not combine independent criteria in one
column of rubric
 Use Student Anchors to (Anchor design, Pg 181)
 Set standards based on student artifacts
 Consistency in judgment of student work
 Equip students to do more accurate and productive self-
assessment
Effective Rubrics
 All potential performances should fit somewhere
in rubric
 Rely on descriptive language (what quality looks
like) not comparative or value language to make
distinctions
 Avoid making lowest score point sound bad,
should describe novice or ineffective performance
 Highlight judging performance’s impact as
opposed to over rewarding just process or effort
Assessor Question 3:
Valid and Reliable
 Does the proposed evidence enable us to infer a
student’s knowledge, skill, or understanding?
 Validity: did we measure what we meant to measure
 Does the evidence indicate understanding of the
expressed outcomes?
 Are the performances appropriate to the understanding
sought?
 Do not pay so much attention to correctness that degree
of understanding is lost.
Validity
 Two key validity questions for assessment tasks:
 A student could do well on this performance task, but
really not demonstrate the understanding you are
after?
 A student could perform poorly on this task, but still
have significant understanding of the ideas and show
them in other ways?
 Activity: determining validity (Figure 8.5)
Validity
 Two key validity questions for rubric:
 Could the proposed criteria be met but the performer
still not demonstrate deep understanding?
 Could the proposed criteria not be met but the
performer nonetheless still show understanding?
Reliability
 Reliable assessments reveal a credible pattern, a clear
trend
 Need for multiple evidence (scrapbook) rather than
just a snapshot of student performance
 Have parallel assessments on the same concept using
multiple assessment formats.
Dr. Robert Mayes
University of Wyoming
Science and Mathematics
Teaching Center
[email protected]