Transcript Tips for Effective Rubric Design

Tips For Effective Rubric Design
How to:
design a rubric that does its job
write precise criteria and descriptors
make your rubric student-friendly
Expert Input
Experts agree:
Rubrics are hard to design.
– Rubrics are time-consuming to design.
– “A rubric is only as useful as it is good. Using a bad
rubric is a waste of time…”
--Michael Simkins in “Designing Great Rubrics”
Experts disagree:
– how to design a “good” rubric
–
Bottom line: Is it working for you and for your students?
The Cookie
Task: Make a chocolate chip cookie that I would want
to eat.
Criteria: Texture, Taste, Number of Chocolate Chips,
Richness
Range of performance:
–
–
–
–
Delicious(14-16 pts)
Tasty(11-13 pts)
Edible(8-10 pts)
Not yet edible(0-7 pts)
The Rubric
Delicious
4
Tasty
3
Edible
2
Not yet
edible
1
# chips
Chips in
every bite
75% chips
50% chips
Less than
50% chips
texture
Consistently
chewy
Chewy
middle,
crispy
edges
Crunchy
Like a dog
biscuit
color
Even golden
brown
Brown with
pale center
All brown
Or all pale
Burned
richness
Buttery, high
fat
Medium fat
Low-fat
flavor
Nonfat
flavor
Assess The Cookie

Overall score
–
–
–
–

Delicious
Tasty
Edible
Not yet edible
By criteria
–
–
–
–
Number of chips
Texture
Taste
Richness
Oops, What Went Wrong?



Did the “product” match
expectations?
Effective rubrics don’t
exist in a vacuum.
The good news…
Holistic Or Analytic—Which To
Use?
HOLISTIC—views product or performance as a whole;
describes characteristics of different levels of
performance. Criteria are summarized for each
score level.
(level=degree of success—e.g., 4,3,2,1 or “Tasty”)
(criteria= what counts, facets of performance—e.g.,
research or number of chips or presentation)
Holistic Or Analytic?
HOLISTIC—pros and cons
+Takes less time to create. Well…
+Effectively determines a “not fully developed”
performance as a whole
+Efficient for large group scoring; less time to assess
- Not diagnostic
- Student may exhibit traits at two or more levels at the
same time.
Holistic Example
Cookie
Delicious level (4)

Chips in every bite

Consistently chewy

Even golden brown

Buttery, high fat
Holistic Or Analytic?
Analytic=Separate facets of performance
are defined, independently valued, and
scored.
Example: Music—skill=string improvisation
development
Facets scored separately: melody; harmonics;
rhythm; bowing & backup; confidence
Holistic Or Analytic?
Analytic—pros and cons
+Sharper focus on target
+Specific feedback (matrix)
+Instructional emphasis
-Time consuming to articulate components and
to find language clear enough to define
performance levels effectively
Sample Of Analytic Rubric
See Packet
The Debate

Is the whole the sum of its parts?
Wiggle room or valid criterion—
Overall Development
Overall Impression
Overall impact (See “purpose”)
Thomas Newkirk
Weighting
Number range (70-74 or 70-79)
Tip #1

Don’t make task-specific rubrics.
–
–
–
–
–
Efficiency issue
Two heads or three or four or five…
Make one, get two or three or four…
“Generalizable” or template rubric
Unless you need it for tomorrow…
(See Tip #8)
Tip #2

Don’t use generic or “canned” rubrics without
careful consideration of their quality and
appropriateness for your project.


These are your students, not someone else’s.
Your students have received your instruction.
Tip #3

Avoid dysfunctional detail.
–
“…in most instances, lengthy rubrics probably
can be reduced to succinct…more useful versions
for classroom instruction. Such abbreviated
rubrics can still capture the key evaluative criteria
needed to judge students’ responses. Lengthy
rubrics, in contrast, will gather dust” (Benjamin
23).
--Includes wordiness, jargon, negativity
Tip #4

Limit the number of criteria
–
–
Well…
Don’t combine independent criteria.

“very clear” and “very organized” (may be clear but not
organized or vice versa).
Tips #5 and #6

Use key, teachable “criteria” (What counts)
–
–
Don’t vaguely define levels of quality.
Concrete versus abstract


“poorly organized” (Organization: sharply focused thesis,
topic sentences clearly connected to thesis, logical
ordering of paragraphs, conclusion ends with clincher)
“inventive” “creative” “imaginative” UNLESS…
Key Question to ask yourself: What does it look like?
Tips #5 and #6

Use measurable criteria.
--Specify what quality or absence looks like
vs. comparatives (“not as thorough as”)
or value language (“excellent content”)
---Highlight the impact of the performance
--Was the paper persuasive or problem solved?
(Note importance of PURPOSE)
--What are the traits of effective persuasion?
--Be sure that the descriptor is not the criterion and vice
versa
Tip #7

Aim for an even number of levels
–
–
–
Create continuum between least and most
Define poles and work inward
List skills and traits consistently across levels
Tip #8


Include students in creating or adapting
rubrics
Consider using “I” in the descriptors


I followed precisely—consistently—inconsistently—MLA
documentation format.
I did not follow MLA documentation format.
Tip #9



Motivate students to use rubric.
Instructional rubric (“Buy one, get one…”)
“At their very best, rubrics are also
teaching tools that support student
learning…” (Andrade 13).
Do they understand the criteria and descriptors?
How do you know?
When do you give the rubric to your students?
Tip #10

Provide models of the different performance
levels.
The Assignment Sheet

Don’t forget the importance of the assignment sheet
 Connection to rubric (Use same language!)
– The lawyers in your class
“But the rubric doesn’t say that…”
Project/paper/presentation must meet all requirements of
assignment
– Due date and late penalty
– Format requirements
– Non-negotiables
 Skills and reasonable expectations
Don’t Forget the Check-in Stage

Use your rubric as a formative assessment to
give students feedback about how they are
doing.
–
–
–
Isolate a particularly challenging aspect
Have student isolate an area of difficulty
Center revision instruction around rubric
Steps in Developing a Rubric







Design backwards—rubric first; then product/performance.
Decide on the criteria for the product or performance to be
assessed.
Write a definition or make a list of concrete descriptors—
identifiable-- for each criterion.
Develop a continuum for describing the range of performance
for each criterion.
Keep track of strengths and weaknesses of rubric as you use it
to assess student work.
Revise accordingly.
Step back; ask yourself, “What didn’t I make clear
instructionally?” The weakness may not be the rubric.
Steps in Modifying a “Canned”
Rubric


Find a rubric that most closely matches your
performance task.
Evaluate and adjust to reflect your
instruction, language, expectations, content,
students
–
–
–
Criteria
Descriptors
Performance levels
It’s hard work…

Expect to revise…and revise…
–
One problem is that the rubric must cover all potential
performances; each should fit somewhere on the rubric.

“There are no final versions, only drafts and
deadlines.”

When you’ve got a good one, SHARE IT!
When to use these rubrics

Usually with a relatively complex assignment,
such as a long-term project, and essay, or
research-based product.
–
–
Informative feedback about work in progress
Detailed evaluations of final projects
The Mini-Rubric

These are the quick ones.
Fewer criteria and shorter descriptions of quality
–
–
–
Yes/no checklists
Describe proficient level of quality and leave other boxes for
commentary during grading.
Use for small products or processes:




Poster
Outline
Journal entry
Class activity
Mini-rubric Example
Vocabulary Poster
Purpose: to inform
Content criterion (50%) 4
3
2
1
____written explanation of denotation—accuracy/thoroughness
____examples in action—accuracy/variety
____visual symbol or cartoon conveys word meaning—
accuracy/clarity
____wordplay---weighs synonyms for subtleties of meaning-accuracy/thoroughness
Presentation criterion (50%)
4,3,2,1--neat
4,3,2,1--clear organizational pattern
4,3,2,1--no error in Conventions
4,3,2,1--uses visual space to catch and hold attention
Miscellaneous Suggestions
#1--Describe proficient level of quality and leave other
boxes for commentary during grading.
#2--“Box” the acceptable—proficient—level
#3--Translate the rubric’s 4,3,2,1 into number that
represents middle of grade range (e.g., B=84)
OR, give a point range (e.g., A=90 (indicates just made
category)
BUT A=95 (indicates solid in category
Caution
Don’t let the rubric stand alone:
ALWAYS, ALWAYS provide specific
“Comments” on your rubric and/or on the
student product itself.
Sentence Stems
To establish 4 levels of performance, try sentence stems.
Example:




Yes, I used surface texture and deep carvings
effectively to create individualizing detail.
Yes, I used surface texture and deep carvings, but I
needed to include more for individualizing detail.
No, I did not use surface texture, but I did use deep
carvings –or vice, versa—to create some
individualizing detail.
No, I did not use surface texture or deep carvings.
Rubric Criterion Across The
Curriculum

Content (substance, support, proof, details)
–
–
–
–
–
–
–
Relevant
Specific
Thorough
Synthesized
Balanced
Convincing
Accurate
Rubric Criterion Across the
Curriculum

Research
–
Uses variety of sources (primary, secondary,
electronic, traditional, human)

–
–
Note: Watch minimums—Is minimum “minimal” or is
minimum “proficient”?
Uses appropriate sources (credible, timely,
scholarly)
Documents sources accurately
The Best Rubrics




Analytic and holistic
Developmental
“Generalizable” and specific
Instructional
The best rubrics WORK
for students and teachers!