Transcript Designing Rubrics - Qatar University

Designing Rubrics
Nancy Allen, Ph.D.
College of Education
Office of Faculty Development
Qatar University
Adapted from: Baggio, C. (n.d.). Tips for designing rubrics. Retrieved on May 29, 2007, from
www.sdst.org/shs/library/powerpoint/rubrics.ppt and
Designing Rubrics
Students as Self Assessors
Teachers as Focused Coaches
What is a rubric?
A rubric is a guideline for rating student performance.
Benefits:



The rubric provides those doing the assessment with exactly
the characteristics for each level of performance on which
they should base their judgment.
The rubric provides those who have been assessed with
clear information about how well they performed.
The rubric also provides those who have been assessed with
a clear indication of what they need to accomplish in the
future to better their performance.
Asmus, E, (1999). Rubrics. Retrieved on May 29, 2007, from
http://www.music.miami.edu/assessment/rubrics.html
What is a rubric?
Quality Continuum
A rubric must define the range of possible
performance levels. Within this range are different
levels of performance which are organized from the
lowest level to the highest level of
performance. Usually, a scale of possible points is
associated with the continuum where the highest
level receives the greatest number of points and the
lowest level of performance receives the fewest
points.
What is a rubric?
A
A
A
A
A
A
rubric is a lesson in quality
public declaration of expectations
communication tool
self-assessment tool for learners
gauge for examining performance
self-fulfilling prophecy
What is a rubric?
Quality
Continuum
PERFORMANCE
LEVEL
POINTS
POINTS
Excellent
4
90-100
Good
3
80-89
Satisfactory
2
70-79
Needs
Improvement
Clearly
unsatisfactory
1
60-69
0
<60
Rubric vs. Checklist
Checklist for a friendly letter
______ Date, flush left at top
______ Address
______ Greeting
______ Body
______ Salutation
______ Signature
Rubric vs. Checklist
Checklists have not judgment of quality.
Checklists can only be used when
“present or absent” is a sufficient
criterion for quality.
Rubric vs. Checklist
Rubrics include descriptors for each
targeted criterion.
Rubrics provide a scale which
differentiates among the descriptors.
What is a rubric?
Descriptors
Each level of performance should have descriptors
which clearly indicate what is necessary to achieve
that level of performance.
Example
Organization of Thought (4-points): “Work is clearly
organized and includes a diagram or step-by-step
analysis.”
criterion
point value
descriptor
The parts of a rubric:
Rubric
Standards of Excellence
Criteria
Indicators
Determining Standards of
Excellence
How many degrees of quality should
you include?
Should you use language or numbers?
If language, what descriptive terms
should you use?
Should you weigh the items?
Criteria
The specific areas for assessment
Focus areas for instruction
Clear and relevant
Age appropriate
Form and function represented
Objectives
Indicators
Descriptors of level of performance for
the criteria
Conclusion includes whether the findings
supported the hypothesis, possible sources of
error, and what was learned from the experiment.
Clear, observable language
Examples for learners
How do rubrics alter
instruction?
The teacher commits to teaching quality.
The teacher commits to assisting the student
self-assess.
The focus is on each product and/or
performance.
The labels are removed from students.
Specificity appears in all communications.
Everyone gives and receives feedback.
Whom does a rubric assist?
It is a feedback system for students to judge a
product or performance.
It is a feedback tool for teachers to provide clear,
focused coaching to the learner.
It is a system that promotes consistent and
meaningful feedback over time.
It is a communication tool for parents.
What makes a quality
RUBRIC?
An even number of
standards of
excellence
Clear essential
criteria
Realistic number of
criteria
Explicit, observable
indicators
If points… clear to
students upfront
Deliberate sequence
of criteria
High interjudge
reliability
Tested out with
students
What makes a good judge?
Knowledge and
experience with
specific skill
Practice with rubri.
Objectivity
Questions rubric in
advance to be sure
all participants
understand
How do I get started?
Critique current
models.
Ask students to
define “quality” in
relation to specific
product or
performance.
Translate into a
modest rubric.
Expert Input
Experts agree:
Rubrics are hard to design.
 Rubrics are time-consuming to design.
 “A rubric is only as useful as it is good. Using
a bad rubric is a waste of time…”
--Michael Simkins in “Designing Great Rubrics”
Experts disagree:
 How to design a “good” rubric

Bottom line: Is it working for you and for your
students?
Holistic Or Analytic—Which To
Use?
HOLISTIC
Views product or performance as a whole;
describes characteristics of different levels of
performance. Criteria are summarized for
each score level.
Holistic Or Analytic—Which To
Use?
Excellent Researcher




included 10-12 sources
no apparent historical inaccuracies
can easily tell which sources information was drawn from
all relevant information is included
2 - Good Researcher




included 5-9 sources
few historical inaccuracies
can tell with difficulty where information came from
bibliography contains most relevant information
1 - Poor Researcher




included 1-4 sources
lots of historical inaccuracies
cannot tell from which source information came
bibliography contains very little information
Holistic Or Analytic?
HOLISTIC—pros and cons
+ Takes less time to create.
+ Effectively determines a “not fully developed”
performance as a whole
+ Efficient for large group scoring; less time to
assess
- Not diagnostic
- Student may exhibit traits at two or more
levels at the same time.
Holistic Or Analytic?
Analytic
Separate facets of performance are
defined, independently valued, and
scored. Facets scored separately
Holistic Or Analytic?
Analytic—pros and cons
+ Sharper focus on target
+ Specific feedback (matrix)
+ Instructional emphasis
- Time consuming
- Takes skill and practice
Task specific or general?
Task specific: Rubric designed for
and references a specific
assignment.
General: Rubric designed for and
references a type of assignment
frequently repeated.
Tip #1
Use as many generalized rubrics as
possible.


Efficient
Builds recognition of excellence
Tip #2
If using pre-designed rubrics carefully
consider quality and appropriateness for
your project.
Tip #3
Aim for concise, clear, jargon-free
language
“…in most instances, lengthy rubrics probably can
be reduced to succinct…more useful versions for
classroom instruction. Such abbreviated rubrics
can still capture the key evaluative criteria needed
to judge students’ responses. Lengthy rubrics, in
contrast, will gather dust” (Benjamin 23).
Tip #4
Limit the number of criteria, but
Separate key criteria.
 “Very clear” and “very organized” may be clear
but not organized or vice versa.
Tip #5
Use key, teachable criteria.
Key Questions: What are my objectives?
Are there other generalized objectives
that should be included?
Tip #6
Use concrete versus abstract and
positives rather than negatives
 Instead of “poorly organized” use “sharply
focused thesis, topic sentences clearly
connected to thesis, logical ordering of
paragraphs, conclusion ends with clincher”.
Key Question to ask yourself: Would
student know what quality “looked like”
by this description?
Tip #7
Use measurable criteria.

“Includes two or more new ideas…” instead
of “creative and imaginative”
Tip #8
Aim for an even number of levels



Create continuum between least and most
Define poles and work inward
List skills and traits consistently across
levels
Tip #9
Include students in creating or adapting
rubrics
Consider using “I” in the descriptors
 I followed precisely—consistently—
inconsistently—MLA documentation format.
 I did not follow MLA documentation format.
Tip #10
Provide models of the different
performance levels.
The Assignment Sheet
Link the assignment sheet and the rubric. Use
same language.
Include all non-negotiable items.
 On time
 Formatted correctly
 Follows standard conventions…
 Etc.
Rubrics for formative
assessment
Encourage students to “check progress”
using the rubric.
Encourage / require self-assessment
and/or peer assessment.
Steps in Developing a Rubric
Design backwards—rubric first; then product/performance.
Decide on the criteria for the product or performance to be
assessed.
Write a definition or make a list of concrete descriptors—
identifiable-- for each criterion.
Develop a continuum for describing the range of performance
for each criterion.
Keep track of strengths and weaknesses of rubric as you use it
to assess student work.
Revise accordingly.
Step back; ask yourself, “What didn’t I make clear
instructionally?” The weakness may not be the rubric.
Rubrics On Line
"Rubistar Rubric Generator" (http://rubistar.4teachers.org/)
"Teacher Rubric Maker" (http://www.teachnology.com/web_tools/rubrics/)
“Rubrician” (http://www.rubrician.com/language.htm”
Rubrics for Web Lessons
(http://edweb.sdsu.edu/triton/july/rubrics/Rubrics_for_Web_Les
sons.html)
An Online Rubric Maker (http://landmarkproject.com/classweb/rubrics/)
References
Andrade, H.(2000). Using rubrics to promote thinking and learning. Alexandria,
VA: ASCD.
Asmus, E, (1999). Rubrics. Retrieved on May 29, 2007, from
http://www.music.miami.edu/assessment/rubrics.html
Baggio, C. Designing rubrics: Revising instruction and improving performance.
Retrieved on March 1, 2007, from http://www.edutech.org.br.
Baggio, C. (n.d.). Tips for designing rubrics. Retrieved on May 29, 2007, from
www.sdst.org/shs/library/powerpoint/rubrics.ppt
Benjamin, A.(2000). An English teacher’s guide to performance tasks and
rubrics. Larchmont: Eye on Education.
Leavell, A. (n.d.). Authentic assessment: Using rubrics to evaluate project-based
learning. WEBLIBRARY.
Matthews, J. (2000). Writing by the rules no easy task. Retrieved on October
25, 2000 from <http://washingtonpost.com/wp-dyn/articles/A635992000Oct23.html>
Simkins, M. (1999, August). Designing great rubrics. Technology and Learning.
Wiggins, G. & McTighe, J. (1998). Tips for developing effective rubrics.
Understanding by Design. Alexandria, VA: ASCD.