Transcript Good Rubric, Bad Rubric What’s the Difference?

Good Rubric, Bad Rubric
What’s the Difference?
Kathleen Norris, MFA, EdD
Plymouth State University
Good, or Emerging?
Rubrics are a phenomenon of the profession of
education.
They have been embraced by some educators,
tolerated by others, and loathed by some.
Some educators are afraid of them, or don’t
understand them.
Good Rubrics
 Do what you want them to do, assess what
you want to assess.
 Are pleasing to look at, are easy on the
eyes, are clear to everyone, are not too big
or small for their purpose.
 Are reliable, valid, fair, and completely
connected to what you are assessing.
 Everybody understands the same thing
when they read one, but they can still be
scary.
Emerging Rubrics
 Assess something, but not necessarily what
you want them to assess.
 Could be clearer, easier on the eyes; might
need to be smaller or larger.
 Are not completely connected to the
standards, the performance (task), or both,
so are not reliable, valid, or fair.
 Not everyone understands them the same
way and they can be very scary.
Authentic Assessment
“A move toward more authentic tasks and
outcomes thus improves teaching and
learning: students have greater clarity about
their obligations (and are asked to master
more engaging tasks), and teachers can
come to believe that assessment results are
both meaningful and useful for improving
instruction.”
Wiggins, Grant (1990). The case for authentic assessment. Practical Assessment, Research &
Evaluation, 2(2). Retrieved January 22, 2007 from
http://PAREonline.net/getvn.asp?v=2&n=2.
Good Rubrics
 Assess the performance of a student in
the completion of a task.
 Complement, but do not completely
replace, traditional assessments.
 Communicate expectations clearly.
 Communicate the level of success a
student is having.
 Inform future instruction.
Rubric Development
 Questions that need to be answered,
and should be answered
collaboratively:
What should the students know and be
able to do? (These are the standards.)
What products indicate that students
have met these standards? (Describe the
authentic task(s) or project(s) that will be
scored.)
Rubric Development
 What does “perfect” performance on these
tasks, or a “perfect” project look like, and what
does an unsuccessful project look like? (These
are the top and lowest score criteria for
judgment.)
 To communicate the expectations for the task,
and, in the end, how well students performed, a
rubric is developed that contains
 the elements of the task or project that meet the
established standards,
 the criteria for the evaluation of each element, and
 ultimately the student’s “score” on each element.
Good Rubric Development
 Collaboration is essential in this work.
 Define the standards to be met.
 Describe the task(s) or project(s) to be
completed.
 Describe the criteria for judging the task or
project on each required element.
 Set the scale from “perfect” to unsuccessful
on each required element.
Sample Writing Rubric
Essential
Elements
Purpose
Development
Details
Organization
ADVANCED(4)
PROFICIENT(3)
BASIC(2)
NP(1)
Purpose is
clearly
expressed
Limited expression
of purpose
Unclear purpose
Lacks purpose
Good
development of
thesis, theme, or
topic; conclusion
is more than a
summary
Rudimentary
development of
thesis, theme, or
topic; limited in
depth or clarity
Undeveloped or
vague thesis,
theme, or topic
Inappropriate
response to thesis,
or lacks thesis; no
topic development
Details are
adequate &
relevant
Details lack
elaboration;
important details
omitted
Few relevant
details
Virtually no
relevant details
Logical
progression of
ideas
Organizational
strategy includes
transitions
Unclear
organizational
strategy
Lacks organization
Designing the Rubric
 Creating a table with scores across the top
and criteria down the side and language
within each box is easy, but you might be
creating an “emerging dog” instead of your
desired, and good, “dog.”
 The language in each of the boxes is the key
to a “good rubric.”
 Rubrics that are “emerging” can be
rehabilitated and become “good” ones!
Language Within a Rubric
 Each “box” will have a description of
what that score value means relative to
the criteria.
 The descriptions will be written in
language that is appropriate for the
grade level of the students and that
avoids any jargon or language that
would be unclear to them or their
parents.
Language Within a Rubric
 The language will be agreed upon by the
developers of the rubric, at a minimum. Best
practice would be to have at least a sample
of the students and parents involved in
revising a draft of the rubric to raise any
questions of clarity about standards, criteria
or the task, and to assist in revising the
language before it becomes the “final” draft.
Testing a Rubric
 The “final draft” of a new rubric should
be “tested” by having a sample of
tasks provided to readers for scoring
on the rubric.
 Ideally, teachers, students and a
sample of parents would be involved in
this, but at a minimum the teachers
would have collaborated on this rubric.
Validity and Reliability
 The rubric can be considered valid if the
different readers assign the same score
values for each sample,
 and it can be considered reliable if a set of
“perfect” projects are all given “perfect”
scores by the various readers, and a set of
unacceptable projects also get unacceptable
scores by the readers, etc.
Fairness
 The task, or project, assigned must be
designed as an authentic assessment
of how well students have achieved the
standards.
 The students should be given the rubric
with the assignment, and should be
taught how to use the rubric to create
their best work.
Fairness
 The rubric must be applied equally to
all tasks or projects; the identity of the
student should not affect the scoring of
the project.
 The connection of the rubric to a final
“grade” on the assignment must be
made clear from the beginning.
FAQ’s
 Choose to run scores from left to right, or
right to left? How big should the matrix be?
 We naturally read English documents from left to
right. Some suggest starting on the left with the
“perfect” score column so students see that first,
but this can feel awkward to traditional learners.
 The matrix should contain no more than 6-7
score points and no fewer than 3. Five seems to
be enough for clarity for most rubrics. You can
put +/- signs in between the columns if you need
finer discrimination in scoring.
 The number of criteria should be the minimum
necessary to assess the project or task.
FAQ’s
 Can a good rubric be created by a person all on their
own? What about all the “rubric maker” software
programs?
 The best rubrics are developed and tested collaboratively.
 If there are predefined standards and established criteria,
an individual can create a good rubric. There are many
free online rubric creating programs as well as websites
and books with sample rubrics that can be adjusted.
 The best rubrics are tested for validity and reliability and
this requires collaboration. Authentic assessment in
general will be used more effectively to inform instruction
when teachers collaborate on the testing of rubrics.
FAQ’s
 If you don’t have a collection of samples to use to
test the rubric, how can you be sure that it’s reliable
and valid?
 You really can’t be sure.
 If you have a “new” task or project then the rubric is being
“tested” with it as the students complete the project or
task. The “untested” rubric will have to be handed out with
the assignment, but even at that stage it can be improved
based on student feedback.
 The particular challenge with reliability and validity comes
with getting inter-rater agreement on the scoring and you
need to have samples and more than one reader to
establish the “goodness” of the rubric.
FAQ’s
 Should the points given “add up” to a grade, or be averaged
for a grade?
 This really depends on the task or project and what the grade is
meant to represent. If the task itself is getting a grade, then the
rubric can be scored to produce a grade. However, many tasks
are part of a larger graded unit, so there might be a rubric
created for the group of assignments that all work together to
produce a grade.
 A writing assignment, for example, can be scored on a rubric for
the quality of the writing, but there may be a number of elements
to the written piece that must be included in order for the paper to
be successful at meeting content standards that are outside of
actual writing standards.
Rubrics
 They are an integral part of the authentic
assessment of worthy intellectual tasks and
projects.
 They can be used effectively to
communicate with students and parents
about what’s expected and how well a
student is meeting those expectations.
 They can inform instruction more effectively
than traditional assessments.
Good Rubrics
 They can be time consuming to produce, but
valid, reliable and fair rubrics will last for as
long as the standards are assessed by a
task or project. The criteria are based on the
standards, so they can be adjusted, if
necessary, for different authentic
assignments, without being rewritten.
 They need to be introduced systematically
for the educators, students and parents who
may be “afraid” of them. Don’t let them
“jump up.”
Student Voices; What’s Best
 "In the end they help make things fair so that a
student can't complain about his grade, and it also
gives the teacher back up for the grade he/she gave.
A student can also use it for them if they feel they
have been unfairly graded.”
 "I think rubrics can help teachers grade better than
just thinking of a grade in their heads.”
 "I like the way they word the scores and also how they
show you to get that score. The guidelines are right
down to the point and are precise to what you need to
get that score.
Transforming Middle Schools and School Systems: What students say about rubrics.
http://www.middleweb.com/CSLB2CubRub.html
Thank You
As you implement best practices in
Curriculum, Instruction and
Assessment, you’ll find that the wise
use of good rubrics will contribute
greatly to the improvement of
instruction and measurable gains in
student learning. Take the time you
need to build good rubrics!