Transcript Using Rubrics to Evaluate Assessments

Using Rubrics to
Evaluate Assessments
Kim Anderson
Course Evaluation Subcommittee Chair
Summer 2009
What is a Rubric?
A rubric is a continuum of scoring
categories and guidelines for teachers
to evaluate and to give feedback to
individual students
AND
to evaluate groups of student work to
assess for course effectiveness.
Uses of a Rubric
• Within a class
– Clarity of assignment
– Defined standards
– Contribute to course grade
• Outside of the course (Assessment of SLOs)
– Faculty agreement of essentials = double duty
– Facilitates discussion
– Improve course’s effectiveness and student
learning
Benefits of a Rubric
Instructor
• Objective, clear, consistent evaluation tool
• Evaluates students’ work for course grade
• Use with a variety of assignments
AND
• Evaluates grouped student work enabling
teachers to gather data to evaluate the
effectiveness of a course within a program
and thus improve student learning
(SLO assessment)
Benefits of a Rubric
Students
Roadmap for excellence on
assignments
Reveals “what counts”, “what is
important”
Clear sense of what is to be achieved
Understand the grades that are earned
Standards of the profession/discipline
Essentials of a Good Rubric
• Criteria must align with the SLO by identify
the various characteristics of the outcome.
• Scoring categories specify various levels of
achievement in each characteristic.
• Assessment task should mirror real-life
challenges as appropriate.
• Two independent raters will arrive at a similar
score for a given response based on the
scoring rubric.
Components of a Rubric
• Assignment = task components of a
product or performance/clear definitions
of each characteristic to be assessed
• Evaluation = criteria standards or
competencies
• Levels of Achievement = scale of
success by letter grades, numbers, or
narrative
• Documentation
Assignment = task
• Product
– Essay
– Portfolio
– Final project
– Shop product
– Document/Map
• Performance
– Speech
– Dance/Act
– Play an
instrument/Sing
– Skill activity
– Lab procedure
Evaluation = criteria
• Elements of the assignment
• Evaluation criteria & standards
– Demonstration of requirements
– Evidence of specific progress
– Feedback on competencies
– Miss, meet or exceed standards
Achievement = scale
• Letter Grade
A, B, C, D, F
• Numbers
1-3 (simple) ; 1-4; 1-5 (detailed)
• Narrative
“beginning, developing, competent,
exemplary”; many versions
Documentation = samples
•
•
•
•
•
Product
Performance
Letter grade
Number scale
Narrative explanations
“Product” Excerpt
Excel Spreadsheet Final Project
Below Expectation
1
Satisfactory
2
Exemplary
3
Requirements
Document Sheet
Requirements
document is
incomplete.
Many sections are
missing. Descriptions
are missing or not
written thoroughly.
Requirements
document is not
complete.
Includes most of the
assigned sections.
Missing 2 or more
sections or lacks
accurate descriptions.
Requirements
document is complete.
Includes all assigned
sections in addition to
thorough and accurate
descriptions.
Worksheet Design &
Formatting
Incomplete-missing
more than two:
Missing labels &
values
Title & subtitle
Column headings
Missing title merged
and centered
3 or more spelling
errors
Somewhat completemissing two:
Pertinent labels &
values
Title & subtitle
Column headings
Title merged and
centered
Fewer than 2 spelling
errors
High level of
professionalism
All labels & values
included
Appropriate titles &
subtitles added
Title merged and
centered
No spelling errors
“Performance” Sample
Golf Bunker Shot
Beginning
Developing
Competent
Stance
Closed stance
Posture – bent over,
stiff knees, weight
even
Open stance
Posture – straight
back, slightly bent
knees, weight slightly
forward
Open stance
Posture - straight
back, slightly bent
knees, weight
forward
Ball Position
Ball positioned too far
forward in the stance
Ball positioned too
close or too far to the
golfer
Ball positioned in the
middle of the stance
Ball positioned at a
distance from the
golfer that impacts
the gold posture
Ball positioned
middle to back of the
stance
Ball positioned the
correct distance from
the golfer that allows
good golf posture
Swing Technique
Closed club face
Halting,
shortened/too large
of a swing
Eyes move
Erratic arc of the
swing
Parallel club face
Jerky, but complete
swing
Eyes down
Complete swing, but
off line
Open club face
Smooth, rhythmic
swing
Eyes behind the ball
“Down the target line”
swing
“Grade” Sample
Political Science Essay
•
•
•
•
•
A = The A-range essay implies a strong argument and provides convincing specific support from the various readings.
The writer demonstrates mature command of language through a variety of sentence structures, word choices, quotes
or paraphrases from the readings (consistently cited correctly). Control of usage and mechanics, despite occasional
flaws, contributes to the writer’s ability to communicate the purpose of the paper. The writer thoroughly understands
the concepts/theories involved and through the essay can convince others of his/her viewpoints or helps make the
reader aware of something completely new or original. These are powerful due to organization and creativity.
B = The B-range essay shows effort and promise for the writer. It presents a thesis (argument) and often suggests a
plan of development that is carried out effectively. Mastery of the readings/theories may not be fully indicated with
the use of quotes or paraphrases, but the writer provides enough supporting details, makes competent use of language,
and sometimes varies sentence structure. Occasional errors in usage and mechanics do not interfere with the writer’s
ability to communicate the purpose of the paper.
C= The C-range essay presents a thesis (argument) and often suggests a plan of development which is generally c
arrived out. The writer may or may not have completed all of the required readings and utilizes generalizations or list
for support. Command of the theories under consideration is weak or shaky. Sentence structure tends to be
repetitious, and errors in usage and mechanics sometimes interfere with the writer’s ability to communicate the
purpose of the paper.
D = The D-range may present a thesis (argument); however, the plan of development is usually not carried out. This
indicates the writer may have spent little time with the readings or thinking about the concepts involved. The writer
provides support that tends to be sketchy and/or illogical. Sentence structure is simplistic, repetitious and
occasionally awkward. Language is often inappropriate in tone or style. Errors in usage and mechanics are frequent.
F = This paper presents a thesis that is vaguely worded, weakly asserted or there is no central argument present.
Support, if any, tends to be rambling and superficial. Sentence structure is difficult to follow and errors in usage and
mechanics interfere with the writer’s ability to communicate the purpose of the paper.
“Numeric” Sample
Philosophy - Critical Thinking
4
Consistently does all or almost all of the following:
Accurately interprets evidence, statements, graphics, questions, etc.; Identifies the salient arguments
(reasons and claims) pro and con.; Thoughtfully analyzes and evaluates major alternative points of
view.; Draws warranted, judicious, non-fallacious conclusions.; Justifies key results and procedures,
explains assumptions and reasons.; Fair-mindedly follows where evidence and reasons lead.
3
Does most or many of the following:
Accurately interprets evidence, statements, graphics, questions, etc.; Identifies relevant arguments
(reasons and claims) pro and con.; Offers analyses and evaluations of obvious alternative points of
view.; Draws warranted, non-fallacious conclusions.; Justifies some results or procedures, explains
reasons.; Fair-mindedly follows where evidence and reasons lead.
2
Does most or many of the following:
Misinterprets evidence, statements, graphics, questions, etc.; Fails to identify strong, relevant
counter-arguments.; Ignores or superficially evaluates obvious alternative points of view.; Draws
unwarranted or fallacious conclusions.; Justifies few results or procedures, seldom explains
reasons.; Regardless of the evidence or reasons, maintains or defends views based on self-interest
or preconceptions.
1
Consistently does all or almost all of the following:
Offers biased interpretations of evidence, statements, graphics, questions, information, or the points
of view of others.; Fails to identify or hastily dismisses strong, relevant counter-arguments.; Ignores
or superficially evaluates obvious alternative points of view.; Argues using fallacious or irrelevant
reasons, and unwarranted claims.; Does not justify results or procedures, nor explain reasons.;
Regardless of the evidence or reasons, maintains or defends views based on self-interest or
preconceptions.; Exhibits close-mindedness or hostility to reason.
“Narrative” Excerpt
English 1/ESL 1 Final Essay Examination
Sampling of Narrative Criteria
A. Respond to the writing task.
The paper must respond to the specific topic and the reading and must show understanding of the reading.
A paper simply restating the content of the reading is not an adequate response. The paper must be at
least 400 words long.
B. Treat the reading fairly.
The paper must avoid plagiarism by attributing the words and ideas of others to their source. Borrowed
words and combinations of words must be set off by quotation marks. The paper must not misinterpret or
misquote others’ words.
C. Maintain a consistent style and sense of audience.
The paper must not have abrupt or confusing shifts in person, voice or tense.
E. Be organized logically and coherently.
The paper must be organized according to a discernible plan, with the main points
supporting the thesis. The reader must be able to follow the plan with ease.
I. Adhere to the conventions of Standard English.
The paper should have no more than minimal errors in spelling, punctuation, capitalization, apostrophe,
and case, and should use standard forms of verbs, adjectives, and adverbs.
Recap
Why a Rubric?
• Can be used for evaluation both within a class and for
assessment of SLOs
✓Assists instructors to precisely define course
expectations across all sections
✓Provides faculty with a tool that can be used to
evaluate student work and ultimately to gather data to
evaluate the effectiveness of student learning
• Assists students with what is needed to be successful
in a course
• Can be used for a product or performance
• Has flexibility with its design