Using Rubrics for Assessment: A Primer

Marcel S. Kerr Summer 2007 

Presentation Purpose

   

Educate faculty on uses of rubrics Preparation for General Education Assessment Plan Learn to write rubrics for multiple purposes

   Student learning assessment Course assessment Program assessment

Presentation Objectives

         Define rubric Differentiate types of rubrics Identify uses of rubrics List advantages of using rubrics Construct a rubric Determine validity & reliability End-of-training Quiz Complete Rubric Training Rubric

What are Rubrics?

    A rubric is a scoring scale used to assess performance along a task-specific set of criteria rather than a single numeric score.

A means of formative assessment Purposes of rubrics:    Describe what “outstanding” or “satisfactory” or “inadequate” performance looks like Articulate the criteria against which student learning is judged Provide an objective guide for assessment

What do Rubrics Assess?

  Assessment level:     Student learning Courses Programs Institutions A stated learning objective:  

Content

discipline – the knowledge specific to a given

Performance

– skills and internal processes required for achievement in a given discipline

Types of Rubrics

  

Analytic

– are more specific & easier to apply; break down the objective into dimensions (criteria) and judge them individually  Used when large number of criteria; criteria are weighted

Holistic

– are more general; assess the entire objective on one scale  Used for quick or minor assessments; open-ended questions; skills

Rubric Basic Structure

  There are four basic features:  Stated Objective or Purpose - title    Scoring Criteria – characteristics of good performance on the task; Levels of Performance – defined degrees of competency Descriptors – spell out briefly what is expected at each level of performance

Rubric Basic Structure

 Objective: Research Paper Criteria Number of Sources 1-4

1

Historical Accuracy Organization Use of APA Format Lots of historical inaccuracies Cannot tell from which source information came Lots of APA errors 5-9

2

Few inaccuracies 10-12

3

No apparent inaccuracies Can tell with some difficulty where information came Few APA errors Can easily tell from which sources information was drawn No apparent APA errors

Uses of Rubrics

  Set evaluator & performer expectations  Criteria by which work is judged  Difference between excellent & weak work  Formative student feedback  Grade assignments  Standardize grading across graders  Assess programs (GEC)

Rubrics for Program Assessment

   Rationale for use:     Faculty created, owned, and operated Low cost & reusable National tests are expensive and problematic Natural part of the course or curriculum Four General Education Literacy areas:    Language Literacy – Analytic Literacy – Written Communication Problem Solving Cultural Literacy – Composition of Art  Social Literacy – Personal & Civic Responsibility

Advantages of Using Rubrics

      Clarity  Expectations, objectives, grading, feedback Objectivity  Standardized, consistent, fair, valid, reliable Legitimacy  Fairness increases student responsibility Efficiency  Easy to make, use and explain Improve skills & End Products  Instructor, students, peers

Writing Rubrics

     Identify and define the assessment objective or purpose Select and write the needed number of scoring criteria Select and write the desired levels of performance If desired, select and write the descriptors

Electronic Rubric Builders

 Here are a few:  Teachnology.com

 Rubistar  Rubric Studio

Calibrating Rubrics: Validity

 

Validity

intended?

– the accuracy with which the rubric assesses the objective or purpose; are we evaluating what we     Self Check Colleague Review Student Review Pilot Test

Calibrating Rubrics: Reliability

 

Reliability

– how consistently the rubric assesses the objective or purpose over time and across raters; are the resulting scores consistent?

 

Interrater

– consistency of scores assigned by two or more raters

Intrarater

– consistency of scores assigned by same rater at different points in time   External factors – time of day, order of assignments Internal factors – mood, fatigue, attention

End-of-training Quiz

 What have we learned?

Please complete the 10-item online quiz. It is a quick formative assessment of what we’ve covered today. Upon completion, you will receive a feedback page with your final score. Please print a copy of this page and forward it to your Dean or VP as evidence that you have completed the assessment training.

Rubric Resources

      Documenting Excellence – General Education Rubrics: http://www.documentingexcellence.com/examples/rubricgened /rubric.htm

Authentic Assessment Toolbox: http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm

Rubrics.com: http://www.rubrics.com/ Teacher Created Rubrics for Assessment: http://www.uwstout.edu/soe/profdev/rubrics.shtml

Sinclair Community College – General Education Rubrics: http://www.sinclair.edu/about/gened/genedrubrics/index.cfm

CSU – Institutional research Assessment & Planning: http://www.csufresno.edu/ir/assessment/rubric.shtml

References

    Moskal, B. M., & Leydens, J.A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation, 7(10). Retrieved June 19, 2007 from http://PAREonline.net/getvn.asp?v=7&n=10 Practical Assessment, Research and Evaluation. A Peer-reviewed electronic journal. ISBN 1531-7714. http://pareonline.net

Richards, K. (2005). A primer on rubrics. Retrieved, June 15 th , 2007 from http://online.lsc.mnscu.edu/Academics/assessment/Documents/Primer_on_Rubrics.pdf

Stevens, D.D., & Levi, A.J. (2004). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback and promote student learning. Stylus Publishing.