Transcript Document

Creating Effective Assessment Tools
Anne Marie Karlberg
[email protected]
January 30th, 2009
1
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
2
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
3
Assessment Data
1. Direct indicators (outcomes):
e.g., essays, capstone projects,
demonstrations, presentations
4
Assessment Data
1. Direct indicators (outcomes):
e.g., essays, capstone projects,
demonstrations, presentations
2. Indirect indicators (perceptions):
e.g., surveys, focus groups, interviews
5
Assessment Data
1. Direct indicators (outcomes):
e.g., essays, capstone projects,
demonstrations, presentations
2. Indirect indicators (perceptions):
e.g., surveys, focus groups, interviews
3. Institutional data:
e.g., retention, graduation, enrollment,
transfer trends
6
Assessment Levels
A. Course level
B. Program level
C. College level
7
Examples of Activities at Each Level of NWIC’s Assessment Program
Type of
assessment
information
College
Program
Course
Direct indicators
of student
learning
(outcomes
assessment)
Develop college outcomes
Implement college outcomes
Develop program
outcomes
Implement program
outcomes
Develop
course
outcomes
Implement
course
outcomes
Indirect
indicators of
student learning
(surveys,
interviews)
Graduate survey
Faculty peer review
Student opinion survey
Alumni survey
Graduate survey
Student exit survey
Course
evaluations
Institutional
data (rates and
numbers)
Track rates and numbers…
Student enrollment
Graduation
Retention
Transfer
Performance after transfer
Track rates &
numbers…
Student enrollment
Retention
Graduation
Track rates &
numbers…
Student
enrollment
Course
8
completion
Examples of Activities at Each Level of a Tribal College Assessment Program
Type of
assessment
information
College
Program
Course
Direct indicators
of student
learning
(outcomes
assessment)
Develop college outcomes
Implement college outcomes
Develop program
outcomes
Implement program
outcomes
Develop
course
outcomes
Implement
course
outcomes
Indirect
indicators of
student learning
(surveys,
interviews)
Graduate survey
Faculty peer review
Student opinion survey
Alumni survey
Graduate survey
Student exit survey
Course
evaluations
Institutional
data (rates and
numbers)
Track rates and numbers…
Student enrollment
Graduation
Retention
Transfer
Performance after transfer
Track rates &
numbers…
Student enrollment
Retention
Graduation
Track rates &
numbers…
Student
enrollment
Course
9
completion
History of course outcomes at NWIC
Beginning in 2003, faculty started
consulting with one another and developed
consistent outcomes for many courses –
regardless of where or how the courses
were being taught (i.e., course outcomes
forms).
10
History of course outcomes at NWIC
Each course outcomes form and syllabus
identifies two types of learning outcomes
that will be assessed during the course…
1. NWIC outcomes
2. Course specific outcomes (unique set
of outcomes)
11
History of course outcomes at NWIC
The outcomes are assessed in two ways:
1. Directly: instructors assess students’
work
2. Indirectly: students identify how well
they think they have mastered each
of the outcomes on course evaluation
forms at the end of the quarter
12
History of course outcomes at NWIC
The outcomes are assessed in two ways:
1. Directly: instructors assess students’
work
2. Indirectly: students identify how well
they think they have mastered each
of the outcomes on course evaluation
forms at the end of the quarter
13
Characteristics of
effective course outcomes
1. Out there
Does each statement describe what
we intend students will be able to
do “out there” in life roles as
opposed to an activity “in here”?
2. Active
Does the statement describe what
they can DO with what they know
or understand?
3. Distinguishing Do the statements as a whole clearly
distinguish the levels of intended
performance between courses?
4. Robust
Are the statements “robust”
(complex) enough to drive content
and assessment tasks?
14
Course Outcomes Plan
The following are steps in developing the course outcomes process…
Goal (% of college-level courses)
Baseline % of
198 courses
(Dec. 2007)
2007
2008
2008
2009
2009
2010
2010
2011
2011
2012
%
%
%
%
%
%
a. articulate course outcomes
41
60
75
100
100
100
b. include course outcomes on syllabi
41
60
75
100
100
100
c. develop course level assessment tools (e.g., rubrics)
0
0
15
30
45
60
d. determine or create the instructional activities, experiences,
projects, or assignments that will be used to teach the outcomes
0
0
15
30
45
60
e. determine or create the activities, experiences, projects,
essays, or assignments that will be used to assess the outcomes
0
0
15
30
45
60
f. attach anchor papers for each level of the rubric scale
0
0
15
30
45
60
g. assess students at the beginning and end of the course for
outcomes
0
0
15
30
45
60
h. analyze the assessment data
0
0
15
30
i. use the data to improve and revise curriculum and to improve
learning
0
15
30
45
j. share the results with faculty (occasionally)
0
15
30
45
k. compare the outcomes of different students in the same
course over time
0
15
30
45
l. document the process
0
15
30
45
m. educate students about role of assessment in their education
0

 15 


45
60
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
16
2. Why use an assessment tool?
Makes the criteria for assessment explicit.
Guides the evaluation of students’ work and
makes it less subjective.
Improves the assessment of learning.
Enhances learning.
Assists in targeting important content and
in sustaining consistent attention to
evaluative criteria.
Can be used for self and peer evaluation.
Increases consistency among instructors.
17
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
18
The checklist
Isn’t about quality—it’s just about the parts
that are necessary.
Is a good indicator of “can do–can't do” and
“done–not done”.
Defines the parts that make up a complete
task (either product or process).
Is useful for formative evaluation of a
product or process.
19
Checklists: Use checklists to…
Record observed performance.
For example,
Activities Checklist
Yes
No
Can name five sports.
Yes
No
Can name five activities other than sports.
Yes
No
Can create simple sentences about likes & dislikes
regarding activities & sports.
Yes
No
Can ask a question about what others do in their free time.
20
Use checklists to…
Keep track of progress over time (e.g., as an
inventory of skills at the beginning and end
of a course). For example,
Receptive oral skills
 Understands simple directions.
 Understands simple sentences.
 Understands simple yes/no questions.
 Understands vocabulary appropriate to age.
 Understands meaning of different intonation patterns.
 Understands more complex directions.
 Understands rapid speech.
 Understands language in classroom situation.
 Understands language of peers.
Adapted from Genesee, F. & Upshur, J.A. (1996). Classroom-based evaluation in second-language education. Cambridge: Cambridge University Press,
p. 88.
21
Use checklists to…
Indicate the minimum requirements of a
product or performance, which are not
included in your qualitative criteria (i.e.,
rubrics).
For example,
Essay - Non-negotiables
Yes
No
My paper is typed, double-spaced.
Yes
No
I wrote at least 500 words.
Yes
No
My paper has an introduction, body, and conclusion.
Yes
No
I included examples from the story.
Yes
No
I proofread my paper.
22
Use checklists to…
Help students fulfill task requirements.
For example,
Essay - Proofreading check - Draft 2
Length requirement
Spell-checked
Subject-verb agreement
Noun-adjective
agreement
Verb conjugations
23
An example of a “process” checklist
For working in a team, giving a presentation,
working to solve a problem, etc.
24
An example of a “product” checklist
For example, papers, portfolios, projects,
solutions, etc.
25
Advantages of checklists
• Easy to construct and use.
• Align closely with tasks.
• Effective for self and peer assessment.
• Make learners aware of task
requirements, allowing them to selfmonitor progress.
• Useful for sharing information with
parents and others.
• Provides basic formative feedback.
26
Disadvantages of checklists
• Provide limited information about how to
improve performance or product.
• Do not indicate relative quality of
performance or product.
• Do not provide information for assessing
grades.
27
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
28
Rubrics
• A scoring scale used to assess student
performance along a task-specific set of
criteria
• Comprised of two components: criteria
and levels of performance
• Some rubrics include descriptors:
describe more precisely what
performance looks like at each level
29
Criteria
Level of performance
2. Written Communication Skills: Outcome a. The student writes standard English
Level of Performance
Criteria
Exemplary (4)
Accomplished (3)
Developing (2)
Beginning (1)
1.Idea and
Content
2.
Organization
/ Structure
3.Voice
30
Criteria
Descriptors
Level of performance
2. Written Communication Skills: Outcome a. The student writes standard English
Level of Performance
Criteria
Exemplary (4)
Accomplished (3)
Developing (2)
Beginning (1)
1.Idea and
Content
Writes clearly and with
focus; relevant details
support the central theme.
Maintains clear focus
throughout the paper with
sufficient appropriate
details indicating
awareness, knowledge, and
insight.
Partially focuses on topic
with minimal or no
support of position.
Writing is basic, too
general for the reader to
develop a clear
understanding.
Writes with unclear purpose
or central theme. Does not
clearly define or support
position on topic. Uses
limited or disconnected
details that disrupt the unity
of the paper.
2.
Organization
/ Structure
Provides clear
introduction and
reinforcing conclusion.
Orders writing logically
with effective transitions,
providing sufficient
information in the
appropriate places.
Supports thesis and purpose
through organization and
paragraphing; most
transitions are appropriate,
but sequence of ideas may
need improvement.
Reiterates introductory
elements in conclusion.
Writes with some signs of
logical organization; may
include abrupt or illogical
shifts and ineffective flow
of ideas. Makes few
transitions between ideas.
Writes with organization
that is unclear or
inappropriate to the thesis;
lacks transitions between
ideas.
3.Voice
Writes expressing own
personality, with
confidence and feeling.
Individual, powerful
commitment to the topic is
obvious, as are strong
connections to the
audience and to the
purpose; evokes strong
emotion in the reader.
Writes so that own
personality pokes through;
confidence and feeling fade
in and out. Commitment to
the topic is apparent, and
connection to the audience
and to the topic are
appropriate. The writing
evokes some emotion in the
reader.
Writes without revealing
own personality; writing
is cautious. Commitment
to topic, and connection
to the audience and to the
purpose are limited.
Writing evokes limited
emotion in reader.
Writes without personality.
Shows lack of commitment
to topic, connection to the
audience and to the
purpose. Evokes no emotion
in reader.
31
Possible terms for level of performance
4
3
2
1
Exemplary
Accomplished
Developing
Beginning
Professional
Experienced
Developing
Novice
Frequently
Sometimes
Rarely
Never
Distinguished Proficient
Apprentice
Novice
Exemplary
Excellent
Acceptable
Unacceptable
Exceeds
expectations
Meets
expectations
Progressing
Beginning
Superior
Good
Fair
Needs work
32
Adjectives and Adverbs for Descriptors
Exemplary
Proficient
Progressing
Beginning / Not
Meeting Expectations
All
More than ___ (or most)
Fewer than ___ (or some)
None (or very few)
Always
Often, usually
Seldom, rarely, sometimes
Never, not at all
Complete
Somewhat complete
Less than complete
Incomplete
Superior
Adequate
Less than adequate
Inadequate
Maximum
Satisfactory
Minimal
Unsatisfactory
Articulate
Understandable
Vague
Unclear
Clear, accurate
Often clear, often accurate
Sometimes unclear/inaccurate
Rarely clear
…to the highest level
…to an acceptable level
…to a minimal level
…to an unacceptable level
Includes all elements of…
Includes most elements of…
Includes few elements of…
Includes no elements of…
Clear
Somewhat proper
Sometimes improper
Improper
Proper
Some degree of clarity
Somewhat unclear
Unclear
Appropriate
Somewhat appropriate
Limited
Inappropriate
All Necessary…
Adequate number of…
Minimal amount of…
Lacks enough of…
Significant
Important
Somewhat relevant
Inconsequential, Unimportant
Critical, crucial
Essential
Somewhat useful
Unnecessary
Logical, rational
Reasonable
Somewhat reasonable
Illogical
Intuitive
Somewhat intuitive
Somewhat instinctive
Random
33
2. Written Communication Skills: Outcome a. The student writes standard English
Level of Proficiency/Performance
Criteria
Exemplary (4)
Accomplished (3) Developing (2)
Beginning (1)
Score
1.Idea and
Content
Writes clearly and with
focus; relevant details
support the central theme.
Maintains clear focus
throughout the paper with
sufficient appropriate details
indicating awareness,
knowledge, and insight.
Writes with unclear
purpose or central theme.
Does not clearly define or
support position on topic.
Uses limited or
disconnected details that
disrupt the unity of the
paper.
3
2.
Organization
/ Structure
Provides clear introduction
and reinforcing conclusion.
Orders writing logically with
effective transitions,
providing sufficient
information in the
appropriate places.
Supports thesis and purpose
through organization and
paragraphing; most transitions
are appropriate, but sequence of
ideas may need improvement.
Reiterates introductory elements
in conclusion.
Partially focuses on topic with
minimal or no support of
position. Writing is basic, too
general for the reader to
develop a clear understanding.
Writes with some signs of
logical organization; may
include abrupt or illogical
shifts and ineffective flow of
ideas. Makes few transitions
between ideas.
Writes with organization
that is unclear or
inappropriate to the thesis;
lacks transitions between
ideas.
2
3.Voice
Writes expressing own
personality, with confidence
and feeling. Individual,
powerful commitment to the
topic is obvious, as are
strong connections to the
audience and to the purpose;
evokes strong emotion in the
reader.
Writes so that own personality
pokes through; confidence and
feeling fade in and out.
Commitment to the topic is
apparent, and connection to the
audience and to the topic are
appropriate. The writing evokes
some emotion in the reader.
Writes without revealing own
personality; writing is
cautious. Commitment to
topic, and connection to the
audience and to the purpose
are limited. Writing evokes
limited emotion in reader.
Writes without personality.
Shows lack of commitment
to topic, connection to the
audience and to the
purpose. Evokes no
emotion in reader.
4
Comments:
Total
9 / 12
34
2. Written Communication Skills: Outcome a. The student writes standard English
Level of Proficiency/Performance
Criteria
Exemplary (4)
Accomplished (3)
Developing (2)
Beginning (1)
1.Idea and
Content
Writes clearly and with
focus; relevant details
support the central theme.
Maintains clear focus
throughout the paper with
sufficient appropriate
details indicating
awareness, knowledge, and
insight.
Partially focuses on topic
with minimal or no support
of position. Writing is
basic, too general for the
reader to develop a clear
understanding.
Writes with unclear
purpose or central
theme. Does not clearly
define or support
position on topic. Uses
limited or disconnected
details that disrupt the
unity of the paper.
2.
Organization
/ Structure
3.Voice
Comments:
Provides clear introduction
and reinforcing conclusion.
Orders writing logically with
effective transitions,
providing sufficient
information in the
appropriate places.
Writes expressing own
personality, with confidence
and feeling. Individual,
powerful commitment to the
topic is obvious, as are
strong connections to the
audience and to the purpose;
evokes strong emotion in the
reader.
Supports thesis and
purpose through
organization and
paragraphing; most
transitions are appropriate,
but sequence of ideas may
need improvement.
Reiterates introductory
elements in conclusion.
Writes with some signs of
logical organization; may
include abrupt or illogical
shifts and ineffective flow
of ideas. Makes few
transitions between ideas.
Writes so that own
personality pokes through;
confidence and feeling
fade in and out.
Commitment to the topic is
apparent, and connection to
the audience and to the
topic are appropriate. The
writing evokes some
emotion in the reader.
Writes without revealing
own personality; writing is
cautious. Commitment to
topic, and connection to
the audience and to the
purpose are limited.
Writing evokes limited
emotion in reader.
Score
Weight
Subtotal
(Score X
Weight)
3
X2
6
2
X3
6
4
X1
4
Writes with
organization that is
unclear or inappropriate
to the thesis; lacks
transitions between
ideas.
Writes without
personality. Shows lack
of commitment to topic,
connection to the
audience and to the
purpose. Evokes no
emotion in reader.
Total
16 / 24
35
Chocolate chip cookie rubric
What cookie elements could we judge?
• Number of chocolate chips
• Texture
• Color
• Taste
• Richness (flavor)
36
Chocolate chip cookie rubric
Level of performance?
Criteria
Number
of Chips
Texture
Color
Taste
Richness
37
Chocolate chip cookie rubric
Level of performance
Criteria
Delicious
Tasty
Edible
Inedible
Number
of Chips
Texture
Color
Taste
Richness
38
Chocolate chip cookie rubric
Level of performance
Criteria
Delicious
Number
of Chips
Chocolate
chip in
every bite
Texture
Chewy
Color
Golden
brown
Taste
Homebaked taste
Richness
Rich,
creamy,
high-fat
flavor
Tasty
Edible
Inedible
39
Chocolate chip cookie rubric
Level of performance
Criteria
Delicious
Tasty
Edible
Inedible
Number
of Chips
Chocolate
chip in
every bite
Too few or too many
chips
Texture
Chewy
Texture resembles a
dog biscuit
Color
Golden
brown
Burned
Taste
Homebaked taste
Store-bought flavor;
stale, hard, chalky;
preservative after
taste
Richness
Rich,
creamy,
high-fat
flavor
Nonfat contents
40
Chocolate chip cookie rubric
Level of performance
Criteria
Delicious
Tasty
Edible
Inedible
Number
of Chips
Chocolate
chip in
every bite
Chips in about
75% of bites
Chocolate in 50%
of bites
Too few or too many
chips
Texture
Chewy
Chewy in middle,
crisp on edges
Texture either
crispy/crunchy or
50% uncooked
Texture resembles a
dog biscuit
Color
Golden
brown
Either light from
overcooking or
light from being
25% raw
Either dark brown
from overcooking
or light from
undercooking
Burned
Taste
Homebaked taste
Quality storebought taste
Tasteless
Store-bought flavor;
stale, hard, chalky;
preservative after
taste
Richness
Rich,
creamy,
high-fat
flavor
Medium fat
contents
Low-fat contents
Nonfat contents
41
Rubrics
Advantages: provides
• Clearer expectations
• More consistent and objective
assessment
• Better feedback
42
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
43
a. Analytic rubrics (scoring guide)
• describes specific details we are looking
for in the work (provides a lot of
information)
• performance is judged separately for each
criterion.
• useful for formative evaluation of a
product or process
• many formats
44
See hand-out
Scoring Guide for Assessing Team Member Performance
Student name: ____________________________________ Project title: _____________________________________
Course name and number: ________________________________ Date: _____________________________________
Learning outcome: Work collaboratively, in a team environment, to complete a project, to standards.
Directions: Circle the number that best summarizes your assessment of a team member’s performance. Please write
suggestions for improvement on the back of this page for any of the indicators you rated “beginning” or “developing”.
Beginning Developing Proficient Exemplary
1. Interpersonal communication, information exchange
 Withheld judgment while listening to others ......................1
 Encouraged and valued the contribution of all members ....2
 Enhanced the flow of information to all members ..............1
 Used technology to stay connected .....................................1
2
4
2
2
3
6
3
3
4
8
4
4
NA
NA
NA
NA
2. Work contribution
 Engaged willingly in the work ............................................3
 Provided useful information and experience.......................1
 Willingly gave and received constructive feedback............1
6
2
2
9
3
3
12
4
4
NA
NA
NA
3. Task coordination
 Helped clarify team’s task(s) ..............................................1
 Helped organized and chart work flow ...............................1
 Contributed to efficient use of team time ............................1
2
2
2
3
3
3
4
4
4
NA
NA
NA
4. Individual responsibility
 Asked for verification of individual responsibilities ...........1
 Assumed responsibilities.....................................................3
 Stayed engaged until the team’s task(s) was finished .........3
 Followed team’s timelines and deadlines............................1
2
6
6
2
3
9
9
3
4
12
12
4
NA
NA
NA
NA
5. Adaptability
 Showed respect for differences in ability and work style....1
 Adapted to change...............................................................1
 Reflected on team’s process and interaction .......................2
2
2
4
3
3
6
4
4
8
NA
NA
NA
6. Interdependence
 Helped build internal/external networks to get work done .1
 Helped synthesize input ......................................................1
 Assisted other team members..............................................3
 Trusted other team members ...............................................1
2
2
6
2
3
3
9
3
4
4
12
4
NA
NA
NA
NA
Comments:
Level of
Achievement
Exemplary
Proficient
Developing
Beginning
Grade Scale
A+
90% - 100%
A
85%-89%
A80%-84%
B+
76%-79%
B
72%-75%
B68%-71%
C+
64%-67%
C
60%-63%
C56%-59%
D
50%-55%
F
0%-49%
Subtotal
Subtotal
Subtotal
Subtotal
Subtotal
Subtotal
Total score (add subtotals from above)
____ / 124 x 100 = ____ %
Check one:
 Self assessment
 Peer assessment by ______________________
 Instructor assessment by _________________
45
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
46
b. Holistic rubrics
• assigns a level of performance by
assessing performance across multiple
criteria as a whole
• describes criteria more holistically
(provides fewer details, does not list
separate levels of performance for each
criterion)
• is time-saving for the instructor
• is useful for summative evaluation of a
product or process (tool to advance)
47
Holistic Rubric for Assessing Team Member Performance
See hand-out
Student name: Joseph Salmon
Project title: Nooksack River Stream Management
Date: January 24th, 2009
Course name and number: ENVS 263
Learning outcome: Work collaboratively, in a team environment, to complete a project, to standards.
Directions: Circle the number that best summarizes your assessment of a team member’s performance. Please write
suggestions for improvement on the back of this page for any of the indicators you rated “beginning” or “developing”.
Level of Performance
Score
Subtotal
(Score X
Weight)
Criteria
Exemplary (4)
Accomplished (3)
Developing (2)
Beginning (1)
1. Interpersonal
communication
Showed superb skill in
listening to others,
withholding judgment of
others, advocating for own
ideas and keeping
information flowing in/out
and through the team.
Participated fully to
further the teamwork;
had some difficulty
listening to or valuing
others and keeping the
information flowing.
Actively
communicated with
others on the team
but needed to either
listen, contribute,
advocate or make a
genuine contribution
to the discussion.
Communicated
minimally with
others on the team;
did not listen to or
express ideas.
4
Weight
X2
8
2. Work
contribution
Became one of the
primary contributors to the
work.
Willingly engaged in
the work all the way
through.
Made an adequate
contribution.
Made a minimal
contribution.
3
X2
6
3. Task
coordination
Exhibited a high level of
organization and
coordination skill.
Took responsibility for
coordinating parts of
the work.
Assisted in
coordinating the work
when directly asked
by other members.
Made no effort to
help organize the
work.
2
X1
2
4. Individual
responsibility
Demonstrated a sense of
responsibility for the
quality and timely
completion of the whole
team task.
Took full personal
responsibility for their
portion of the task.
Assumed
responsibility now
and then during the
work.
Showed no sense of
real personal
responsibility for
the work.
3
X3
9
5. Adaptability
Used member differences
and unpredictable
happenings to realign and
complete the work tasks to
quality standards.
Adjusted well to
individual differences
among team members.
Tried hard to adjust
to member
differences and other
unpredictable
changes; still had
difficulty.
Was unable to
adjust to member
differences and/or
needs for change
2
X2
4
6. Interdependence
Networked with all team
members, established trust
and synthesized ideas to
move the work forward.
Willingly sought and
assisted other team
members; strengthened
the network.
Sought out team
members and looked
for ways to integrate
their work; had
difficulty.
Seemed to “stand
alone”.
1
X2
2
Total Score
Comments:
Level of
Achievement
Exemplary
Proficient
Developing
Beginning
Grade Scale
A+
90 - 100%
A
85 - 89%
A80 - 84%
B+
76 - 79%
B
72 - 75%
B68 - 71%
C+
64 - 67%
C
60 - 63%
C56 - 59%
D
50 - 55%
F
0 - 49%
31 / 48
Total score (add subtotals from above)
31 / 48 x 100 = 65 %
Check one:
 Self assessment
 Peer assessment by ______________________
 Instructor assessment by _________________
48
Compare analytic and holistic rubrics
Holistic
Analytic
Description
provides a single score
based on an overall
impression of a student’s
performance on a task
provides feedback along
several dimensions;
descriptors may or may not be
included
Advantages
quick scoring provides an
overview of student
achievement
more detailed feedback;
scoring more consistent
across students and graders;
effective tool for self, peer,
and instructor evaluation
Disadvantages
does not provide detailed
information, may be
difficult to provide one
overall score
time consuming to score
Use when you
want
a quick snapshot of
achievement and a single
dimension is adequate to
define quality
to see relative strengths and
weaknesses or detailed
feedback; to assess
complicated skills or
performance; to self- or peer
assess performance
49
Examples of rubrics
50
Media Literacy Reporting
Student: __________________________
Date: ____________________________
outstanding = 5
very good = 4
satisfactory = 3
unsatisfactory / not possible to evaluate = 1
Outcome: To engage students in the process of educating themselves, and one another, about current
developments in these areas of the media literacy movement, using the Internet as a source.
Provides an insightful comparison of two or more organizations or initiatives.
2
6
8
10
Focuses on mission/goals/purpose/funding and approaches/strategies.
1
3
4
5
Presents in a coherent, articulate and systematic manner.
1
3
4
5
Provides necessary context and background for audience to understand presentation.
1
3
4
5
Uses visual aids (e.g., Internet or Powerpoint projections, other) in presentations.
1
3
4
5
Presents oral information in an engaging and rehearsed manner.
1
3
4
5
Stays within 10-15 minute presentation time-frame.
1
3
4
5
Is overall a creative and original presentation.
1
3
4
5
Pays attention overall to detail.
1
3
4
5
Comments:
Total:
/ 50
51
Rubric for Assessing Chocolate Chip Cookies
Student name: Adib Jamshedi
Project title: Chocolate chip cookie bake-off
Date: January 30th, 2009
Course name and number: Culinary Arts 101
Learning outcome: Students will be able to bake a delicious chocolate chip cookie.
Directions: Please write suggestions for improvement in the “Comments” section at the bottom of this page for
any of the criteria you rated “inedible”.
The cookies must…
 measure between 1 and 2 inches in diameter
 have been baked within 2 hours of the competition
Level of Performance
Subtotal
(Score x
Weight)
Criteria
Delicious (4)
Tasty (3)
Edible (2)
Inedible (1)
Number of
Chips
Chocolate chip
in every bite
Chips in about
75% of bites
Chocolate in
50% of bites
3
X4
12
Texture
Chewy
Chewy in
middle, crisp
on edges
Texture either
crispy/crunchy
or 50%
uncooked
Too few or
too many
chips
Texture
resembles a
dog biscuit
3
X2
6
Color
Golden brown
1
X1
1
Home-baked
taste
Either dark
brown from
overcooking or
light from
undercooking
Tasteless
Burned
Taste
Either light
from
overcooking or
light from
being 25% raw
Quality storebought taste
2
X2
4
Richness
Rich, creamy,
high-fat flavor
Medium fat
contents
Low-fat
contents
Store-bought
flavor; stale,
hard, chalky;
preservative
after taste
Nonfat
contents
3
X1
3
Score
Weight
Total Score
Comments: The yellow color of the
cookies was unappealing.
Level of
Achievement
Exemplary
Proficient
Developing
Beginning
Grade Scale
A+
90 - 100%
A
85 - 89%
A80 - 84%
B+
76 - 79%
B
72 - 75%
B68 - 71%
C+
64 - 67%
C
60 - 63%
C56 - 59%
D
50 - 55%
F
0 - 49%
26
Total score (add subtotals from above)
26 /40 x 100 = 65 %
Check one:
 Self assessment
 Peer assessment by ______________________
 Instructor assessment by AMK
52
Other examples of rubrics?
53
See hand-out:
Instructions for
creating a rubric
54
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
55
Rubric building websites
Rubric Builders and Generators
The Canadian Teacher: www.thecanadianteacher.com/tools/assessment
Tools to create a variety of assessments for student work
MyTL4:http://myt4l.com/index.php?v=pl&page_ac=view&type=tools&tool=rubric
maker
Make customized assessments for student work
NCRtec: http://goal.learningpt.org/spsg/GetProd.asp
Create customized rubrics for student work
PBL Checklists: http://pblchecklist.4teachers.org/checklist.shtml
Create checklists for writing, science, oral presentations and
multimedia
RubiStar: http://rubistar.4teachers.org/index.php
An online tool to develop scoring guides
Rubric Builder: http://landmark-project.com/rubric_builder/index.php
Create scoring guides online
Rubric Generator: http://www.teach-nology.com/web_tools/rubrics/
Personalize ready made scoring guides or create your own
Rubric Studio: http://www.rcampus.com/indexrubric.cfm
Create a rubric or select from already created rubrics
Subject Specific and General Rubrics
Rubrics 4 Teachers: http://www.rubrics4teachers.com/
Educational rubrics created and ready to use
Teacher Helpers Assessment and Rubric Information:
http://school.discoveryeducation.com/schrockguide/assess.html
A variety of subject specific assessment rubrics
Rubric Construction Kit
Assessment Rubrics: http://edtech.kennesaw.edu/intech/rubrics.htm
Criteria for creating rubrics
Atomic Learning: http://movies.atomiclearning.com/k12/la_rubrics_wd03
Directions for creating a rubric in MS Word
2Learn: http://www.2learn.ca/projects/together/START/rubricc.html
Planning worksheet and generator
Rubric Checklist:
http://www.ncsu.edu/midlink/rubrics/Rubric.Checklist.ELS.11.00.doc
Helpful guide for constructing an effective rubric
Rubric Template: http://its.leesummit.k12.mo.us/Rubric%20Template%207.00.xls
An Excel template for creating your own rubrics 56
Rubric Vocabulary: http://www.ncsu.edu/midlink/rubrics/Rubric%20Words.doc
Aids for defining rubric dimensions
Overview
1. What has been done at NWIC in terms of course
outcomes?
2. Why use an assessment tool?
3. 2 Basic Assessment tools
(i) Checklists
(ii) Rubrics:
a. The analytic rubric (scoring guide)
b. The holistic rubric
4. Rubric websites
5. Break into groups: Work on creating rubrics for
course outcomes
57
Create rubrics for course outcomes
• Break into groups or work individually
• Hand-outs:
• How to create a rubric
• Blank templates
• Remember the key to successful rubrics:
→Steal good
ideas!
58
Thank you!
59