Transcript GRADING TO INCREASE LEARNING

ASSESSMENT AND EFFECTIVE
GRADING PRACTICES
DAVID W. KALE, PH.D.
DIRECTOR OF ASSESSMENT
MOUNT VERNON NAZARENE
UNIVERISTY
ACKNOWLEDGEMENTS
• Most of the ideas in this workshop come
from two excellent books on the topic.
One is Effective Grading: A Tool for
Learning and Assessment by Barbara
Walvoord and Virginia Johnson Anderson.
Any really good ideas you hear in this
session probably came from that book or
Knowing What Students Know by the
National Research Council.
WORKSHOP OBJECTIVES
• 1. Faculty will be able to relate principles of
•
•
•
effective grading to their courses;
2. Faculty will be able to discern various levels of
learning in their courses;
3. Faculty will be better able to use grades for
motivation, organization and evaluation;
4. Faculty will be able to use grading to more
effectively reflect student learning.
WORKSHOP OUTLINE
1.
2.
3.
4.
5.
Principles of effective grading
Motivating students with grades
The Assessment Triangle
Linking Assessment with Grades
Linking Assessment, Grading and
Classroom Activities
GUIDING PRINCIPLES FOR
EFFECTIVE GRADING
SUPERIOR
Principle #1
Appreciate the complexity of
grading
• No grading system is immutably right by
some external, eternal standard.
• Collaborate with students to make the
grading process meaningful to them and
to you.
• The goal is to construct a system that will
lead to meaningful change for both you
and the students.
Principle #1
Appreciate the complexity of
grading (con’t)
• The purpose of grades is to provide a
means of:
(1) Evaluation,
(2) Communication,
(3) Motivation, and
(4) Organization.
This is a large order for any grading process
to achieve well.
Principle #2
Substitute Judgments for
Objectivity
• Recognize that grading is a professional
judgment influenced by a wide variety of
factors.
• These factors are meaningful to you and
you will work to make them meaningful to
your students.
Principle #3
Distribute time effectively
• Recognize the time necessary to make a
consistent, thoughtful, professional
judgment.
Principle #4
Be Open to Change
• Be ready to make changes when students
are not demonstrating the learning you
want to see in them.
• If changes in the grading system are to be
made in the middle of a course (which
hopefully will not often be necessary), be
very clear with students as to what is
being changed, why it is being changed
and when the changes take effect.
Principle #5
Communicate and collaborate with
your students
• The more actively students are involved in
the learning process, the better they will
understand your grading system.
• Clearly explain your criteria and your
standards for effective performance.
(Rubrics really help here, but that is the
topic for another workshop.)
Principle #6
Remember, Student learning is
your primary goal.
• Give students a picture of how a
professional would judge their work.
• Support your judgments with clear and full
explanations.
• Provide information on how students can
improve.
• Grades, when used effectively, can be a
powerful motivator of student learning.
Principle #7
Be a Teacher First and a
Gatekeeper Last
• Use grades for both formative and
summative purposes.
• Help guide students through the process,
using grades as a way of helping you and
the students track what they have
learned.
Principle #8
Encourage Learning Centered
Motivations
• Counter student perceptions that:
1. Hard work doesn’t matter;
2. They are powerless to affect their own
welfare;
3. Failure is due to circumstances beyond
their control;
4. Grades are mostly an indication of who
the professor likes or does not like.
DAILY GRADING
• I am convinced some students enter courses
•
•
asking themselves how they can get through the
course with the least effort possible.
Others have a sincere desire to learn, but want
their learning to be recognized and rewarded.
Daily grading both keeps the pressure on the
first student and motivates the second student.
It treats learning as something that is
continuous throughout the course.
DAILY GRADING
• The motivational aspect of grading suggests that
•
•
students should feel that learning is something
that is expected in each class period and not just
when the time comes to cram for a test.
Provide motivation to learn as often as you can,
in every class session if possible.
Perhaps there is a grade for participation in
discussion which could be as simple as a plus,
check or minus for each class.
DAILY GRADING
• In Senior Colloquium I have questions posted on
•
•
Blackboard at the beginning of the course for all
assigned readings.
I tell students their class participation grade
comes largely from whether they know the
answers to those questions when I call on them.
I make an effort to call on every student in
every class period and with 25 students in the
class I generally accomplish that.
MOTIVATING STUDENTS
• Barbara Walvoord has demonstrated that
student motivation can change from one
course to another and within a particular
course.
• Some students who enter a course with a
feeling that there is little they can do to
affect their outcome can get highly
motivated when they see that their initial
impression was not true.
MOTIVATING STUDENTS
• A sociologist, talking about teaching a
general education course, “You always get
the students who are interested right at
the start, but the ones I really like, the
ones that get my adrenalin going, are the
ones who are slouching back, thinking,
‘What a jerky course this is.’ Then you
show them what sociologists do, and how
much fun it is, and sometimes, wow,
They get it.”
SCAFFOLDING
• Establish a building relationship among
the objectives and the assignments in the
course. Create a course outline (some
call it a course map) which makes clear
both:
– how the whole course structure holds
together, and
– how each assignment or test builds on what
the student has already learned.
THE ASSESSMENT TRIANGLE
COGNITION
OBSERVATION
INTERPRETATION
PELLEGRINO, ET. AL., KNOWING WHAT STUDENTS KNOW
COGNITION
• “In any particular assessment application,
a theory of learning in the domain is
needed to identify the set of knowledge
and skills that is important to measure for
the task at hand, whether that be
characterizing the competencies students
have acquired thus far or guiding
instruction to increase learning.”
Pellegrino, et. Al., Knowing What Students
Know, p. 44.
COGNITION
• What are the dominant learning theories
in your area about how students learn
material in that discipline?
COGNITION
• Instructional strategies should then lead
•
students through the learning process in the
way which most closely matches the way in
which they learn this material best.
For example, my experience has been that
students do not learn good speechmaking by
reading sample speeches nearly as well as they
learn by giving speeches and getting prompt
feedback on how to improve.
COGNITION
• As a matter of fact, having students read
speeches mixes the written with the oral
media which could lead students to
believe that as long as they are good
writers they can be good speakers. This
could be an instructional strategy which
leads students directly away from the
objective I am trying to achieve.
COGNITION
• For most disciplines, students must master
a first exposure level of learning which is
then followed by a higher processing level.
LEVELS OF LEARNING
• In teaching Advanced Public Speaking, I always
•
found that skill in audience analysis was an
important processing objective in that course.
In the first exposure level, students learned the
factors that were important in audience analysis,
(e.g., age, culture, level of education, etc.) but
(1) using those factors to do an actual analysis
and then (2) building a speech based on that
analysis were two important higher level
processing course objectives.
LEVELS OF LEARNING
• The reading for homework would provide a
•
review of audience analysis categories students
were exposed to in Public Speaking, providing a
more in depth discussion than they had in the
earlier course.
My in-class assessment was to have students
watch a speech and then work in groups to
identify the evidences that the speaker had
adapted the speech to that particular audience.
LEVELS OF LEARNING
• In Senior Colloquium, familiarizing
students with the decision making model
for moral and ethical decisions was the
first exposure level.
• I would then have them read a case study
involving a moral or ethical decision as a
homework assignment and work through
the case study in class to see if they were
at the processing level.
LEVELS OF LEARNING
• This would give me an idea as to how
many case studies I would need to do in
class to help them get to the level of
analysis I wanted them to achieve.
• It would also tell me what aspects of the
model I needed to emphasize in the case
studies.
LEVELS OF LEARNING
• Hopefully by mid-term exam time they were
•
•
ready to be shown a case study for the first time
when they walked into class and then use the
decision making model to analyze the case as
the mid-term exam.
I remember well how distressed I was the first
time I did this and in scoring the tests realized
that the weakest part of the students answers
was in using Christian principles as the basis for
their decision making. (:<)
I knew I had to make some changes.
COGNITION
• What is the “exposure level” learning and
what is the “higher processing level”
learning you want your students to
acquire in your course?
• Bloom’s Taxonomy would be a good
starting place for identifying the types of
learning students are likely to go through
in coming to grips with the material of
your discipline.
BLOOM’S TAXONOMY
• Bloom’s taxonomy is a means of
identifying the level of cognitive
processing you are expecting from
students.
• This can be applied to the objectives you
are setting for your course as well as to
the means you are using to determine the
degree to which those objectives are
being achieved.
Original Terms
New Terms
• Evaluation
•Creating
• Synthesis
•Evaluating
• Analysis
•Analysing
• Application
•Applying
• Comprehension
•Understanding
• Knowledge
•Remembering
(Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8)
THE ASSESSMENT TRIANGLE
COGNITION
OBSERVATION
INTERPRETATION
PELLEGRINO, ET. AL., KNOWING WHAT STUDENTS KNOW
OBSERVATION
• In this stage you are identifying the
evidence you would expect to observe
that indicates that a particular level of
learning has taken place.
• For each of your course’s objectives,
identify the evidence that would indicate
to you and the student that the objective
has been achieved.
THE ASSESSMENT TRIANGLE
COGNITION
OBSERVATION
INTERPRETATION
PELLEGRINO, ET. AL., KNOWING WHAT STUDENTS KNOW
INTERPRETATION
• Here is where the grading process comes
in because this is where you determine
the degree to which the objective has
been achieved.
• The grade should therefore be closely tied
to the evidence and to your theory about
how the learning takes place.
LINKING ASSESSMENT WITH
GRADING
MAKE YOUR GRADES HIT THE BULL’S EYE OF
STUDENT LEARNING
GRADING
• Once the evidence of learning pertaining
to a particular objective has been
identified, you are then ready to attach
grades to the various levels.
• See the examples which are included in
your handout.
GRADING
• Link your grading to the level of cognitive
processing in your objectives and your
assignments. Assignments and tests that
require higher level thinking skills should
have greater weight in your grading
scheme.
LINKING ASSESSMENT WITH
OBJECTIVES AND GRADING
• Review the number of questions on multiple
•
•
choice tests in light of the course objectives to
which they are linked.
Are all questions related to important course
objectives for a unit or the course?
Do the most important objectives have the most
questions associated with them or at least the
greatest weight in the test grade?
LEVELS OF LEARNING AND
COURSE ASSESSMENTS
• Make sure your assessments are lined up with
•
the level of learning at which you expect
students to perform, whether at the first
exposure level or the higher processing level.
Don’t ask first exposure type questions if you
are expecting students to be processing at a
higher level and don’t ask higher level questions
if students are only at the first exposure level.
LINKING ASSESSMENT, GRADING
AND CLASS ACTIVITIES
ASSESSMENTS
OBJECTIVES
GRADING
CLASS
ACTVITIES
LEVELS OF LEARNING
• What are the evidences you will look for to
•
indicate that the student has moved from the
first exposure to the processing level of
learning?
What activities will you have planned for your
students to help them move from the exposure
level to the processing level? Is there a reading
to be done before class, a case to be read and
analyzed, a set of problems to be solved, etc.?
LEVELS OF LEARNING
• What built in class activity can you design to see
•
whether students have moved from the first
exposure level to the higher level processing
level? This becomes an important in-class
assessment. The students may consider the
activity to be solely instructional, while in fact it
may also have an important assessment
purpose.
We call this course-embedded assessment which
may be a case study to analyze, a problem to
solve, a group activity to complete, etc.
LEVELS OF LEARNING
• Note you may have an activity to help
students move to a higher level and a
subsequent activity to determine whether
students have moved to the higher level.
• On the other hand, you may have one
activity serve both purposes.
LEVELS OF LEARNING
• The key is to use class time as much as
possible for process oriented teaching and
use outside of class time for first exposure
to the concepts involved.
• By the end of the class period, students
should be demonstrating evidence as to
the level of learning at which they are
performing.
THE CLASSROOM “FLIP”
• Jeremy Strayer demonstrated in his
doctoral dissertation research that
students learned just as well when the
first exposure learning was done outside
of class, rather than in an in-class lecture,
and the application of the learning was
done during the class period. This has
been called the classroom “flip.”
LEVELS OF LEARNING
• Sometimes this will require a
demonstration by the faculty member of
the process oriented learning you are
expecting of the student.
• You will need to decide whether that
demonstration or example can be in
reading they do before class or whether it
needs to be an in-class demonstration.
LEVELS OF LEARNING
• Walvoord gives an example of a physics
•
professor who said students could not
understand a physics text without the guidance
of a faculty member so he had to lecture in
class.
Colleagues suggested that he have his lectures
videotaped so students could review them as
they read the material before class and then use
class time for application purposes.
LEVELS OF LEARNING
• He has students work in groups to do
homework problems so they taught each
other.
• If a group got stuck, they could get help
from the professor right when they
needed it.
• Once that demonstration tape was made,
it could be re-used with future classes.
LEVELS OF LEARNING
• Even in large classes, the class could be
broken up into groups to work on
problems and call for help from the
professor when they needed it.
• The degree to which groups were able to
arrive at the correct answers on their own
was an important in-class assessment of
student learning.
LARGE CLASSES
• For large classes, break up the class into
on-line discussion groups on Blackboard to
discuss readings outside of class.
• Blackboard will give you the opportunity to
“drop in” on their discussions to see if
they are being productive and give
comments to them if they are “off track.”
SCIENCE LABS
• Science professors often complain that students
•
do not apply the concepts learned in the lecture
when they get to the laboratory.
Some science professors require students to
have a “ticket” to get into the lab. The “ticket”
is an explanation, in their own words, of the
concepts from the lecture or reading the lab is
intended to demonstrate. (Walvoord and
Anderson, p. 62-63.)
LEVELS OF LEARNING
• My experience is that giving students questions
•
to be answered based on reading assignments
outside of class is an excellent way of preparing
them to move from the first exposure to the
process oriented level of learning.
These questions should be carefully crafted so
they do not assume a higher level of learning
than what should be expected after first
exposure, but yet cause some cognitive
dissonance with staying at that level.
LEVELS OF LEARNING
• I always tried to include at least one
question that students could not answer
strictly based on the reading, but which
could be answered if the student gathered
some additional information or did some
critical thinking about the issue involved.
LEVELS OF LEARNING
• In Senior Colloquium, students also know
that debates are coming up in which they
will be expected to debate both sides of a
controversial issue.
• I emphasize to them that developing the
ability to use the decision making model
will help them as they prepare for the
debates.
PRIMARY TRAIT ANALYSIS
• Barbara Walwoord has suggested that we
think about the primary traits we are
attempting to develop in our students.
• These traits should grow directly out of
the course objectives.
PRIMARY TRAITS IN WORK
RELATED PERFORMANCE
• Comprehension
• Problem identification and solution
• Organization
• Creativity
• Analysis
• Synthesis
PRIMARY TRAITS IN WORK
RELATED INTERACTIONS WITH
OTHERS
• Collaboration
• Listening skills
• Leadership skills
• Team work
• Group facilitation
PRIMARY TRAIT ANALYSIS AND
LEVELS OF LEARNING
• For each primary trait you identify, work
out a hierarchy of student learning which
you hope to see students achieve.
• What are the levels of learning within each
particular trait?
• What evidence would you expect to see
that students were performing at a
particular level?
SUMMARY
1. Grading is most effective when it
provides as accurate a reflection of
student learning as we can possible
achieve.
2. For that to happen, we need to have as
close an association as possible among
course objectives, assessment strategies
and classroom activities.