Transcript Classroom Assessment Techniques (CATs)

Classroom Assessment
Techniques
• Teaching Day
• August 23, 2006
• Nancy Hollins
Classroom Assessment
Techniques (CATs)
• CATs
are simple methods faculty can use to collect
feedback, early and often, on how well students are
learning what they are being taught. The purpose of
classroom assessment is to provide faculty, and students,
with the information and insights needed to improve
teaching effectiveness and learning quality. It differs from
tests and other forms of student grading in that it is
generally not used to assign grades.
• Classroom assessment can be used as a type of
classroom research; as such it should be viewed as a
legitimate form of scholarship.
Example: Written work
• Paraphrasing, summarizing, making
connections, applying to new situations
• Limit writing by limiting the time allowed for
writing or ask a specific question to focus the
writing or ask for one page only
• Analysis: Sort written work into categories
(adequate/inadequate) and count –What are the
common misconceptions? What is the general
level of understanding and is that level
sufficient?
Example: Worksheets
• Recall and list, categorize, identify pros
and cons, outline, concept maps
• Easily scanned and therefore very time
efficient
• Analysis: identify commonly missed items
Example: Problem Solving
• Identify type of problem, critique solutions,
generate solutions
• Can be written work or based on a worksheet.
Documented problem solving (identifying the
steps taken) is more time intensive for the
student but can take the form of a flowchart so
assessment is efficient for the professor.
• Analysis: Sort into adequate/inadequate - look
for superficial thinking, lack of critical thinking,
missing knowledge
Example: Self Assessment
• Study skills, knowledge, ability to apply,
attitudes
• Journals, surveys, anonymous writings
that are read publicly.
• Analysis: Sort into adequate/inadequate –
address those issues that are common
e.g. misperceptions, biases, time issues
with studying.
Example: Classroom Management
• Assignments, class activities
• Journal, survey, suggestion box, class
student representatives
• Analysis: Sort into types of problems –
address those that are common.
Why should I use CATs?
For faculty, frequent use of CATs can:
• Provide feedback about learning and teaching when it is
still possible to make changes.
• Provide information about student learning with a much
lower investment of time as compared to tests, papers, and
other traditional means of grading.
• Foster rapport with students.
• Encourage the perspective that learning is a joint enterprise
between faculty and student.
Why should I use CATs?
For students, frequent use of CATs can:
•
•
•
•
Help students monitor their own learning.
Point out the need to alter study skills.
Help students learn new study strategies.
Provide concrete evidence to students that the instructor
cares about their learning.
How do I use CATs?
• Start small. As you read through the classroom
assessment techniques described in the slides that
follow, consider which one has the most potential for a
course you are teaching. What do you think you might
learn by using this technique?
• Will you need to modify the basic procedure to suit your
particular situation? How will you modify it?
• Decide whether or not student performance should be
anonymous. With anonymity you get more honest
feedback. But sometimes, even 1% of the final grade will
encourage students to take the CAT seriously.
• As you design the activity, keep it simple. What do you
want students to do? What kind of response do you think
you will get? Don't ask for more data that you need or
are willing to use.
How do I use CATs?
• Introduce the CAT by letting students know how the
information you collect will help them.
• Once you have collected the students' responses, sort
and analyze the data. For the most part, student
responses will probably sort easily into a few general
categories.
• When you have analyzed the data, share at least some
part of that analysis with your students. How will you use
the information? How can the students use the
information?
• Finally, don't feel you have to rush into using these
techniques in every class or during every session. Begin
slowly in order to avoid the only real danger in classroom
assessment--too much data and not enough time or
experience to know what to do with it.
Internet Resources
• Southern Illinois University
http://www.siue.edu/~deder/assess/catmain.html
• Honolulu Community College
http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess2.htm
• Penn State
http://www.schreyerinstitute.psu.edu/Resources/class_assessment.asp
• Middle Tennessee State University
http://www.mtsu.edu/~itconf/proceed99/Martin.htm
• Iowa State University
http://www.celt.iastate.edu/teaching/cat.html
• Angelo, T.A. & Cross, P.K. (1993).
Classroom Assessment Techniques (2nd
ed.). San Francisco: Jossey-Bass.
Specific Techniques
Teaching strategies and CATs are often
very similar. The difference is that
CATs are used specifically to generate
data to be used to make changes in
teaching – with the goal of improving
learning. The analysis of the
assessment results is not focused on
individual student performance but
rather on overall class performance.
Goal Ranking & Matching
• This CAT is best done at or near the beginning of a course when
adjustments to the syllabus are possible.
• The professor begins by identifying the goals for the course, ranking
those goals in order of importance.
• Students then identify their goals for the course and rank them.
• Analysis: List the professor’s goals in order of importance. Next to
them, list students’ goals in order of importance. Do the lists match
in content? Do they match in order of importance? Are there
significant differences between professor and students and between
student and other student?
• The professor uses the data to generate a discussion. This
discussion may involve the “selling” of goals to students – why is it
important for them to achieve a particular goal deemed important by
the professor? On the other hand, perhaps the professor can
incorporate student goals into assignments, class activities. And, if
some goals just can't be realized, an early, honest response can
align expectations in a positive way in order to prevent
disappointments.
Background Knowledge
Learning Probe
• Research suggests that one of the best predictors of student
learning is what the student already knows before coming to class.
Students bring a lot of internalized old knowledge with them. They
try to place new knowledge into existing knowledge and, when it
doesn't fit, it is often discarded (not learned).
• This CAT uses a series of multiple choice questions to probe
background knowledge. Only a few questions need to be used to
sample students' previous knowledge. Probes are useful at the
beginning of individual topics as well as whole courses.
• Analysis: Like multiple choice exams, probes are easy to score.
Common misconceptions can be identified and addressed. Common
gaps in knowledge can be filled. How this remediation takes place is
obviously determined by the constraints of the situation.
Misconception/Preconception
Check
• The misconception/preconception check is a variant
of the background knowledge probe, but it focuses
directly on those kinds of prior knowledge (or beliefs)
that may actually hinder learning. This technique can be
particularly useful in courses dealing with controversial
or sensitive issues, or those in which students may have
developed intuitive but inaccurate theories.
• When preparing multiple choice questions for a
misconception check, begin by asking yourself the
following questions: What misconceptions might be
commonplace among students who take this course?
Which of these are most likely to interfere directly with
learning for the course? How can I deal with these
misconceptions once they are identified?
• Analysis: As with the background probe, common
misconceptions can be identified and addressed.
Minute Paper
• This CAT is typically used at the end of a class session.
Students are asked to write for one minute on a
particular topic. The topic may be the most important
point of a session or the most disturbing or the most
surprising. The professor might choose a topic that
seems to be most puzzling to the students.
• At the end of the minute, students are asked to generate
a question they still have about the topic. This requires
students to integrate the topic into their existing
knowledge base and identify what they still don’t know.
This approach also provides an easier (because more
private) way to ask questions.
• This CAT requires students to organize a "chunk" of
information about a particular topic. Analysis: Look for
commonalities in conceptual understanding (or
misunderstanding) as well as difficulty with organizing
information into a brief written work. The questions
themselves can be incorporated into a review during the
next class session.
RSQC2
• RSQC2 stands for Recall, Summarize, Question, Connect, and
Comment. This is a modification of the One Minute Paper. Students
take two minutes to recall and list in rank order the most important
ideas from a previous day's class. Then they take another two
minutes to summarize those points in a single sentence in order to
"chunk" the information. Next, students are asked to write one major
question that they want answered. Finally, students identify a thread
or theme to connect this material to the course's major goal. As an
option, students may add a comment regarding their confidence in
their understanding of the specific topic.
• Analysis: Analysis is less time consuming if students are told to use
the subheadings (recall, summarize, etc.). The professor then looks
at each subsection to identify a) what important points were not
recalled, b) what misconceptions were found in the summaries, and
c) what relationships between ideas are missing or inaccurate. The
confidence subsection can be viewed as a type of self-confidence
survey – do students lack confidence and therefore need more
practice? The questions can be answered at the next session.
Directed Paraphrase
• Students are given five minutes to summarize a key idea
that was presented during a class period (current or just
past).
• To paraphrase, students must demonstrate that they
understand the idea well enough to write about it in their
own words.
• The directed part specifies the audience for whom the
student is writing. Can the student choose language to
match the needs of a specified audience?
• Analysis: Sort the papers into two piles - adequate
understanding vs. inadequate understanding. Determine
whether misconceptions are individual or class-wide.
Then sort the papers into adequate language vs.
inadequate language. Determine whether or not the
class as a whole has the ability to adjust language to a
specific audience’s needs.
Double-Entry Journals:
• In this CAT, students make notes on one side of a
notebook and use the other side to comment or ask
questions about their notes, react to or reflect on the
topic. These notes can be either a) taken during a
lecture or b) based on assigned course readings.
• This approach can also be used to document problem
solving. On the one side students can record a sample
problem solution and on the other side they can record
the steps they took to solve it.
• Analysis: At the beginning of the next lecture session,
the instructor can start the class with the students
discussing a portion of the questions and comments. By
collecting and scanning the entries, the professor can
also identify any issues that need further teaching.
Word journal
• Students are asked to summarize a specific
topic into a single word, then write a short
paragraph explaining the word selection.
• Analysis: This CAT can be used to examine the
depth of reading comprehensive, creativity in
summarizing information, and skill at defending
selection. Similar to a one-minute paper, this
CAT focuses the students’ attention on the use
and precision of language. The professor can
provide feedback by reading examples in the
next class and having the students discuss the
response as adequate/inadequate.
Muddiest Point
• In this CAT, students are asked to write down what was
least clear to them. This requires the students to rate
their own understanding across several topics and then
reflect on why one particular topic should be selected as
least understood.
• Analysis: Determine those topics that seem “muddy” to a
number of students. Decisions on action may include
additional class time if a significant percentage of the
students have the same “muddy” point. Tutorial sessions
can be scheduled if the number is high but not high
enough to devote more class time. Handouts might
address those points that are “muddy” to only a few.
• This CAT should be used infrequently. Focusing on
muddiest points too often can result in an emphasis on
the negative.
What’s the principle?
• The professor Identifies several basic principles that
students are expected to learn in the course.
• The professor then finds or creates sample problems or
short examples that illustrate each of these principles.
Each example should illustrate only one principle.
• Create a What's the Principle? form that includes a
listing of the relevant principles and the list of specific
examples or problems. Students are then asked to
match the principles to the examples.
• Analysis: Scan for those principles that are inaccurately
identified by a number of students. These principles
need reinforcement.
Characteristic Features
• This CAT involves listing characteristics or properties of
a item, concept or principle. These characteristics are
those that help differentiate the topic from others. This
assessment technique is particularly useful for seeing
whether students are separating items or ideas that are
easily confused. By selecting especially critical
differentiators, a professor can both highlight and assess
the students' use of analysis to help them characterize
central concepts.
• The student is asked to place a plus or minus sign in
front of characteristics that do/do not belong to the topic.
• Analysis: Scanning the results is easy and rapid so this
CAT is suitable for use in large classes. Even simple
tallies will reveal the extent to which students are paying
more attention to some traits and less attention to others.
Memory Grid
• Students fill in cells of a two-dimensional diagram for
which the professor has provided labels. For example, in
a music course, labels might consist of periods
(Baroque, Classical) by countries (Germany, France,
Britain); students enter composers in cells to
demonstrate their ability to remember and classify key
concepts.
• Analysis: Tally the numbers of correct and incorrect
responses in each cell. Analyze differences both
between and among the cells. Look for patterns among
the incorrect responses.
Categorization grid
• This CAT is similar to a memory matrix. Students
are given a grid containing 2 or 3 important
categories and a scrambled list of terms,
images, or equations. Students are asked to sort
the list into the categories.
• Analysis: This CAT can be used to discover how
students categorize information and how well
learners understand "what goes with what“.
Because of the grid, scanning of answers is
fairly easy and efficient.
Pro and con grid
• Students are given a grid on which they
list the pros/cons, costs/benefits,
advantages/disadvantages of a solution to
a particular problem.
• Analysis: The grid format allows for quick
scanning to determine the depth/breadth
of students’ analyses and their capacity for
objectivity and critical thinking.
Empty Outlines
• This technique works when content is delivered by
lecture. The professor asks the student to outline all or
part of the lecture. The student can be asked to start
from scratch or the instructor may provide a partial
outline of the lecture. This technique measures a)
student understanding of relationships within the content
and b) the effectiveness of student listening. The first
time the technique is used, the instructor may want to
provide a partial rather than blank or “empty” outline.
• Analysis: a review of the outlines should indicate
whether or not the students are understanding
connections between information. If results are mixed,
the instructor may ask the students for feedback – did
they not understand the concepts or could they not
organize them? This technique has the added benefit of
enhancing students’ use of the technique of studying
through outlining.
Concept Map
• A concept map is a two-dimensional, hierarchical node-link diagram
that depicts the structure of knowledge within a discipline as viewed
by a student, an instructor or an expert in a field or sub-field. The
map is composed of concept labels, each enclosed in a box or oval;
a series of labeled linking lines, and an inclusive, general-to-specific
organization.
• Analysis: By reading the map, an instructor can:
a) gain insight into the way students view a scientific topic;
b) examine the valid understandings and misconceptions students hold;
and
c) assess the structural complexity of the relationships students depict.
• In addition to these applications in assessment, faculty have also
used concept maps to organize their ideas in preparation for
instruction, as a graphic organizer during class, and as a way to
encourage students to reflect on their own knowledge and to work
together and share their understandings in collaborative group
settings.
Problem recognition tasks
• The professor provides students with a range of
problems from which they must identify the type
of problem that each example represents. This
can be done individually or a list of problems
and types can be matched.
• Analysis: This CAT can be used to examine how
well students can identify problem types and
match problems with possible solution methods.
Specific types that are inaccurately identified by
a number of students can be the focus of further
course work.
Documented problems
• Documented problems are similar to the common request to "show
your work." By asking students to describe their reasoning, the
professor can get a direct measure of students’ problem solving
ability.
• Documentation of a problem can be something as simple as a brief
paragraph or two of what was done (and why) or as extensive as a
line-by-line report of each step (as in a mathematical proof). Having
students write out the reasoning behind each step of a problem
gives the professor very detailed feedback about the students' skills
and understanding. To make the task manageable, you might select
a sample to scan.
• Analysis: The professor categorizes the problem solving strategies,
identifying those strategies that are least effective and most
effective. Additional practice and/or discussion may be added to a
course if a number of students use the least effective strategies. The
professor can also put a solution on the board and ask the students
to analyze it, discussing its effectiveness and offering alternatives.
“To Use or Not to Use" Analysis:
• The professor prepares a sample problem or the
description of a situation and recommends a solution.
Students are requested to discuss reasons why the
solution is or is not appropriate. The professor might also
provide two differing solutions that students debate.
Depending on the scope of the course, the final decision
can be based strictly on a student’s personal choice, or it
may include a more objective analysis of theory,
cost/benefit, research support, etc.
• Analysis: The professor categorizes the responses into
the whys and why nots. If there are significant
differences in the responses, the responses may be
used to guide a class discussion or provide written
feedback.
Application cards
• After teaching about an important theory, principle, or
procedure, the professor asks students to write down at
least one real-world application for what they have just
learned to determine how well they can transfer their
learning.
• Analysis: Read through the applications and categorize
them according to their quality. Pick out a broad range of
examples and present them to the class. Have the
students categorize their quality, discussing what
determines a good answer from an adequate or poor
response.
Transfer & Apply
• In this CAT, students are given a new situation to which
they are asked to apply a theory or principle taught in
class. Using a form with several questions can be
helpful in focusing this writing and therefore making
analysis more efficient. This exercise can be written
individually, in pairs, or in small groups and can take
anywhere from 5–15 minutes.
• Analysis: The professor scans the written papers to
determine the ability of students to transfer learning to
new situations. Poor performance may indicate more
practice with new situations is desirable to enhance
learning.
A Self-Confidence Survey
• This CAT helps to identify areas where students feel competent and
where they do not. The professor identifies the skills needed for
achieving specific learning objectives. These skills are then listed
with students indicating on a rating scale how confident they feel in
the performance of that skill.
• Analysis: The professor identifies the level of confidence and
matches it with his/her own perception of performance. Overconfident students may not feel a need to study. Under-confident
students believe they "can't get this stuff“, becoming demotivated.
Students are placed into small groups to discuss the results of the
survey, The students themselves may discover and offer potential
remedies for both situations.
• The rating scale allows the professor to determine an “average”
confidence level for a particular skill. This average in itself may
identify areas needing further practice.
Assessing Group Effectiveness
• This CAT asks group members individually and anonymously to
describe the task assigned to a group, explain the organization they
see as necessary to accomplish the task, and reflect on the
individual talents and team effectiveness required to conclude the
task successfully. The focus is on self-assessment – what does the
student him/herself contribute to the group, not on assigning blame
or voicing complaints. The assessment should be undertaken early
--- within a couple weeks of the group's formation. Ideally, the
completed assessment forms are circulated within the group,
becoming the focus of constructive discussion. The discussion itself
can be summarized by the group in a formal, written, one-page
report to the professor.
• Analysis: A professor's role in providing feedback might consist of
meeting with the group and orally reinforcing points of agreement
and identifying differences as well. Responsibility for adjusting
behavior to increase effectiveness rests with the group, not with the
professor. The entire process can be repeated later in the semester,
especially if a group is having difficulty.
Student- generated test
questions
• Allow students to write test questions and model
answers for specified topics, in a format
consistent with course exams. This will give
students the opportunity to evaluate the course
topics, reflect on what they understand, and
what are good test items.
• Analysis: Evaluate the questions and use the
goods ones as prompts for discussion. Evaluate
the answers for commonly held misconceptions.
You may also want to revise the questions and
use them on the upcoming exam.
Exam Evaluations
• Select a test that you use regularly and add a few
questions at the end which ask students to evaluate how
well the test measures their knowledge or skills. Use the
feedback to make changes in the exam for future
classes (if reasonable).
• Analysis: Track student performance on the exam over
time…did your changes make a difference.
Self Assessment
• This CAT is an excellent way to make visible different
styles of thinking and different attitudes on controversial
topics...without embarrassing anyone individually.
Students are informed that their written responses will be
private.
• The professor presents students with alternative ways of
looking at a controversial issue and asks them to
indicate, by writing on a 3x5 card, which perspective they
hold and why. The responses are then swapped 3-4
times face down, thus allowing the overall results to be
read publicly without compromising confidentiality.
• Analysis: The professor counts the cards to determine
the most commonly held perspective. The argument for
adoption of this perspective is analyzed for critical
thinking. Points needing clarification or further discussion
are the focus of the next class.
• This technique is most successful when the level of trust
is high, middle to end of semester.
Personal Journals
• Students are asked to keep journals that detail their
thoughts about their learning. Content may include such
items as awareness of multiple perspectives, reflection
on the application of course material, confidence in
performance of specific skills. The journal entries may be
general or address a specific issue such as reflection on
performance of an assignment. The students turn in the
journals several times during the semester so the
professor can give feedback, as well as chart
development. Journals may or may not be anonymous.
• Analysis:The journals will reveal the development of
individual students, but as a CAT the journal can also
reveal patterns of development in the students as a
whole that can be addressed directly.
Assessment of Effective Study
Time
• Effective study can be thought of as a function of time multiplied by
effort. A self Assessment of Effective Study Time can bring habits
of effective study to the surface by focusing a student's attention on
these two factors.
• Students engage in this assessment by filling out a study log (when
do they study, what, and how) for one week and then examining the
log for trends.
• A professor's participation in this assessment can involve individual
consultation with students as well as reporting trends in the logs as
a whole. Students might need help to see opportunities to change
their habits and increase their study effectiveness. Students may
also report significant time expenditures that suggest that their
course (and work) workload is overly heavy or they are spinning
their wheels using ineffective study methods.
Chain notes
• Students pass around an envelope (or envelopes) on
which the teacher has written one question about the
class. When the envelope reaches a student he/she
spends a moment to respond to the question and then
places his/her response in the envelope. These
questions might ask about such things as the like/dislike
of the format of the class sessions, the clarity of a
particular topic, or the perceived relevance of an
assignment.
• Analysis: Go through the student responses and
categorize the data with the goal of detecting response
patterns. Discuss the patterns of responses with
students.
Class Journals
• Students are asked to keep journals that detail their
thoughts about the class. Content may include such
items as organization of lecture material, usefulness of
handouts, effectiveness of group work, or relevance of
an assignment. Specific questions can be introduced
throughout the semester to address such as their
thoughts on a specific media or classroom activity. The
journals should be anonymous so that honest feedback
is obtained. One possible way to do this is to have the
students place some sort of identifying phrase or
password on the journal so they can retrieve it. The
students turn in the journals several times so the
professor can get frequent feedback.
• Analysis:The journals may reveal consistency in the
positive and negative aspects of the course.
Student Group Reps
• Students in a course are asked to select several
students or volunteers to meet as a small group with the
professor on a regular basis. This group will discuss
how the course is progressing, what they are learning,
and make suggestions for improving the course. The
volunteers are periodically given time in class to get
feedback from the other students without the professor
present.
• Analysis: The group reps should also be involved in the
analysis of the information they bring to the meetings
and in identifying possible solutions to any problem.
Some issues will be for your information, some should
be addressed in class.
Suggestion Box
• The professor may put a box near the classroom door
and/or office door and ask students to leave notes about
any class issue.
• Analysis: Review and respond to these suggestions
during the next class session.
Caveat!
• Keep CATs simple.
• When just starting out, limit the use of CATs to
only one or two per course.
• You have to analyze the results quickly in order
for any remediation to be effective.
• Don’t use a CAT when you can’t get to the
analysis in a timely manner.
• Document the results of using a CAT. This
documentation will come in helpful when your
teaching is reviewed by others as well as when
you reflect on your teaching at a later date.