Transcript Assessment Institute, Summer 2010

Assessment Institute, Summer 2011
Heather McGovern
Overview
 Defining “assessment of student learning”
 Writing goals
 Measuring learning
 Applying information
What is assessment of student
learning?
Assessment of student learning is “A natural, scholarly act that
can bring important benefits” with 3 steps (Walvoord 2010)
or 4 steps (Suskie):
Where is your project in
terms of these steps?
Goals
Information
Teaching
Action
What common changes occur because
of assessment?
“Changes to curriculum, requirements, programmatic
structures, or other aspects of the students’ course of
study.” Examples!—WGSS, POLS, PHIL, COMM, MATH
2. “Changes to the policies, funding, and planning that
support learning” Examples—class size, qualifications of
teachers,
tutors have you/your program made because
What changes
of informal Examples!
or formal Writing
assessment?
3. “Faculty development”
(information
literacy) (Walvoord)
See list of questions assessment can inform in Suskie, pgs. 6566
1.
What is bad assessment?
“Mere compliance with external demands”
“Gathering data no one will use”
a minute
paper on your fears about and bad
“Making theWrite
process
too complicated”
experiences with assessment.
(Walvoord)
What does a good assessment do?
 “yield reasonably accurate and truthful information about
what students have learned, so that we can use the assesment
results with confidence to make plans and decisions”
Write a minute paper about your hopes for and
 “have a clear purpose, so that [they] are valued and don’t end
good
experiences
with
assessment
up sitting on a shelf
 “Engage faculty and staff, so the assessment becomes a useful
part of the fabric of campus life”
 “Flow from and focus on clear and important student
learning goals, so the results provide information on matters
the college or university cares about”
(Suskie, pg. 37)
Writing Goals
Writing goals: What do we want
students to know or be able to do? I
We might consider goals at the institutional level, school level,
program level, or course level (Walvoord pg. 14)
We can identify learning goals through our own reflection or
collaboration with other faculty, staff, and/or students to
reach consensus (pg. 127, Suskie—see discussion questions
on pg. 128)
Which
of the
items onWhat
this slide
youwant
most
Writing
goals:
doarewe
interested
in?
Reflect
for
a
moment
on
your
own,
students to know or be able to do?
then we’ll share.
II
We might think about goals in terms of Kirkpatrick’s 4 levels of
Note,experience
I recommend
you don’t
get caught
learning
outcomes
(in Suskie,
pg. 22):
up in vocabulary, but if you want to, you
1) Reaction
can read Suskie pg. 116-117
2) Learning
3) Transfer
4) Results
We might consider formative items like a) learning processes
(pg. 25), b) inputs (pg. 26), and C) context (pg. 27) as well
as the summative d) outcomes (“knowledge (knowledge,
skills, attitudes, and habits of mind”) (Suskie, pg. 23)
Which of the items on this slide are you most
interested in for your project? Write down all the
Common
terms
you thinklearning
might apply.outcomes (Suskie)
 Knowledge and conceptual understanding (pg. 118):
“remembering, replicating a simple procedure, and defining,
summarizing, and explaining concepts or phenomena”
 Thinking and other skills (pg. 120): application, analysis,
evaluation, problem-solving, decision-making, synthesis, creativity,
critical thinking, information literacy, performance skill,
interpersonal skills
 Attitudes, values, dispositions, and habits of mind (pg. 123):
“appreciation, integrity, character, enjoying and valuing learning,
becoming more aware of one’s values, attitudes, and opinions and
their evolution,” metacognition, “capacities to work independently,
set personal goals, persevere, organize, be clear and accurate,
visualize, be curious”
Writing goals: language I
Walvoord, pg. 14: Say “Students will be able to….”
NOT
The curriculum emphasizes…
The institution/program/course values
Students are exposed to…
Students participate in…
With a partner, write 2 good learning goals.
Writing goals: language, II
More advice in Suskie, pgs. 129-132
Bottom line—try using action-word goals where possible. Try
not to be too broad or too specific.
Sometimes, broader words like “know,” “understand,”
“appreciate” are appropriate when combined with a rubric or
rating scale. For example, trying to break “communicate
effectively in writing” down might result in so many specifics
that the major goal is lost. (Suskie, pg. 131)
Measuring learning
Two main kinds of data
 Indirect (surveys (NSSE and others), grades,
retention/graduation rates, placement rates, IDEA forms,
GRE scores, etc.)
 Direct (field or employer ratings; scores and pass rates on
licensure exams or Major Field Tests; capstone experiences,
portfolios, written work, performances, projects, etc, scored
using a rubric; score gains on local tests or the CLA;
systematic observations of student behavior)
See table in Suskie, pg. 21
What
kinds ofkind
measurement
might
appropriate for
Connect
of data
tobemeasurement
each of the two learning goals you have? Why or why
(Suskie,
pg.
34)
not? Discuss with your partner.
 Attitudes/values/
 Survey, focus group,
disposition/reaction
 Thinking/Performance
skills
 Knowledge/conceptual
understanding
 analysis/application
interview, reflection
 Assignments with rubric,
performance task
 Multiple choice tests
 Assignments with rubric,
performance task or case
study, portfolio
Toolbox for measuring
 Scoring guide/rubric for an assignment, performance,





portfolio, performance task, etc. (Chapter 9, Suskie)
Assignment for assessment (Chapter 10, Suskie)
Making a local test for assessment—particularly see the test
blueprint on pg. 167 (Chapter 11, Suskie)
Portfolio for assessment (Chapter 13, Suskie)
Using a published test or survey (Chapter 14, Suskie)
Using reflective writing, short questions, prompts, journals,
interviews, focus groups, surveys (Chapter 12, Suskie)
Two (or three) ways to get your data
 Use existing student work/tests/surveys/IDEA/NSSE/CLA
 Embed new assignments/tests/surveys in classes
 Add-on work
Program default plan, Walvoord
Have learning goals for each degree
2) Have two measures of how students are meeting each goal,
one direct (Walvoord suggests a sample of student work at
the end of the course of study) and one indirect (Walvoord
suggests a student survey or focus group)
1)
Read chapter 3, Walvoord. See the list of examples, pg. 64-66
Side note: General studies
considerations, Walvoord
-Each unit makes course learning goals, assignments and tests,
and ways to measure these consistent with the general
education goals
-Each unit regularly sends a two page report on
strengths/concerns/actions to a body or committee
-Institution-wide survey like NSSE
Suskie notes things that make assessing general studies difficult,
pgs. 110-111.