Transcript Slide 1

Introduction to Assessment of
Student Learning
Kathleen M. Morley, Ph.D.
University Director of Assessment
7/18/2015
1
Overview
I. Assessing Learning
II. Goals & Objectives
III. Measures
IV. Rubrics
7/18/2015
2
I. Assessing Learning
7/18/2015
3
Why Assess Student Learning?
“Natural human, scholarly act” – Barbara
Walvoord, 2009
Identify student strengths and weaknesses
National movement that is not going away
7/18/2015
4
What is Assessment?
“Effective assessment is best
understood as a strategy for
understanding, confirming, and
improving student learning”
– Higher Learning Commission
7/18/2015
5
ASSESSMENT FRAMEWORK
Long Island University Mission
Campus Mission
School/College Mission
Academic Program Goals
Program Effectiveness
Student Learning
(e.g., admissions,
(e.g., student knowledge,
curriculum, satisfaction)
skills, and attitudes)
7/18/2015
6
Student Learning Assessment Cycle
Clear
Learning
Goals
Action for
Improvement
Analyze Findings
7/18/2015
Clear Learning Objectives
(i.e., how goal is
demonstrated)
Identify Measures
Collect Data
7
Assessment of Student Learning
How do we know students are
achieving program learning
outcomes?
• Establish program learning
goals
• Apply curricular mapping
7/18/2015
• Implement a program-level
assessment plan
8
II. Goals & Objectives
7/18/2015
9
Definitions
Goal:
“General statement about knowledge, skills,
attitudes, and values expected in graduates” *
Objective:
“A clear concise statement that describes how
students can demonstrate their mastery of a
program goal” *
* Allen, Mary J., Assessing Academic Programs in Higher
Education. Anker Publishing Company, 2004.
7/18/2015
10
Examples
Goal:
Students will utilize quantitative methods in
social scientific research.
Objective:
Students will independently design and carry
out correlational research that yields valid
results.
7/18/2015
11
Bloom’s Taxonomy
Bloom’s Taxonomy:
• classification for levels of learning
• moves from the least to most complex
• has list of verbs associate with each level:
www.odu.edu/educ/roverbau/Bloom/blooms_taxono
my.htm
7/18/2015
12
Bloom’s Taxonomy
www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm
7/18/2015
13
Writing Objectives: The 3 Ms
Meaningful
• How does the objective support the
department goal or mission?
Manageable
• What is needed to foster the achievement
of the objective?
• Is the objective realistic?
Measurable
• How will you know if the objective has
been achieved?
• What will be the assessment method?
7/18/2015
StudentVoice.Com
14
Characteristics of a Good Objective
• Explains how students can
demonstrate the mastery of a
learning goal
• Uses an active verb
• Specifies definite observable
behaviors
• Is clearly linked to a goal
7/18/2015
15
III. Measures
7/18/2015
16
Why Grades Are Insufficient

Do not tell you about student
strengths and weaknesses in learning
content

Cannot use average grade to tell you
where to make changes in the
curriculum
7/18/2015
17
Definitions
Direct Measure:
A measure that directly demonstrates that
students achieved a learning objective
Indirect Measure:
A measure that offers students’ opinion
about meeting the learning objective
7/18/2015
18
Examples
Direct Measure:
 Set of test items
 Written assignment scored with a rubric
 Direct observation scored with a rubric
Indirect Measure:
 Student reflections/self-assessments
 Student opinion surveys
 Focus groups
7/18/2015
19
Choosing Measures

Begin with what you are already assessing

Use what will tell you about student
strengths and weaknesses

Look for culminating experiences
7/18/2015
20
Example Assessment
AY09-10
Phase I – due November 15, 2009
Phase II –due May 31, 2010
Program
Learning
Goal:
Students
will…
Student
Learning
Objective
s
Students
will…
Direct and
Indirect
Measures
Findings (Learning
Outcomes)
Interpretation
Actions Planned
Goal 1
(continued
):
Learning
Objective
2:
Measure 1:
Read
proficientl
y
Analyze
collegelevel
readings
proficientl
y
Measure 1:
On average, the 210
ENG XX students tested
correctly answered XX%
of the interpretive
comprehension questions
they attempted within the
time limit, or XX% of all
the interpretive
comprehension
questions. There was a
considerable spread of
results from low to high
scores.
Measure 1:
In contrast with their fine
performance
comprehending the literal
meaning of texts (see
above measure), ENG XX
students on average tend to
be quite challenged in
interpreting meanings that
are not literally spelled out.
Measure 2:
On a 5-point scale, the
Exit Exam essays of 481
ENG XX students most
frequently received
ratings of 3 (XX%) or 4
(XX%), with an
“average” rating of
X.XX.
Measure 2:
The rubric defines a rating
of 4 as signifying a “good,
fairly concise synthesis of
most of the key points,”
and a rating of 3 as
signifying a “fair, less
focused synthesis that may
misidentify key points.”
ENG XX students’ grasp of
key points in readings
could be better developed.
Measures 1 & 2:
The (program) will provide (faculty)
with pedagogical resources on inference
building, such as a handout for students
on reading skills (e.g., connotation,
denotation, etc.).
Senior (faculty) will be leading a series
of seminars in 2010-11 to educate
faculty both within (the Department) and
across campus on reading pedagogy.
The Department also plans to expand its
assessment of students’ reading
proficiency by administering the NelsonDenny Reading Test to students at other
stages in the (program).
Scores for the
interpretive
questions on
the
Comprehensio
n portion of
the NelsonDenny
Reading Test
Measure 2:
A rubric
assessing
students’
written
summaries of
readings
7/18/2015
21
IV. Rubrics
7/18/2015
22
Introduction to Rubrics
by Stevens and Levi, 2005
7/18/2015
23
Definition
Rubric:
“At its most basic, a rubric is
a scoring tool that lays out
the specific expectations for
an assignment.”
- Stevens and Levi, 2005
7/18/2015
24
Student
Learning
Descriptive Scoring Guides
Exhibit 9.6. Descriptive Rubric for a Slide Presentation on Findings from Research Sources*
Well Done (5)
Satisfactory (4-3)
Needs Improvement (2-1)
Incomplete (0)
Organization
Clearly, concisely
Logical progression Vague in conveying
Lacks a clear
written. Logical,
of ideas and
viewpoint and
point of
intuitive
supporting
purpose. Some
view and
progression of
information.
logical progression of
logical
ideas and
Most cues to
ideas and supporting
sequence of
supporting
information are
information, but cues
information.
information. Clear
clear and
are confusing or
Cues to
and direct cues to
direct.
flawed.
information
all information.
are not
evident.
Persuasiveness
Information is
Motivating questions
Includes persuasive Includes persuasive
incomplete,
and advance
information
information with few
out of date,
organizers convey
facts.
or incorrect.
main idea.
Information is
accurate.
Does not orient
Introduction
Presents overall topic. Clear, coherent, and Some structure but does
the
not create a sense of
related to topic.
Draws in audience
audience to
what follows. May be
with compelling
what will
overly
detailed
or
questions or
follow.
incomplete.
relating to
Somewhat appealing.
audience’s
interests or goals.
Etc.
7/18/2015
*Adapted from Suskie, L. (2008). Assessing Student Learning: A Common
Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p. 144.
Do You Need A Rubric?
• Carpal tunnel from writing the
same comments?
• Weeks behind in your grading?
• Worry last papers may be graded
differently than first?
- Stevens and Levi, 2005
7/18/2015
26
Benefits of a Rubric
• Timely feedback
• Increased likelihood of feedback
being read
• Encourages critical thinking
• Helps to refine teaching skills
7/18/2015
- Stevens and Levi, 2005
27
Rubric Components
1. Assignment description
2. Scale
3. Dimensions
4. Level of achievement
- Stevens and Levi, 2005
7/18/2015
28
Rubric Components
3 Level Rubric
Task Description:
Dimensions
7/18/2015
Exemplary
3
Competent
2
Developing
1
29
Rubric Components
Scoring Guide Rubric
Task Description:
Dimensions
7/18/2015
Description of highest level
of performance
Comments
Points
30
Rubric Components
Scoring Guide Rubric
Task Description:
Dimensions
Description
▫
Comments
Points
2
1
▫
7/18/2015
31
Rubric Components
Scoring Guide Rubric with Boxes and Circles
Task Description:
Dimensions
7/18/2015
Description
Comments
Points
▫ text, text, text
text, text, text
text, text, text
2
text, text, text
▫ text, text, text
1
32
Look for Models
(http://www2.acs.ncsu.edu/UPA/assmt/resour
ce.htm#gened_rubrics)
7/18/2015
33
Steps to Creating a Rubric
1. Reflecting on the assignment
2. Listing learning objectives
3. Grouping and labeling
4. Applying to rubric grid
- Stevens and Levi, 2005
7/18/2015
34
Student
Learning









Resources on Rubrics*
How to create a rubric:
http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Create_Rubric/
Create_rubric.html
Rubric for a given task: http://webquest.sdsu.edu/rubrics/rubrics.html
Designing scoring rubrics: http://pareonline.net/getvn.asp?v=7&n=25
Scoring rubrics: http://pareonline.net/getvn.asp?v=7&n=3
Recommendations for scoring rubrics:
http://pareonline.net/getvn.asp?v=8&n=14
Scoring rubric development: validity and reliability:
http://pareonline.net/getvn.asp?v=7&n=10
Rubrics for web lessons:
http://www.teachervision.fen.com/teaching-methods-andmanagement/rubrics/4521.html
Creating rubrics: http://pareonline.net/getvn.asp?v=9&n=2
Performance criteria: http://pareonline.net/getvn.asp?v=9&n=2
*Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition.
Jossey-Bass, San Francisco, CA, p. 154.
Resources
Kathleen M. Morley
University Director of Assessment
[email protected]
x4258
Assessment Website:
http://vpaa.liu.edu/assess/htm
7/18/2015
36