Transcript VALUE METARUBRICS - Office of Assessment & Evaluation

Assessing Essential Learning
Outcomes with the “VALUE”
METARUBRICs
Using Rubrics for
Teaching, Learning, &
Assessment
Introductions

Office of Academic Assessment
101 Hillcrest Hall (0157)
–
–
–

Ray Van Dyke, 231-6003, [email protected]
Steve Culver, 231-4581, [email protected]
Kate Drezek, 231-7534, [email protected]
Others here today
Today’s agenda

Rubrics as an assessment tool
–
–
–

Definition and Overview of Rubrics
Using Rubrics for Program Assessment
Pros/Challenges
VALUE METARUBRICS – Assessment
toolkit
–
–
–
Background
Purpose
Application at Virginia Tech
Overview: What is Assessment of
Learning Outcomes?

“Assessment of student learning is the systematic
gathering of information about student learning,
using the time, resources, and expertise available,
in order to improve the learning.” – Walvoord

A student learning outcome states a specific
skill/ability, knowledge, or belief/attitude
students are expected to achieve through a
course, program, or college experience.

Example: Upon completion of a B.A. degree in
English, a student will be able to read critically
and compose an effective analysis of a literary
text.
What is The Process for Assessing
Student Learning Outcomes?
1. Identify
And Articulate
Student
Learning
Outcomes
2. Gather and Analyze
Information About
Student
Achievement
Of Outcomes
3. Use Information
Gathered
To Improve
Student Learning
Rubric as an Assessment Tool

A rubric is an explicit summary of the criteria
for assessing a particular piece of student
work, plus levels of potential achievement
for each criterion.

Not only can rubrics can be used to assess
an individual student’s performance for a
course grade, rubrics can be used to assess
the achievement of course, program, or
college/ institutional-level student
learning Outcomes
Rubric as an assessment tool

To the extent that a given student assessment
measures or relates to a programmatic
outcome, it can potentially be aggregated and
used for program assessment (courseembedded assessment)

The same student product (e.g., essay,
capstone project) is used twice: once for
individual student assessment and again for
program
Rubric as an assessment tool

Pros
– Course-embedded assessment
helps to overcome student
motivation problem
– Inherent efficiencies – “two birds,
one stone”
– Authenticity of student learning
“artifact”
Rubric as an assessment tool

Challenge – the rubric!
– Think-pair-share


Take a moment and jot down as many
challenges associated with rubrics as you
can
Pair up with a neighbor and share your list
Rubric as an assessment tool

Challenge – the rubric! Needs to be
well-designed
– Time – to create, to test, to
implement, to assess
– Reliability/validity – needs to be
well-designed
Rubric as an assessment tool

Well-designed rubrics increase an
assessment’s reliability by setting
criteria that raters can apply
consistently and objectively.

Well-designed rubrics increase an
assessment’s construct and
content validity by aligning
evaluation criteria to standards,
curriculum, and instruction tasks.
If, in theory, rubrics can be a
useful assessment tool…
AAC&U’s VALUE METARUBRICS ARE A
TANGIBLE “TOOL KIT” TO
FACILITATE THE APPROPRIATE AND
MEANINGFUL ASSESSMENT OF THE
ESSENTIAL LEARNING OUTCOMES
THAT ARE THE HALLMARK OF A 21st
CENTURY LIBERAL EDUCATION
Assessing essential learning outcomes
VALUE METARUBRICS:
ASSESSMENT TOOLKIT
VALUE METARUBRICS:
Assessment toolkit
Background

AAC&U
–
–
–
Organization
role in assessment, accountability
“LEAP” project – Liberal Education &
America’s Promise
VALUE METARUBRICS:
Assessment toolkit
Liberal Education is an approach to
learning that:

Empowers individuals and prepares them to deal with complexity,
diversity, and change.

Provides students with broad knowledge of the wider world (e.g.
science, culture, and society) as well as in-depth study in a specific
area of interest.

Helps students develop a sense of social responsibility, as well as
strong and transferable intellectual and practical skills such as
communication, analytical and problem-solving skills, and a
demonstrated ability to apply knowledge and skills in real-world
settings

usually includes a general education curriculum that provides
broad learning in multiple disciplines and ways of knowing, along
with more in-depth study in a major.
http://www.aacu.org/leap/What_is_liberal_education.cfm
VALUE METARUBRICS:
Assessment toolkit
AAC&U's LEAP Campaign has
defined a robust set of
"Essential Learning
Outcomes" that students
develop during an excellent
contemporary liberal
education. Beginning in
school, and continuing at
successively higher levels
across their college studies,
students should prepare for
twenty-first-century
challenges by gaining all
these outcomes
http://www.aacu.org/leap/What_is_liberal_education.cfm
VALUE METARUBRICS:
Assessment toolkit
VALUE project: PURPOSE
•Valid Assessment of Learning in Undergraduate Education
•seeks to contribute to the national dialogue on
assessment of college student learning.
• builds on a philosophy of learning assessment that
privileges multiple expert judgments of the quality of
student work over reliance on standardized tests
administered to samples of students outside of their
required courses.
•The assessment approaches that VALUE advances are
based on the shared understanding of faculty and
academic professionals on campuses from across the
country.
http://www.aacu.org/value/
VALUE METARUBRICS:
Assessment toolkit
VALUE project: ASSUMPTIONS
•to achieve a high-quality education for all students,
valid assessment data are needed to guide planning,
teaching, and improvement;
•colleges and universities seek to foster and assess
numerous essential learning outcomes beyond those
addressed by currently available standardized tests;
•learning develops over time and should become more
complex and sophisticated as students move through
their curricular and co-curricular educational
pathways toward a degree;
http://www.aacu.org/value/
VALUE METARUBRICS:
Assessment toolkit
VALUE project: ASSUMPTIONS
•good practice in assessment requires multiple
assessments, over time;
•well-planned electronic portfolios provide
opportunities to collect data from multiple assessments
across a broad range of learning outcomes while
guiding student learning and building self-assessment
capabilities;
•e-portfolios and assessment of work in them can inform
programs and institutions on progress in achieving
expected goals.
http://www.aacu.org/leap/What_is_liberal_education.cfm
VALUE METARUBRICS:
Assessment toolkit
VALUE project: Connections to VT

Essential Learning Outcomes
–
–
–
–
Incorporated into purpose statement
for cle: http://www.cle.prov.vt.edu/purpose.html
Charge to QEP from Provost
Alignment with SCHEV Core
Competencies
VT “Partner Institution” in VALUE
project
http://www.aacu.org/value/
VALUE METARUBRICS:
Assessment toolkit
VALUE project: Connections to VT

VT “Partner Institution” for Value
Project
–
–
–
Opportunity to use, provide feedback,
shape rubrics
Join Google Group with access to
rubrics as they are revised
Final rubrics – fall 2009
http://www.aacu.org/value/
VALUE METARUBRICS:
Assessment toolkit
VALUE project: Why META?????

Best thinking of higher ed faculty
professionals

Specific enough to be useful, broad
enough to be useful, meaningful in
multiple contexts, at multiple levels
for assessment
http://www.aacu.org/value/
To the METARUBRICS!
1.
2.
3.
4.
5.
6.
7.
Civic engagement
Creative thinking
Critical thinking
Ethical reasoning
Foundations and skills
for lifelong learning
Information literacy
Inquiry and analysis
8.
9.
10.
11.
12.
13.
14.
Integrative learning
Intercultural
knowledge and
competence
Oral communication
Problem solving
Quantitative reasoning
Teamwork
Written communication
To the METARUBRICS!



Identify learning outcome(s) of
interest to program?
Which essential learning
outcome(s) are in alignment with
program goals/outcomes?
Which rubric(s)/elements of the
rubric(s) can be incorporated into
assessment efforts?
The Big Message



Avoids Re-Inventing the wheel,
staring at a blank page
Potential connections to other
campuses, best practices for using
metarubrics
More sophisticated, authentic
approach to capturing information
on our students’ learning
Rubrics help us to Reflect
Before we React to our work
“We are being pummeled by a deluge of
data and unless we create time and
spaces in which to reflect, we will
be left with only our reactions.”
– Rebecca Blood
Questions/Comments?
(Please take a Few minutes to
complete the session feedback
form. Thanks!)