Transcript The Institutional Proposal and Beyond

CSU Channel Islands
CSU Sacramento
University of the Pacific
Student Affairs Assessment:
Using Student Learning Outcomes for
Continuous Improvement
Evaluating Institutional Learning-Centeredness Conference
Center for Educational Leadership, Innovation, and Policy in
Postsecondary Education
Department of Administration, Rehabilitation and
Postsecondary Education, SDSU
July 13, 2007
1
CSU Channel Islands
Sacramento State
University of the Pacific
Workshop Presenters:
Elizabeth Griego, VP for Student Life, Pacific
[email protected] (209) 946-2365
George Morten, Asst VP for Student Affairs, CSUCI
[email protected] (805) 437-8510
Lori Varlotta, VP Student Affairs, Sacramento State
[email protected] (916) 278-6060
2
CSU Channel Islands
CSU Sacramento
University of the Pacific
Learning Outcomes for Conference Participants
This session will help participants:
• Understand how the University’s mission, values,
traditions, and aspirations guide the co-curricular
assessment effort
• Articulate the potential benefits of assessment in
student affairs
• Write direct learning outcomes that measure
learning goals
• Identify useful types of assessment and evidencegathering tools
3
CSU Channel Islands
Sacramento State
University of the Pacific
4
CSU Channel Islands
Sacramento State
University of the Pacific
Potential Value of Assessment in
Student Affairs :
• Target the learning we expect students to achieve via their
participation in the co-curriculum
• Demonstrate the value-added nature of our programs and
the difference they make in the lives of students
(accountability)
• Focus staff on learning rather than on mere satisfaction with
services; on results rather than number of activities
• Build a culture of inquiry in student affairs
• Engage creatively what really matters
• Become a partner with academic affairs in the learning
process
• Help programs continuously improve
5
CSU Channel Islands
Sacramento State
University of the Pacific
What is learning?
For the purposes of this workshop, learning is
defined as:
“a comprehensive, holistic, transformative activity that integrates
academic learning and student development, processes that
have often been considered separate, and even independent
of each other. When we say learning then, we do not mean
exclusively or primarily academic instruction, the acquisition
of disciplinary content, or classroom learning.”
Learning Reconsidered: A Campus-wide Focus on the Student
Experience
6
CSU Channel Islands
Sacramento State
University of the Pacific
Process for Measuring Learning
1. Identify expected outcomes
2. Plan programs and services that accomplish
outcomes
3. Implement programs and services
4. Assess the learning responsive to the outcomes
through sound data collection and interpretation
5. Use results to improve learning opportunities
7
What is assessment?
CSU Channel Islands
Sacramento State
University of the Pacific
“Assessment is an ongoing process aimed at
understanding and improving student learning.
It involves making our expectations explicit and public;
setting appropriate criteria and high standards for
learning quality;
systematically gathering, analyzing, and interpreting
evidence to determine how well performance matches
those expectations and standards
and using the resulting information to document, explain,
and improve performance….
8
Organizational Context
ASSESSMENT LEVELS
Institutional Level
Division Level
Department Level
Program Level
Modes of Delivery
CSU Channel Islands
Sacramento State
University of the Pacific
EXAMPLES
WASC Review, OCR Review, etc.
Division of Student Affairs, Assessment
of Strategic Plan
Housing & Resident Life, Athletics, etc.
Department level Reviews
Student Leadership Training, Visual
Impaired, etc. Program Review
Workshops, Presentations, Role-plays,
and others strategies
9
CSU Channel Islands
Sacramento State
University of the Pacific
Writing Good Learning Outcomes:
• Be clear about what you are assessing: Am I assessing program
learning outcomes or student learning outcomes?
• Clearly state what you want students to know and be able to do.
• Make sure that your outcomes link to mission, values, strategic
plan.
• Ensure that your outcome is measurable: Are my outcomes
observable or operationally defined?
• Make sure that your outcome is meaningful: Assessment makes
a difference when it begins with issues of use and illuminates
questions that people really care about (9 principles of good Practice
for Assessing Student Learning (AAHE 1998)?
10
Components of Learning Outcomes
• Audience: For whom is the program aimed?
• Behavior: What do you expect the audience to
know/be able to do?
• Conditions: Under what conditions or
circumstances will the learning occur?
• Degree: How much will be accomplished, how
well will the behavior need to be performed, and
at what level?
An Example of a Learning
Outcome
Students who successfully complete the
communications workshop will be able
to 1) demonstrate giving and receiving
accurate feedback during a role-play
exercise and 2) score at least 75% on a
feedback survey administered by a
facilitator.
12
An Example of a Learning
Outcome
Students who successfully complete this
Problem Solving Workshop will be able
to 1) demonstrate knowledge of
effective problem solving using a pencil
paper exam, and 2) correctly identify the
three elements of problem solving:
empathy, invention, and selection.
13
An Example of a Learning
Outcome
Roommates who successfully complete
this exercise will be able to
demonstrate knowledge of their
roommate’s salient life experiences,
cultural values, and unique likes and
dislikes by correctly identifying 80%
of the correct answers found on their
Roommate’s Characteristics Checklist.
14
CSU Channel Islands
Sacramento State
University of the Pacific
Indirect Learning Outcomes
Indirect Learning Outcomes - self-reported statements/
comments that reveal a perceived increase in
understanding or appreciation. The perception is not
verified through any demonstration of knowledge
acquisition or observed behavioral / attitudinal change.
15
CSU Channel Islands
Sacramento State
University of the Pacific
Methods to Assess Indirect
Learning Outcomes
•
•
•
•
•
•
Satisfaction surveys
Program / evaluation surveys
Questionnaires
Inventories
Face-book response
Informal peer-to-peer conversations (e.g. with RAs,
orientation leaders)
16
CSU Channel Islands
Sacramento State
University of the Pacific
Direct Student Learning Outcomes
Direct Student Learning Outcomes - Demonstrate the
abilities, information, knowledge, attitudinal or behavioral
changes student can demonstrate after participating in a
program or utilizing a service
17
Methods to Assess Direct
Learning Outcomes
•
•
•
•
•
•
•
•
•
•
•
•
CSU Channel Islands
Sacramento State
University of the Pacific
Portfolios
Capstones
Performances
Common assignments
Narratives with reflection
Tests
Competency observations
Observations of employee or student behaviors
Juried art exhibits
National licensure examinations
Standardized tests
Work samples
18
CSU Channel Islands
Sacramento State
University of the Pacific
Differentiating Direct and Indirect
Learning Outcomes
Direct or Indirect?
1)
T / F This orientation session has helped me understand the
foreign language at Sacramento State.
2) Which of the following examples fulfill the foreign language
requirement necessary to graduate from Sacramento State?
A) Demonstrated fluency in a language other than English
B) Passed the AP foreign language exam with a score of 3 or
higher
C) Successfully completed, with C- or better, 3 years of high
school foreign language
D) All of the above
19
CSU Channel Islands
Sacramento State
University of the Pacific
Changing Indirect Learning Outcomes to
Direct Learning Outcomes
Distribute Orientation Survey Handout
Initiate group activity
20
CSU Channel Islands
Sacramento State
University of the Pacific
Direct Student Learning Instrument
Distribute Post-Test Orientation Assessment Handout
21
CSU Channel Islands
Sacramento State
University of the Pacific
“Best Advice” Examples
22
Resources
CSU Channel Islands
Sacramento State
University of the Pacific
WASC website http://www.wascweb.org
CAS Standards http://www.cas.edu/gags.html
Kansas Study Website http://Kansasstudy.org
National CC Benchmark Project http://www.NCCBP.org
Powerful Partnerships: A Shared Responsibility for
Learning http://www.aahe.org/assessment/joint.htm
Student Learning Imperative
http://www.acpa.nche.edu/sli/sli.htm
Association of Theological Schools http://www.ats.edu
Learning Reconsidered, National Association of Student
Personnel Administrators (NASPA) http://www.naspa.org
23