Transcript learning outcomes - Gavilan College

Student Learning Outcomes for
Student Services
ACCCA
Santa Clara, CA
February 2005
Terrence Willett
Director of Research
Gavilan Community College
Acknowledgements





Gavilan Student Services Division especially:
– John Baker, Margery Regalado, Jane Maringer
Gavilan Instructional Division especially:
– Marty Johnson
– Marlene Dwyer & her Curriculum Committee
– Department Chairs
– Academic Senate, Enrique Luna
Research and Planning Group especially:
– Harriett Robles, Brad Philips, Fred Trapp, Bob Gabriner, Jerry
Rudman
Other colleagues:
– Rachel Ruiz, Mesa College
– Janet Fulks, Kate Pluta, Sue Dixon, Bakersfield College
WASC
– Darlene Pacheco, Barbara Beno
Learning Outcomes for Today






Summarize history of accountability and how new standards
came to be
Define learning outcomes and explain the underlying
philosophy
Describe the assessment cycle and its role in the college
Write appropriate student learning outcomes for your
program
Explain at least two methods to assess at least one student
learning outcome including needed resources and potential
barriers
Identify at least 3 resources to help you succeed in the
learning outcomes challenge
Your expectations and needs

1960’s
History
– In the past, year-end reports demonstrated proper
management but not whether the programs directly aided
student learning
– Colleges measured enrollments, participation rates,
budget expenditures

1990’s
– More recently have expanded to measure success,
retention, persistence, graduation, and transfer
– Focus has also been on instructional delivery versus
student learning

2000’s
– Student Learning Outcomes
Standard IIB

The institution recruits and admits diverse students who are
able to benefit from its programs, consistent with its mission.
Student support services address the identified needs of
students and create a supportive learning environment. The
entire student pathway through the institutional experience is
characterized by a concern for student access, progress,
learning, and success. The institution systematically
assesses student support services using
student learning outcomes, faculty and staff
input, and other appropriate measures in order
to improve the effectiveness of these services.
What are Learning Outcomes?

“Learning outcomes” not a new concept
– Used in California in K-12, special ed, voc programs
– Used in other college systems (i.e. Maryland, England)
– Barr and Tagg called for learning outcomes in 1995
Focuses more on students’ learning rather than just
how we instruct or support them; outcome based
 Emphasis on higher more complex level of learning
 Clearly states educational intentions for students

Sounds like behavioral objectives!

Learning outcomes differ from behavioral objectives
by
– Being broadly stated
– Having instructors assess gains in skills rather than
creating detailed list of specific topics and abilities to be
mastered (Harden 2002)
Some authors do not perceive that the difference
between learning outcomes and behavioral
objectives is important (Prideaux 2000)
 Important to distinguish the two to gain acceptance

6 Principles – O’Banion
The Learning College…
…creates substantive change in individual learners
 …engages learners in the learning process as full partners
who must assume primary responsibility for their own
choices
 …creates and offers as many options for learning as possible
 …assists learners to form and participate in collaborative
learning activities
 …defines the roles of learning facilitators in response to the
needs of the learners
 …succeeds only when improved and expanded learning can
be documented for learners

Levels of Analysis
Institutional
 Department or Program
 Class or Service
 Assignment or Student Interaction

New Perspective
“We must treat the college as a learning laboratory”
– John Baker, VP SS Gavilan
 Dealing with whole student-their uncertainty,
response to that uncertainty, and their wisdom and
courage-all matters of being (Harden 2002)
 Shift from instruction as “core of the college”
towards learning as central focus
 True marriage between academic and life skills
learning

Institutional Concerns
Students have a fundamental right to services and if
these services are not or cannot be related to
learning outcomes, their existence could be unfairly
jeopardized (Collins 2002)
 Uncritical application of business models and
concepts to education
 Fear all this measuring will be a waste of effort and
not enhance student achievement – Luna, Gavilan
College Academic Senate
 Do students really care about learning? - Borden

Gavilan’s Approach






Discussion began in Student Services
Had SLO presentations at convocation and
meetings of department and program leads as well
as attending external workshops
Institutional learning outcomes selected
Course level learning outcomes written
Currently developing program level outcomes
Next steps include publishing outcomes in catalog
and implementing assessment of outcomes
Gavilan Institutional
Learning Outcomes
(from Palomar College)
Communication
 Cognition
 Information Competency
 Social Interactions
 Aesthetic Responsiveness
 Personal Development and Responsibility

Gavilan Institutional
Learning Outcomes

Communication
– Listening
– Reading
– Writing
Gavilan Instutional
Learning Outcomes

Cognition
– Analysis & Synthesis
– Problem Solving
– Creative Thinking
– Quantitative Reasoning
– Transfer of Knowledge & Skills to a New
Context
Gavilan Instutional
Learning Outcomes

Information Competency
– Research
– Technological
Gavilan Instutional
Learning Outcomes

Social Interaction
– Teamwork
– Effective Citizenship
Gavilan Instutional
Learning Outcomes

Aesthetic Responsiveness
– Differentiate between people who can sing
and people who can’t

e.g. Brittney Spears vs. Norah Jones
– Distinguish between art that offends you, art
that is cheesy, art you can’t understand, and art
that is too expensive

Gavilan Instutional
Learning Outcomes
Personal Development and Responsibility
– Students will develop individual responsibility, personal
integrity, and respect for diverse people and cultures
– Self-management: Students will demonstrate habits of
intellectual exploration, personal responsibility and
physical well being
– Ethics and Values: Students will demonstrate an
understanding of ethical issues that will enhance their
capacity for making sound judgments and decisions
– Respect for Diverse People and Cultures: Students will
respect and work with diverse people including those
with different cultural and linguistic backgrounds and
different abilities
Service →
Learning
Outcome ↓
Counseling
Student
understanding and
explaining
Communication
goals/plans; use
advice from
counselor
Understand GE
Cognition
patterns; Develop ed
plan
Information
Competency
Social
Interaction
Financial Aid
Admissions and Records
Student discussing
Student explaining what
and understanding
is desired: registering,
requirements with staff transcripts, adding, etc.
Calculating income,
need, and unit loads
Calculating units and
fees
Using web to gather Comprehension of
Comprehension of forms
program information forms and instructions and instructions
Working with college Working with college Working with college
personnel
personnel/mentoring personnel/mentoring
Form design, catalog,
Aesthetic
Form design, web
Form design, web
schedule web pages
Responsiveness page layout
page layout
layout
Personal
Understand concept Provide accurate
Provide accurate
Development
of balancing school, information; meet
information; meet
And
work, and family
deadlines
deadlines
Responsibility
Information
Competency
Social
Interaction
Aesthetic
Responsiveness
Cognition
Communication
Personal
Development
20
18
16
14
12
10
8
6
4
2
0
Your Turn!
Think of an instance you directly observed a
student learning while interacting with your
program or service outside of a classroom
experience
 Write a mission statement for your program
or service

Writing Learning Outcomes
Should be based on our mission and values
 In an environment that nurtures creativity
and intellectual curiosity, Gavilan College
serves the community by providing a high
quality learning experience which prepares
students for transfer, technical and public
service careers, life-long learning, and
participation in a diverse global society

Writing Learning Outcomes
Outcomes must be observable so you can
measure or record them
 Course level learning outcomes sound like
behavioral objectives (or really are the same)
but LO tie into institutional learning
outcomes
 Keep in mind that outcomes are tools to
guide evidence collection for assessment

Assessment Cycle
Create or Modify Goals
and Outcomes
Evaluate
Implement
Programs or
Strategies
Dimensions of Evidence

Quantitative or qualitative
– Not everything that can be counted counts and not everything that
counts can be counted -Einstein
Direct or indirect
 Norm- or criterion-referenced
 Should be representative and relevant
 Need several pieces of evidence to point to a conclusion

– e.g. Student complains of fever and aches, their temperature is 102º
F, tonsils are not inflamed, eyes are red and irritated, posture appears
weak. Notice mix of types of evidence that all point to same
conclusion…flu!
Common Assessment Methods







Tests
– Locally developed or Standardized
Performances
– Recital, Presentation, or Demonstration
Cumulative
– Portfolios, Capstone Projects
Surveys
– Attitudes and perceptions of students, staff, employers
Database Tracked Academic Behavior
– Grades, Graduation, Lab Usage, Persistence
Embedded Assessment
– Using grading process to measure ILO
Narrative
– Staff and student journals, interviews, focus groups
Follow Up Counseling
Program Assessment Example
Beginning Fall 2000, students on academic dismissal
are required to visit a Follow Up counselor before
enrolling
 The main goals of follow up counseling include:

1) students raising their GPA above 2.0,
2) increasing the number of completed units,
3) reducing withdrawals,
4) encouraging re-enrollment in subsequent terms
Follow Up Assessment Details






68 students in Fall 2000
85 students in Spring 2001
25 of those students were seen both terms
105 visits resulting from outreach efforts consisting
of at least 153 letters, 115 cards to those not
responding to the letter and 53 calls to those not
responding to the card
In Fall 2000, 79% of students visited at least once
In Spring 2001, 66% of students visited at least
once
Outreach Efficacy
Visits
Outreach effort
0 1
2
3 4 5 Median
Letter Only
15 7
7
2
2
Letter and Card
3 37 15 6 1
1
Letter, Card, and Call 45 5
3
0
Unknown
6
1
note: median number of visits significantly differs using a
Kruskal-Wallis test (χ2(2) = 79.9, p < 0.0005).
Increasing efforts showed diminishing returns suggesting
sufficient outreach effort occurred
No relation between visits and GPA change


GPA Difference
0.20
0.00
























































gpa diff = 0.00 + 0 .01 * visits
R-Squa re = 0.01
-0.20


0
1
2
3
4
Follow Up Counselor Visits
5
Difference in Units Earned Unchanged
Uni ts Earned per Term

5.00

4.00
3.00
2.00
1.00
Before
After
Follow Up Services
Significant reduction in drops
Drops per Term
1.00

0.80

0.60
0.40
0.20
Before
After
Follow Up Services
Significant increase in persistence
Fall to Spring Persistence Rate
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Fall 2000 students Fall 2000 students Fall 1999 academic
contacted and
contacted but not dismissal students
visiting counselor visitng counselor (before mandatory
counseling)
Significant reduction in units attempted, which was not
one of the original objectives. Evidence of learning?
Units Attempted per Term

8.00

6.00
4.00
2.00
Before
After
Follow Up Services
Example
Method
Strength of
Evidence
Randomly
assign
students to
service
Can claim
causality
Denies access
to a service
Ethical
that may or
Consideration may not be
effective for
some students
Randomly
Correlate use
assign
with
students to
performance
receive
or skills
information
measures
about services
Causality
cannot be
claimed,
Weaker
useful in
causality claim
conjunction
with other
information
All students
have access
but some
receive less
information
Surveys
Causality
cannot be
claimed,
useful in
conjunction
with other
information
Use student
No restriction
time to
of access or
complete
information
survey
Case study and
journals
Causality cannot
be claimed but
complex and
difficult to
measure effects
can be noted
Confidentiality
Embedded Assessment Example
Criteria
Joe
Jane
Liam
Celeste
Avg
Arithmetic with
fractions
3
4
5
4
4
Calculating
basic areas
3
4
3
5
3.75
Order of
operations
3
4
5
5
4.25
Right triangles
3
4
4
4
3.75
Total
12
16
17
18
Student Grade
C
B
B
A
Down for grades, across for outcomes assessment
–after Nichols
Assessment Tips






Collect data from a representative sample rather
than everyone in population
Collect only a few well chosen pieces of data
Have assessment stem from activities you already do
if possible
Team up with others to share assessment work
Design activities and outcomes with assessment in
mind
Ask for advice from your colleagues - especially
vocational faculty who relate to learning outcomes
as core competencies
Writing Learning Outcomes

Good verbs (Blooms’ Taxonomy):
–
–
–
–
–
–
–

Compile
Analyze
Design
Apply
Demonstrate
Explain
Compare
Classify
Operate
Write
Calculate
Compose
Assess
Critique
Bad verbs
– Know

Arrange
Identify
Solve
Differentiate
Formulate
Predict
Estimate
Understand
Appreciate
Learn
Good verbs are clear and observable, bad verbs are
vague states of mind
Example Learning Outcomes at
program/service level

Disabled Student Services
– The student will be able to explain his/her individual
academic strengths and weaknesses

Maryland Health Education
– Students will demonstrate an understanding of health
promotion and disease prevention concepts to establish a
foundation for leading healthy, productive lives.

Admissions and Records
– Students can “work the system”
– Demonstrate patience while waiting in line
More Student Service Examples

From Ventura College SLO Workshop
– DSPS = Students will demonstrate self-advocacy skills
with instructors and staff
– EOPS = Student will identify and describe resources
available on the campus.
– Counseling = Student will be able to state informed
academic goal(s).
– Financial Aid = Faculty and staff will be able to describe
basic aspects of financial aid available to students and
how students can apply for financial aid.
Gavilan Counseling SLO’s

Students will identify college resources, procedures, and
policies that support their academic success. (Cognition)
– Students will complete the college orientation and score at least an
80% on the orientation quiz.

Students will use technology to gather, process, and articulate
career options and choices. (Info Comp)
– Students referred to the Eureka system (online/center) will complete
the assessments and discuss outcomes with counselor. (track
completion on SARS)
– Students enrolled in Guidance 1 will utilize the internet for research
and declare a major at the end of the course.
Instructional Program Level SLO
Examples

Sinclair College
– Math = Apply mathematical models to real world
problems
– Amer. Gov. = Communicate the significance of
facts, concepts, and ideas in spoken and written
English, which is clear, precise, and logical
– ECE = Demonstrate professionalism in the child
care setting
More Instructional PSLO

Parkland Community College
– English = Students will create college-level written text
for multiple purposes and diverse audiences that
demonstrate depth of critical thought and that observe
the conventions of standard written English.
– Psychology = Our students will apply major theoretical
perspectives and methodological approaches used in the
discipline of psychology to particular social phenomena
or autobiographical circumstances.
Your Turn!

Write at least 2 learning outcomes for your
program and at least 2 assessment strategies
for each outcome
Reflect on Your SLO’s:






Will the student understand?
Is this truly an overarching outcome?
Can you blend?
Does this take into consideration the other areas
your program interfaces with?
If you had only 6 outcomes for your program, is
this one of them?
In reality, can this be done?
Questions to Contemplate





Why should I use student learning outcomes for
evaluating my program?
What are some strategies for writing effective
learning outcomes?
What are some differences between qualitative and a
quantitative measurements?
How can anecdotal reports or case studies be used
in program assessment?
What are the pros and cons of using sampling
methods for assessment?
Thank You