Transcript Learning Outcomes Assessment in Student Services

Assessment of Student Learning at
Frederick Community College
Evaluating Institutional Learning
Centeredness Conference
San Diego, California · July 2007
Presenters
• Dr. Richard Haney
Vice President of Learning Support
• Dr. Debralee McClellan
Associate Vice President for Student Development
About Frederick Community College
• Located in central Maryland
– 45 miles west of Washington D.C. and Baltimore
– Northern end of the I-270 high-tech corridor
– County population of 220,000
– Largest county by land mass in the State
• Enrollments
– 4,800 credit students per semester
– 11,000 continuing education students annually
About FCC
•
•
•
•
•
Average student age of 27
62% attend part-time
19% are students of color
61% are enrolled in transfer programs
Annual operating budget of $36 million
Genesis for Assessment
National focus
• For 10 years, professional organizations have underscored the need to
assess student services
• NASPA conferences – emphasis on learning outcomes assessment in
student services
• Middle States Commission on Education’s Standards of Excellence
Genesis for Assessment
State focus
• Maryland Higher Education Commission (MHEC)
– State accountability reports
Genesis for Assessment
State focus
• Maryland Association of Community Colleges (MACC) Affinity Groups
• Chief Student Services Officers
– Registrars and Admissions Officers
– Financial Aid
– Athletics
– Student Life
– Academic Advising/Counseling
– Testing Center
– Career Counseling
– Tutoring
Genesis for Assessment
State focus
• Maryland’s Community College Deans/VP of Student’s Learning
Outcomes Project
– Developed process for statewide assessment
– Contracted with Dr. Marilee Bresciani to guide process
– Conducted series of workshops for Deans/VPs and affinity group
members
– Developed core goal – “Self-directed learner” to be applied by the
affinity groups
Context for Assessment
College focus
• FCC’s re-organization as a “Learning College”
– Vision – “Student Learning First”
– Mission – “FCC, as a learning college prepares individuals to meet
the challenges of a diverse, global society through quality,
accessible, innovative, lifelong learning. We are a studentcentered, community-focused college. FCC offers courses,
degrees, certificates, and programs for workforce preparation,
transfer, and personal enrichment. Through these offerings, FCC
enhances the quality of life and economic vitality of our region.”
Context for Assessment
College focus
• Division of Student Development became Division of Learning
Support
• Learning Support mission statement – “To provide services in a
changing, professional environment that encourages and supports lifelong
learning.”
Learning Support
Organizational Structure
•
•
•
•
•
•
•
Athletics
Enrollment Management
Financial Aid
Information Technology
Student Development
Student Life
Welcome & Registration Center
Overview of Assessment Process
in Learning Support
• College’s transition to a learning college shifted the focus of
assessment to student learning
– Shift from student satisfaction to student learning
– Movement away from evaluating programs by the numbers to
measuring the learning that has occurred as a result of the student’s
involvement in the program and/or services
– Forced departments to re-think their true purpose
– Emphasis shifts from what we do to what we want students to be able
to do
What we did
• Division-wide retreats in 2003 – “The Student Learning Imperative”
and “Outcomes Assessment in the Learning College”
Key questions we asked of ourselves as we developed our
assessment plan
• What are we trying to do and why?
• How does my program contribute to student learning?
• How well are we doing?
• How do we know?
• How do we use the information to improve or celebrate successes?
• Do the improvements we make work?
What we did
• Provided professional development and encouraged staff to
become active participants in the statewide affinity group
• Created assessment template as a tool to provide structure
and to facilitate the development of a comprehensive
assessment process for each unit in Learning Support
Learning Support Assessment Plan FY
AREA:
GOAL/OBJECTIVE:
EXPECTED OUTCOMES
1.1
1.2
MEASUREMENT
1.1
1.2
BENCHMARK
ASSESSMENT TOOLS
Learning Support Assessment Plan FY - continued
STRATEGIES
ASSIGNED TO
1.1
1.2
STATUS/FEEDBACK LOOP (How has data been used to enhance learning, services, processes?)
What we did
• Used the statewide goal of a self-directed learner as a model for
developing learning outcomes across functional areas
– Managers charged to develop assessment objectives for their
program areas to address the goal - “Student will become a selfdirected learners”
Definition of Self-directed Learner
The student will:
• Define a need or problem and employ effective
decision-making to resolve it
• Plan ahead/set goals
• Acquire knowledge
• Use available resources
• Seek assistance from appropriate people/experts
• Apply critical analysis to consider options
• Evaluate decisions
Examples of Learning Outcomes in
Learning Support Areas
• Athletics
– Goal - Assist students in becoming self-directed learners by promoting
development of sportsmanship and citizenship skills
– Learning Outcome - Student-athletes exhibit good sportsmanship and
citizenship skill
• Welcome & Registration Center
– Goal -Assist students in becoming self-directed learners by enhancing
one-stop operations
– Learning Outcome - Students demonstrate the ability to navigate
registration processes in subsequent semesters
Examples of Learning Outcomes in
Learning Support Areas
• Financial Aid
– Goal – By providing information, assistance, and directions to students
regarding sources of financial aid and the application process students
will more effectively negotiate the financial aid process
– Learning Outcome – Students will demonstrate increased awareness of
financial aid opportunities
– Learning Outcome – Students will receive an increased number of
Maryland State Scholarship Awards
Incorporating Assessment into
Student Affairs Programming
Academic Advising
Steps in Assessment
1. Develop Functional Area Goal(s)
– Goal should be broadly defined
– What is the overall purpose of the area?
2. Develop Outcomes that Operationalize the Goal
– What do we want students to know or to be able to do?
– Must be measurable – how will you know if student
accomplished the outcome?
Steps in Assessment
3. Develop Strategies to Ensure Outcomes will be Met
– Services, resources, programs offered
– Opportunities for students to accomplish what you want
them to do or to learn what you want them to know
4. Establish Benchmarks
– The level of student accomplishment will you accept as
evidence that students have met the outcome
Steps in Assessment
5. Develop Assessment Instrument to Measure each
Outcome
– Survey, focus group, portfolio of student work, rubrics,
pre and post measurement
6. Use Findings to Make Improvements
– Can include changing a program or a service, or
developing a new service to address assessment findings
Assessment of Academic Advising
• Advising Goal – Students will become self-directed learners
• Advising Outcomes – Students will:
– Demonstrate knowledge of academic requirements
– Articulate an educational goal
– Develop an educational plan outlining steps necessary to
reach their goals
– Demonstrate knowledge of educational policies and
procedures
– Demonstrate awareness of how to access college resources
and services
Assessment of Academic Advising
• Advising Strategies to Address Outcomes
– First Year Advising Program
– ICAP – Individualized College Academic Plan
• tool which helps an academic advisor to frame an
advising session within the context of goal
development and establishment of an educational plan
Assessment of First Year Advising
Program
• Applied Advising Outcomes to First Year Advising
Program
– Assessment instrument created to measure attainment of
outcomes
– Questions developed to address each advising outcome
• What would student have to be able to do or to know in
order for us to feel they met the outcome?
• Grouping survey items by outcome gave us the ability to
evaluate each outcome independently
Sample Outcome/Survey Items
Advising Outcome
• Students will demonstrate
knowledge of academic
requirements
– e.g., curriculum for intended
major, how to select general
education courses
Survey Items
• “I know how to select general
education courses”
• “I understand how to select courses
for my major”
• “I understand how placement test
results determine which classes I
can take”
Assessment of First Year Advising
Program
• Highest mean scores were obtained for outcomes relating to
student’s:
– Knowledge of academic requirements
– Ability to articulate an educational goal
– Ability to develop an educational plan outlining steps needed
to reach their goal
Assessment of First Year Advising
Program
• Lowest mean scores were obtained for outcomes relating to
student’s:
– Knowledge of educational policies and procedures
(related to the registration process)
– Awareness of how to access college resources and services
(related to setting career goals)
Assessment of First Year Advising
Program
• Evaluated effectiveness of ICAP
– Reviewed scores obtained on survey items related to
effectiveness of goal setting, degree of assistance obtained
with goal setting and educational planning
– Compared to baseline scores from prior years
Spring 2005 – Pre-ICAP
Spring 2006 – ICAP Piloted
Summer 2006 – ICAP fully implemented
Effectiveness of the ICAP
Survey Items
Mean Scores
• “I feel that I was helped with
establishing realistic academic
goals”
– Spring 05
3.68
– Spring 06
3.73
– Summer 06 3.81
• “My advisor was able to help me to
start to develop an educational plan
to meet my goals”
– Spring 05
3.52
– Spring 06
3.72
– Summer 06 3.81
Using Results to Improve Practice
• Reviewed outcomes and individual survey items that
received lowest scores from students in 2006 to revise First
Year Advising Program for 2007
– Focused on two lowest rated outcomes and the lowest rated
survey item within a highly rated outcome
Using Results to Improve Practice
Outcomes to be Addressed
• Students will demonstrate
knowledge of academic
requirements (general education
courses)
• Students will demonstrate
knowledge of educational policies
and procedures (related to
registration)
• Students will be aware of how to
access college resources and
services (career goal setting)
Strategy Implemented
• Developed consistent delivery
regarding how to select general
education courses
• Implemented Registration
Services module
• Implemented Career Development
module
Assessment of Student Leadership
• Goal – Students will become self-directed learners
• Outcome – Student leaders will develop competency in six areas:
• personal development
• organizational awareness
• community awareness
• communication skills
• interpersonal skills
• critical-thinking/problem solving skills
• Assessment tool – Co-curricular portfolio
• Measurement – Evaluation of portfolio using leadership rubric
Building a Culture of Assessment
• Set expectation that assessment is a requirement of each
manager’s and department’s annual performance
• Annual Assessment Report required for each department
• Learning outcomes published in College catalog
• Results reported to Student Learning Outcomes Assessment
Council
• Tie assessment results to the budget process
• Assessment is a formative process – the process and the
findings should not be feared
Did it work?
• Middle States Commission on Education Accreditation Team
Feedback – March 2006
“The Division of Learning Support deserves enthusiastic
commendation for their work with outcomes assessment. There are
articulated assessment plans in place that include clear functional
area mission statements, maintain realistic benchmarks, and
demonstrate evidence of continuous quality improvement. The model
created by this division should serve as a standard.”
What we learned
• Provide professional development on assessment of student
learning for staff
• Involve staff at the beginning of process to facilitate buy-in
• Template was an asset in helping staff understand assessment
• Encourage staff to read the literature on assessment
• Provide time to do the work
• Keep process as streamlined as possible
What are the challenges?
• Change in staff and acclimating new staff to assessment
• Ensuring results lead to improvements
• Interfacing departmental assessment with the College’s
strategic plan
• Expertise in research and data analysis
– Office of Institutional Research resources
– Variability of skills of individual staff
Where do we need to go?
• Deepen the staff development program
• Have managers present their assessment results on an
ongoing division-wide basis
• Review and update assessment plans
• Expand assessment plans to address additional “student
learning” outcomes in Learning Support
• Develop student learning outcomes in Learning Support
based upon the College’s general education program
outcomes
• Infuse Learning Support efforts into the institutional
effectiveness process
Contact Information
[email protected]
[email protected]