Transcript NCA Site Team Recommendations

Learning Connections 24/7:
Assessing Learning across Curricular, CoCurricular and Student Development Programs
Margaret Bloom, Vice Provost
Christine Krueger, Core of Common Studies
Mark McCarthy, Student Affairs
Susan Mountin, Manresa Project
International Assessment and Retention Conference,
St. Louis, MO, June 8, 2007
Marquette University



Research Extensive
Private, faith-based
Mission driven
“Marquette University is a Catholic, Jesuit university.
Our mission is the search for truth, the discovery and
sharing of knowledge, the fostering of personal and
professional excellence, the promotion of a life of faith,
and the development of leadership expressed in service
to others.”
Learning Assessment at Marquette
The Marquette assessment system is
focused on two essential learning outcomes:

Institutional learning outcomes which address the
integrated learning expected of every student upon
graduation.

Program (undergraduate major and graduate
degree) learning outcomes expected of every
student at program completion.
Framework for Undergraduate
Learning Assessment
Institutional
Learning Outcomes
Major/Program
Learning Outcomes
College Competencies
Discipline Competencies
Required Courses
Learning Objectives
Common Core
Learning Outcomes
9 Knowledge Areas
Competencies
Co-Curricular
Learning Outcomes
Values, Commitments &
Leadership
Services/Programs
Core Courses
Learning Objectives
Governing Principles of the
Marquette Assessment System

Common Definitions and Assessment Process
Each program and unit uses the common MU assessment
system vocabulary and has an assessment plan that
contains the specified MU common assessment system
components.

Program and Unit Flexibility and Decision-Making
Each program faculty/unit staff define learning outcomes,
performance indicators, identify and use measures, and
evaluate their data.
MU Assessment System Components
MARQUETTE MISSION
CONSTITUENTS
LEARNING OUTCOMES
FALL 2005
ACTIONS TO IMPROVE
LEARNING
Fall 2007
FEEDBACK FOR
CONTINUOUS
IMPROVEMENT
EVALUATION
INTERPRETATION
OF EVIDENCE
August 2007
Modified from Rogers, G. Rose-Hulman Institute of Technology, 2005
MEASURABLE
PERFORMANCE
INDICATORS
DIRECT &
INDIRECT
MEASURES
Spring 2007
ASSESSMENT
COLLECTION,
ANALYSIS OF EVIDENCE
Late Spring 2007
MU Assessment Systems:
Integrated Levels of Assessment
Learning Outcomes
Actions to Improve Learning
Evaluate
Performance Indicators
Institutional
Measures
Collect & Assess
Undergraduate
Major
Business
Core
Business
Administration
Student
Major
Coursework
Core of Common
Studies
Student Affairs
Mission Related
Programs
Broad Participation a Key Factor
Faculty and Staff Directly Involved in Development of the
Marquette Assessment System
Institutional Level
University Assessment Committee
Board of Undergraduate Studies
Assessment Steering Team
36 Faculty/Staff/Administrators
Program
Assessment Leaders
108 Faculty
Core Curriculum
Review Committee
15 Faculty
Student Affairs and
Co-Curricular
Program Directors
16 Directors
Marquette (Institutional) Undergraduate
Learning Outcomes
Upon completion of an undergraduate degree at Marquette, students are able to…
1.
Utilize critical thinking and reflect to effect positive change in themselves,
others, and their communities.
2.
Exercise just, responsible, and competent leadership in professional,
intellectual, and societal contexts.
3.
Apply the knowledge and skills of an academic discipline, program, or
profession to a career or graduate study.
4.
Communicate in modes appropriate to various subjects and diverse
audiences.
5.
Pursue an integration of knowledge into a comprehensive, transcendent
vision of life.
6.
Act for social justice within the diverse, global human family.
Majors and Degree Programs
58 Majors
35 Masters Degree Programs
17 Doctoral Degree Programs
3 Professional Degree Programs
Majors and Graduate Programs

Each Academic Program’s Assessment Plan indicates direct
and indirect measures.

Located at www.marquette.edu/assessment/plans/index

Direct measures often include senior-level papers, case
studies, or projects.

Indirect measure: University Senior Survey contains up to 5
items for students to self-rate on their specific program learning
outcomes.
1st Annual Program Assessment Reports due Sept. 31, 2007
Core of Common Studies
Assessment and
Integration
University Core of Common Studies
Integrated Learning Outcomes
Upon completion of the Core of Common Studies, students are able to…
1.
Apply the perspectives, concepts and traditions of multiple
disciplines to personal, professional, intellectual and societal
challenges.
2.
Communicate in modes appropriate to various subjects and
diverse audiences.
3.
Pursue an integration of knowledge into a comprehensive,
transcendent vision of life.
4.
Act as responsible members of the global human family, with
knowledge of, and respect for, individuals and cultures in their
diversity.
Pilot Core Assessment
On-line Integrated Assessment Instrument
Integrated foundational competencies for responsible leadership

Reading comprehension of brief text offering multidisciplinary
analysis of global problem

Interpretation of quantitative and qualitative evidence via
multiple-choice questions seeking best answer

Communication of diverse content to diverse audiences via 3
randomly assigned tasks:

Summary for senator

Letter to editor of non-US student newspaper

Graph illustrating data extrapolated from reading
Pilot Core Assessment Results

25 student volunteers (17 completed)

Preliminary Results


Quantitative/qualitative evidence questions (12%-71% outstanding)

Prose communication (80% > proficient)

Graphic communication (20% proficient)
Understanding the Results


Student focus group
Actions

Refinement of instrument and rubrics

Implementation for incoming seniors

Math across the curriculum??
Student Affairs Assessment
Counseling Center
Office of Residence Life
Recreational Sports
Office of Student
Development
Student Health Services
Student Affairs Division
Shared Learning Outcomes
As a result of their experiences with the Student Affairs
services and programs, students are able to…
1.
More clearly understand who they are in regards to mind,
body and spirit.
2.
Utilize their skills to create positive change in themselves,
others and their communities.
3.
Collaborate with others to accomplish goals.
4.
Engage with others who are different from themselves.
Office of Student Development
Learning Outcomes
After participating in student development
programs students are able to…

Use their talents to benefit others.

Interact effectively with people from diverse
backgrounds and lifestyles.
Population Based Assessment

Multi-Institutional Study of Leadership (Spring 2006)


Based on the Social Change Model of Leadership (HERI, 1996)
Two groups participated in the study

Random sample of MU undergraduate students

Select sample of students involved in Student Affairs
sponsored leadership and service experiences.
MSL Initial Findings
Related to Student Development Outcomes
No mean difference reported between the random and
student affairs involved groups on pre-test scales for:



Cognitive Development
Leadership Efficacy
Appreciation of Diversity
Significant mean differences found between groups on
the scales for:




Citizenship,
Leadership Efficacy,
Diversity Discussions
Appreciation of Diversity
Program Based Assessment
Leadership Summit
One-day urban experience followed by keynote address on
integrating faith and learning in action and reflection activities
Assessment Results
90%
•
•
•
stated that as a result of participation I…
Am more likely to take action to address needs and problems in the
community.
Have a greater appreciation for different perspectives.
Am more likely to educate others on issues affecting the Milwaukee
community.
Key findings related to outcomes
Leadership Summit assessment results
80%
stated that as a result of participation I…
• Better understand the importance of reflection.
• Learned about methods that leaders use in creating change.
Learning Themes Identified in statements of reflection
Deeper understanding of the broader Milwaukee community.
Increased commitment to serve others.
Greater understanding of themselves and incorporation of values
such as faith, justice, compassion, empathy and hope into work
and life.
Assessment Challenges

Whom to assess

Controlling for pre-college experiences and
perceptions

Alignment of program goals with assessment
strategies

Impact of singular experiences on student
development

Complexity of developing and using direct measures
Integrative Programs
Mission Related Programs
Manresa Project
Mission Related Programs
The Manresa Project
After participation in Manresa Project activities students are able to…
1.
Understand the difference between discernment and
decision-making.
2.
Practice Ignatian Spirituality as a component of their faith
and life choices.
3.
Understand Jesuit Global Mission ideals.
4.
Explore individual meaning and purpose in life (vocation).
Assessment Tools Used
Direct measures

Spiritual Autobiography

Site-supervisor observations

Course assignments

Weekly group reflection
Indirect measures

Surveys: Multi-Institutional Study of Leadership (MSL)

Interviews

Focus groups
The Manresa Project
Indirect Assessment Results 2006

Used the MSL data comparing Marquette students with
students at other Catholic universities

Marquette students significantly higher on items
(outcomes)

Developing understanding of social justice

Strengthening spiritual life

Defining values

Becoming more independent

Knowing how “God has called me” to use my gifts and talents

Integrating service into life
The Manresa Project
Assessment Challenges

Program has many diffuse opportunities for
engagement






First year reading
Manresa supported courses
Sponsored speakers
Seminars, workshops and conferences
Retreats
Data collection difficult


Relies on external partners (faculty, site supervisors,etc.)
Direct contact with small number of students, but a broad
impact across the Marquette community
Assessing 24/7 Learning:
Unanticipated Cultural Changes

Faculty working across disciplines on
integrating global understandings:
 Shared
Futures Project with AAC&U

Assessment has been an integrating
influence across campus

Inclusion of Mission based learning in the
common understanding of Marquette
student learning