Transcript Assessing Core Learning in Study Abroad

Assessment of Learning Objectives
in Study Abroad
An Integrative Model
Diane Jonte-Pace, Santa Clara University
Barbara Molony, Santa Clara University
Joan Gillespie, IES
IES April 8, 2010
Assessment and accreditation
 New expectations for accountability/ transparency /assessment
– National regulations
– Regional accrediting bodies
 Change is hard
– Challenges on university campuses
– Further challenges face Study Abroad
 SCU offers meaningful & manageable assessment model
– Designed during re-accreditation process
– Continues in systematic multi-year cycle
A proposal
 Step 1: Fall 2010 Pilot
– Integrate IES Rome into SCU Core assessment plan
– One Religious Studies course
 Step 2: Future years
– Integrate SCU model into IES practices more broadly
Accreditation and Assessment
Accreditation Agencies request assessment of student learning






Middle States (MSCHE)
New England (NEASC-CIHE)
North Central (NCA-HLS)
Northwest commission (NWCCU)
Southern Association (SACS)
Western Association (WASC
Examples
 The institution’s student learning outcomes and expectations for student
attainment are clearly stated at the course, program and, as appropriate,
institutional level. WASC Standard 2.3
 The institution’s faculty takes collective responsibility for establishing,
reviewing, fostering, and demonstrating the attainment of expectations for
student learning and attainment. WASC Standard 2.4
Key Concepts for SCU
 Goals and Objectives
– Broad Goals
– Goals
– Measurable Learning Objectives
 Course level & program level
 Explicit assignment mapping
 Direct vs. Indirect assessment
 Program Level Assessment
 Rubric Scoring vs Grading
 Systematic multi-year assessment plan
Similar to IES Key Concepts
 Learning Outcomes
– Each syllabus should list the expected outcomes for the
course, i.e. what the student is expected to know and
be able to do upon completion of the course.
 Required Work & Assessment
– The required work and content should be clearly linked
to learning outcomes.
 Assessment
– Grading, percentages
 Other IES Concepts and Practices
Broad Learning Goals SCU
Knowledge

Global Cultures
Habits of Mind and Heart

Arts & Humanities

Critical Thinking

Scientific Inquiry

Mathematical Reasoning

Science & Technology

Complexity

Diversity

Ethical Reasoning

Civic life

Religious Reflection

Communication
Engagement with the World

Perspective

Collaboration

Social Justice

Civic Engagement
SCU goals distributed across multiple Core areas
Selected Core Religion,
Areas 
Theology,
Culture
Goals 
Knowledge
Habits of Mind Crit Thinking
& Heart
Complexity
Civic
Engagement
Cultures &
Ideas 3
Civic Life
Global
Cultures
Communicat.
Diversity
Critical
Thinking
Religious
Reflection
Engagement
with the World
Collaboration
Civic
Engagement
Perspective
From Goals to Objectives at SCU
 Faculty Core Committees translate goals into measurable
program learning objectives for Core Area
 Bloom’s taxonomy of cognitive development provides
framework
– Identify, Describe, Recognize
– Compare, Apply, Analyze
– Evaluate, Synthesize
 All syllabi in Core Area include Area (Program) Objectives
Goals transformed into objectives
Religion, Theology & Culture 2
 Goals: Critical thinking, complexity, religious reflection
Objectives: Students will:
 2.1
Analyze complex and diverse religious phenomena, such
as architecture and art, music, ritual, scriptures, theological
systems, and other cultural expressions of religious belief.
 2.2
Integrate and compare several different disciplinary
approaches to a coherent set of religious phenomena.
 2.3
Clarify and express beliefs in light of their critical inquiry
into the religious dimensions of human existence.
Assignment Mapping: An Example from SCU
 The first paper will ask students to focus on a single issue in
Hinduism (e.g. spiritual practice, religious iconography, nature of
self & divine) from a single disciplinary perspective (Core LO 1)
 The second paper will ask students to analyze a single religious
phenomenon such as a devotional song or icon from multiple
disciplinary perspectives (e.g., theological, historical, ethnographic,
art historical) (Core LO 1 & 2)
 The third paper will focus on the experience of contemporary
Hindus, with a focus on belief and practice in communities
centering around female gurus in India and abroad (Core LO 1 & 3)
SCU multi-year assessment plan
(partial list)
 Fall 2010 Religion Theology Culture 2
 Fall 2011 Natural Science
 Fall 2012 Civic Engagement
 Fall 2013 Experiential Learning Soc Justice
 Fall 2014 Cultures & Ideas 3
 Fall 2015 Pathways
Proposal, Step 1
Include IES Rome Religion Course in
Multi-year Assessment Plan
 IES RL 435
Monotheisms: The Children of Abraham
– Term paper
– Midterm
– Final
 Fall 2010, ~20 SCU students enrolled
– All student work for targeted learning objective
included in SCU assessment
How would the pilot assessment
project work?
 One learning objective selected for assessment
 At SCU, work gathered from randomly selected students in each
RTC2 class for selected learning objective
 In Rome (with fewer students), all student work gathered for
selected learning objective
 Rubric scoring party January 2011
 Invite IES to SCU rubric scoring party via tele-conference
 Close feedback loop: Reflect on assessment reports
 Goal: Enhance teaching & learning
Rubric Scoring Parties
 6-10 volunteers
 2-3 hours
 Rubrics for one (or two) learning objectives
 Readers calibrated with common readings
 Two readers per paper; third reader if needed
 Food
Proposal Step 2
Integrate SCU model into IES more broadly
Learning objectives at the program level
Explicit assignment mapping at the course level
Direct assessment of selected student work
Rubric Scoring
Systematic multi-year assessment plan
Challenges
 Aren’t SCU’s processes and assumptions different from
IES’s?
 Logistical obstacles: distance, language, distinct pedagogical
culture in each IES site
Challenges not insurmountable
 IES shares practices with SCU
 SCU and IES Learning Goals are aligned with national vision
of best practices
 Logistics: Exciting opportunity for global collaboration through
systematic plan for rubric development
SCU & IES: Shared Vision & Practice
 IES 3-D paradigm similar to SCU’s Goals
– Cognitive Goals/Ends
– Interpersonal Goals/Ends
– Intrapersonal Goals/Ends
 IES Syllabus Guidelines already require assignment alignment
 IES is committed to ongoing assessment & improvement of
student learning
IES 3-D Program Model & SCU’s Core
Means 
Curriculum
Ends 
Cognitive
Knowledge
Interpersonal Habits of
Mind & Heart
Intrapersonal Engagement
with the
World
Cocurriculum Community
SCU Learning Goals with IES “Ends”
Knowledge/ Cognitive Ends

Global Cultures

Arts & Humanities
Habits of Mind and Heart/
Interpersonal Ends

Scientific Inquiry

Critical Thinking

Science & Technology

Mathematical Reasoning

Diversity

Complexity

Civic life

Ethical Reasoning

Religious Reflection

Communication
Engagement with the World/
Intrapersonal Ends

Perspective

Collaboration

Social Justice

Civic Engagement
Alignment with National Vision
AAC&U
Knowledge of Human Cultures
Intellectual and Practical Skills
Inquiry and analysis
Critical and creative thinking
Written and Oral
Communication
Information Literacy
Teamwork
Problem Solving

IES
Cognitive
SCU
Knowledge

Interpersonal
Habits of Mind
& Heart
Personal and Social Responsibility
Integrative and Applied Learning

Intrapersonal
Engagement
with the World
IES & SCU: Similar Practices
IES
SCU
 Require clear learning
 Require learning objectives
objectives on syllabi at
course level
 Request alignment of
assignments
 Require assessment of
student learning at
course level
on syllabi at course &
program level
 Require explicit assignment
mapping
 Assess student learning at
program level as well as
course level
Major Difference: Program Level
Assessment
How does Program Level Assessment work?
 Determine Program Level Objectives
 Select program level learning objective for assessment
 Develop rubric for assessment of study work
 Gather randomly selected student work
 Host 2-3 hour rubric scoring party, assess student work for
selected learning objective
 Provide feedback to participating departments
Logistics of Proposal
IES incorporates SCU practices
1.
Develop program level learning objectives
2.
Include program level learning objectives on syllabi
3.
Map assignments explicitly to learning objectives
4.
Develop multi-year assessment plan
Summary
 Fall 2010 SCU includes IES Rome course in Core Assessment
plan for Religion, Theology, Culture 2
 IES incorporates key aspects of SCU assessment model
–
–
–
Measurable Learning Objectives at program level
Explicit assignment mapping
Multi-year assessment plan
Direct assessment of student work in rubric scoring parties
Feedback; reflection; improvement of teaching & learning
 SCU includes SCU student work from other IES sites in multi
year assessment plan
Contact us
 Office of Undergraduate Studies
 Santa Clara University
 www.scu.edu/assessment
 www.scu.edu/core
 Diane Jonte-Pace, Vice Provost, Undergrad Studies
 Carol Ann Gittens, Director of Assessment
 Barbara Molony, Chair, History Dept; SCU Core
Coordinator; IES Curriculum Committee member
 Joan Gillespie, Associate Vice President of Academic
Affairs, IES