Transcript Council of Deans January 19, 2006

Report of the Ad Hoc Working Group on
Assessment of Learning Outcomes
N. John Cooper
Dean, School of Arts and Sciences &
College of General Studies
Convener, Ad Hoc Working Group on
Assessment of Learning Outcomes
Council of Deans
November 9, 2006
Shared Values
 Commitment to excellence in instruction that
spans regional campuses, professional schools,
and liberal arts schools
 Commitment to importance of evaluating the
outcomes of instructional programs
 Commitment to the use of outcome evaluations
in strategic planning processes
Move to Implementation
Commitment
to excellence
in instruction
Commitment to
importance of
evaluating the
outcomes of
instructional
programs
Commitment to
the use of
outcome
evaluations in
strategic planning
processes
Leads to Creation of New Culture
Culture of
Assessment
Processes of that Culture Must Be
 Consistent with
University goals
 Comprehensive
 Documented
 Meaningful to others
 Sustainable
To Be Consistent with University Goals
Each school’s and campus’ goals for student learning outcomes
must be consistent with the University’s goals for all of our
graduates, that they are able to:
 Think critically and analytically
 Gather and evaluate information effectively & appropriately
 Understand and apply basic, scientific, & quantitative
reasoning
 Communicate clearly and effectively
 Use information technology appropriate to their discipline
 Exhibit mastery of their discipline
 Understand and appreciate diverse cultures
 Work effectively with others
 Have a sense of self, responsibility to others, and
connectedness to the University
To Be Consistent with University Goals
 Assessment must be based on student learning
outcomes
 As articulated by Middle States in Standard 14,
Assessment of Student Learning:
“Assessment of student learning demonstrates that
the institution’s students have knowledge, skills,
and competencies consistent with institutional
goals and that students at graduation have
achieved appropriate higher education goals.”
Faculty Must Be Engaged in Development
of Learning Outcomes Assessment
Questions asked of Arts and Sciences Department Chairs
and Program Directors for 2006 Departmental Activity
Reports:
 What attributes, skills, and knowledge do you expect
graduates in your major(s) to acquire that are
characteristic of the discipline?
 What attributes, skills, and knowledge do you expect your
graduates to acquire that are hallmarks of your program
at Pitt?
 What qualitative and/or quantitative evidence can you
collect on an ongoing basis to show how well your
graduates are meeting these goals?
Student Outcomes May Be Measured
Directly







Course papers
Course assignments
Exhibits
Performances
National disciplinary exams
Performance on licensure or professional exams
Authentic assessments
Student Outcomes May Be Measured
Indirectly





Job placements and placement rates
Student surveys
Graduate follow-up surveys
Focus groups
Exit interviews
Especially Rich Opportunities to Measure
Student Outcomes







Capstone courses
Senior projects
Undergraduate research projects
Thesis/dissertation
Internships
Portfolios of work and reflections on that work
Embedded questions in assignments
/examinations in specific courses in the
curriculum
Approaches to Measurement of
Outcomes
 Program faculty review
assignments from
capstone courses and
assess how well the
students have achieved
desired outcomes
 Program faculty review approved dissertations and
assess how well the students have achieved desired
outcomes
 Faculty external to the course score embedded
standardized questions
To Be Comprehensive
Assessment processes expected for:
 All degree or certificate-granting programs listed in the
graduate and undergraduate bulletins
 At a minimum each major at each degree level should
be assessed, e.g.,
• Associate of Science in Nursing
• Bachelor of Science in Civil Engineering
• Master of Arts in History
• Doctor of Philosophy in English
• MD, JD, MPPM, MAT, EdD, MSW
• Etc.
 School- and campus-level general education curricula
To Be Documented
Need Standard Minimal Components:
 An articulation of program mission/goals which reflects
national disciplinary norms as well as any unique
features of the University of Pittsburgh program
 Identification of three to five prioritized learning
outcomes
 Identification of methods of assessment and
determinations of when in the curriculum—and how—
learning outcomes will be assessed
 Standards of comparison/targets for results of
assessment
To Be Meaningful to Ourselves
and to Others
 Assessment must include
direct evidence
 If a specific course is used,
there must be periodic
validation external to the
course and instructor
 A process of faculty and
administrative review that
ensures results are used
for program improvement
To Be Sustainable
 Must be respectful of
faculty, administrative,
and staff resources
• Do not have to measure
every outcome every year
using every student
 Must be embedded in
annual planning process
 Must be seen to drive
change
Distributed Responsibility Is Critical to
Sustainability
 Program faculty are responsible for the development and
administration of the assessment processes of individual
programs in accordance with the appropriate programmatic
or departmental governance structure
 Department chairs are responsible for coordinating the
assessment process for departmentally-based programs;
deans & campus presidents are responsible for school- and
campus-based programs
 Schools & regional campuses are responsible for developing
internal procedures for documenting program assessment
 Deans & campus presidents are to report annually to the
Provost on their assessment activities and relevant results as
part of their planning processes
Processes of a Culture of Assessment
Must Be
 Consistent with
University goals
 Comprehensive
 Documented
 Meaningful to others
 Sustainable
 STARTED!
Proposed Time Table
To ensure that we are prepared to report to our accrediting
agency, Middle States, in a timely manner:
 Deans & campus presidents will be asked to submit
documentation of their assessment processes
• for each degree program as part of their annual
planning documents in March 2007
• for general education programs as part of their
annual planning documents in March 2008 if not
embedded in degree program
 Initial assessments should be conducted and results reported
• for each degree program in AY 2008
• for general education programs in AY 2009 if not
embedded in degree program