Transcript THE DIVERGENT BUT COMPLIMENTARY PATHS OF …

THE DIVERGENT BUT COMPLEMENTARY
PATHS OF COMPREHENSIVE PROGRAM
REVIEW AND STUDENT LEARNING
OUTCOMES ASSESSMENT
A Case Study Presentation
At the 2004 AAHE Assessment Conference
On June 15, 2004
By Ed Rugg & Val Whittlesey
Kennesaw State University
TWO MAJOR DOMAINS
OF ASSESSMENT
PROGRAM
REVIEW
LEARNING
OUTCOMES
FOCUS OF
PROGRAM REVIEW







UNIT OF ANALYSIS
A DEGREE PROGRAM
AN ORGANIZATIONAL UNIT
ASSESSMENT ISSUES
INPUTS, PROCESSES, OUTPUTS
QUALITY INDICATORS
PRODUCTIVITY INDICATORS
CENTRALITY & VIABILITY
ROI & INVESTMENT POTENTIAL
FOCUS OF LEARNING
OUTCOMES






UNIT OF ANALYSIS
THE GRADUATE OF THE PROGRAM
STUDENT LEARNING
ASSESSMENT ISSUES
KNOWLEDGE OF GRADUATES
SKILLS OF GRADUATES
ATTITUDES/VALUES OF GRADUATES
EDUCATIONAL GROWTH/GAINS
COMPLEMENTARY
OVERLAP
Learning Outcomes
Constitute an Important,
but Only One,
Set of Outputs Evaluated
in Program Review
ACCREDITATION’S QUEST
FOR QUALITY ENHANCEMENT
Southern Association’s (SACS/COC)
PRINCIPLES OF ACCREDITATION
is subtitled:
FOUNDATIONS FOR QUALITY
ENHANCEMENT
“The concept of quality enhancement
is at the heart of the Commission’s
philosophy of accreditation….an
institution is expected to document
quality and effectiveness in all its
major aspects.” (p. 5)
SACS CORE REQUIREMENTS
FOR PROGRAM REVIEW
2.5 The institution engages in
ongoing, integrated, and institutionwide research-based planning and
evaluation processes that incorporate
a systematic review of programs and
services….(Institutional Effectiveness)
2.12 The institution has developed an
acceptable Quality Enhancement
Plan…that …is part of an ongoing
planning and evaluation process. (QEP)
SACS COMPREHENSIVE
STANDARDS FOR
LEARNING OUTCOMES
3.4.1 …each educational program for
which academic credit is awarded
…establishes and evaluates program and
learning outcomes.
3.5.1 The institution identifies collegelevel competencies within the general
education core and provides evidence
that graduates have attained those
competencies.
SACS COMPREHENSIVE
STANDARD ON
INSTITUTIONAL EFFECTIVENESS
3.3 Institutional Effectiveness
3.3.1 The institution identifies
expected outcomes for its
educational programs and its
administrative and educational
support services; assesses
whether it achieves these
outcomes; and provides
evidence of improvement based
on analysis of those results.
IMPLEMENTING A DUAL
FOCUS ON ASSESSMENT
AT KSU
 COMPREHENSIVE PROGRAM
REVIEW
 ASSURANCE OF LEARNING
IMPORTANCE OF A
WEB-BASED ARCHIVE OF
DOCUMENTATION
• Managing Complex MultiYear CPR and AOL Processes
• Anytime Access, Referral &
Communication on CPR & AOL
• Sharing Experiences & Good
Practices
• Documenting Compliance &
Improvement for Accreditation
ASSESSMENT REPORTING
PRIOR TO 2003


From 1994 to 2003, academic
departments at KSU sent their
annual assessment reports to the
Office of Institutional Planning.
These nonstandard annual reports,
ranging from 1 to 50+ pages, were
simply filed away with no follow-up
or feedback.
DISSATISFACTION WITH THE
“A” WORD & REPORTS



Unfocused Reports – assessments
rarely tied to learning outcomes or
program goals.
Uneven Reports – from
exemplary/sophisticated content to
inadequate/unsophisticated.
Unappreciated – widespread
aversion to and avoidance of
assessment.
ASSURANCE OF LEARNING
INITIATIVE AT KSU


Established an Assurance of
Learning (AOL) Council in Spring of
2003 with High-Level Academic
Leadership (AVPAA as Chair).
Council Membership Includes Two
Assessment-Oriented Faculty from
each College, the Director of
Excellence in Teaching & Learning,
and the Director of Institutional
Effectiveness.
AOL COUNCIL FUNCTIONS


The AOL Council serves as a steering
committee and advisory/consultative group
to academic programs as they design,
implement, and improve their assessments
of student learning outcomes.
The AOL Council provides common
structures and good practices for learning
outcomes assessment and collegial
formative feedback on annual reports at
the program level.
EIGHT ELEMENTS OF
STUDENT LEARNING
OUTCOMES ASSESSMENT








Articulating Student Learning Outcomes
Connecting Outcomes to the Curriculum
Connecting Outcomes to Assessment
Methods
Articulating Expected Results with
Respect to Outcomes
Articulating the Assessment Plan for
Collecting and Analyzing Data
Collecting and Analyzing Data
Comparing Actual and Expected Results
Using Results for Improvement
BEGINNING AT THE
BEGINNING IN 2003-2004


Articulating General and
Specific Student Learning
Outcomes for Each Degree
Program and General Education
Connecting Learning Outcomes
to the Curriculum
IMPLEMENTATION OF A
COLLABORATIVE MODEL






Reporting and Evaluation Guidelines
Formal Workshops
Informal Brown Bag Consultations
Council Review of and Formative
Feedback on AOL Reports
Assessment Resources
Examples of Good Practice
AOL COUNCIL REVIEW
PROCESS IN SPRING 2004



AOL Council used Evaluation
Guidelines to provide formative
feedback on the program reports.
Used subcommittee assignments for
Council’s reviews.
48 AOL program reports reviewed
with their authors by the Council.
LESSONS LEARNED
&
NEXT STEPS
QUESTIONS??
PowerPoint & Handouts
Available for Download at
http://ie.kennesaw.edu