Transcript Integrating Higher Education Planning and Assessment

Integrating Higher Education Planning and
Assessment:
Concepts and Strategies
Michael F. Middaugh
Associate Provost for Institutional Effectiveness
University of Delaware
&
Commissioner and Member of Executive Committee
Middle States Commission on Higher Education
Andrea A. Lex
Dean of Planning and Institutional Research
Prince George’s Community College
“The nicest thing about not planning is that failure comes
as a complete surprise and is not preceded by a period of
worry and depression.”
John Preston, Boston College
“A common mistake that people make when trying to
design something completely foolproof is to underestimate
the ingenuity of complete fools.”
Douglas Adams, Hitchhikers
Guide to the Galaxy
Our (Flexible) Agenda
– Foundation
• Accreditation
• Characteristics
– Standards 14, 7, 2, 1
– Looking Closely at
Mission & Goals
– Linking Planning &
Assessment
– RECEPTION &
DINNER
– Recap
– Planning to Plan
– What Should Be
Assessed?
– Some Measures
– Tying It All Together
Who are we?
>>> 49 Participants from 30 Institutions
>>>13 teams representing a cross section
• Institutions
• Individuals
– 33 % public 4-year
– 41% public 4-year
– 23% private 4-year
– 18% private 4-year
– 37% two-year
– 35% two-year
–
–
7% specialized
6% specialized
Components of This Curriculum
• An Overview of Accreditation
• The Characteristics of Excellence
• Closing the Planning & Assessment Loop
• Best Practice in Planning
• Best Practice in Assessment
• Real Life Case Studies
Our Lens
A Brief Overview of Accreditation
&
The Characteristics of Excellence
What is accreditation?
• A means of providing quality assurance in
the areas of
– Student learning and achievement
– Curriculum development and support
– Faculty
– Facilities and equipment
– Finance, administration, and governance
– Integrity and compliance
Why is it done?
•
•
•
•
•
•
Voluntary self-regulation
Voluntary self-examination
Opportunity for peer review
Quality assurance
Public accountability
And, in the case of regional accreditation
– Ability to receive Federal financial aid dollars
What is the subject?
THE ENTIRE
INSTITUTION
What “camera” is used?
The
Characteristics
of Excellence
What is the focus?
The Effectiveness of
THE ENTIRE
INSTITUTION
What lens is used?
Institutional
Effectiveness
(Institutional
Assessment)
Educational
Effectiveness
(Assessment of
Student Learning)
West County
Community
College
Standards 1 through 6
•Mission
•Planning
•Resource Allocation
•Leadership
•Governance
•Administration
•Integrity
Educational Effectiveness
(Standard 14: Assessment of Student Learning)
Standards 8 through 13
•Admissions & Retention
•Student Support Services
•Faculty
•Educational Offerings
•General Education
•Related Educational
Activities
•Mission
•Planning
•Resource Allocation
•Leadership
•Governance
•Administration
•Integrity
Educational
Effectiveness
•Admissions & Retention
•Student Support Services
•Faculty
•Educational Offerings
•General Education
•Related Educational
Activities
Standard 7: Institutional Assessment
The institution has developedand implemented an
assessment process that evaluates:
• Its overall effectiveness in achieving its mission
and goals and
• Its compliance with accreditation standards.
Educational Effectiveness
Standard 14: Assessment of Student Learning
Assessment of student learning demonstrates that,
at graduation, or other appropriate points, the
institution’s students have knowledge, skills, and
competencies consistent with:
• Institutional and
• Appropriate higher education goals.
Fundamental Elements
Standard 14
• Articulated expectations for
student learning
• Written institutional strategic
plan(s)
• Documented, organized, and
sustained assessment
processes
• Documented, organized, and
sustained assessment
processes
• Evidence that student learning
assessment information is
shared & used to improve
teaching and learning
• Evidence that assessment
results are shared & used in
institutional planning, resource
allocation and renewal
• Evidence that institutional
• Documented use of student
assessment findings are used
learning assessment information
to improve student success
as part of institutional success
Fundamental Elements
Standard 14
• Articulated expectations for
student learning
• Written institutional strategic
plan(s)
• Documented, organized, and
sustained assessment
processes
• Documented, organized, and
sustained assessment
processes
• Evidence that student learning
assessment information is
shared&used to improve
teaching and learning
• Evidence that assessment
results are shared&used in
institutional planning, resource
allocation and renewal
• Documented use of student
• Evidence that institutional
learning assessment information
assessment findings are used
as part of
success
to improve student success
What has Accreditation Review
Revealed?
• The quality of an institution’s
accreditation self study hinges on how it
addresses the standards related to
– assessing institutional effectiveness
(Standard 7)
– assessing student learning outcomes
(Standard 14)
• implementing a vital and comprehensive
strategic planning process (Standard 2).
Assessment
Plan
Resource
Allocation
Goals
Strategies
Standard 14
• Articulated expectations for
student learning
• Written institutional strategic
plan(s)
• Documented, organized, and
sustained assessment
processes
• Documented, organized, and
sustained assessment
processes
• Evidence that student learning
assessment information is
shared & used to improve
teaching and learning
• Evidence that assessment
results are shared & used in
institutional planning, resource
allocation and renewal
• Documented use of student
• Evidence that institutional
learning assessment information
assessment findings are used
as part of institutional success
to improve student success
Standard 2: Planning, Resource Allocation
and Institutional Renewal
– An institution conducts ongoing planning and
resource allocationbased on its mission and goals,
– develops objectives to achieve them,
– and utilizes the results of its assessment activities
for institutional renewal.
– Implementationand subsequent evaluation of the
success of the strategic plan and resource allocation
– support the development and change necessary to
improve and to maintain quality.
Selected Fundamental Elements for
MSCHE Standard 2
• Clearly stated goals and objectives, at both the institutional and unit
levels, that reflect conclusions drawn from assessments that
are used for planning and resource allocation at those levels
• Planning and improvement processes that are clearly
communicated, provide for constituent participation, and
incorporate the use of assessment results
• Well-defined decision-making processes and authority that
facilitates planning and renewal and the assignment of
responsibility for improvement and assurance of accountability
• Periodic assessment of the effectiveness of planning, resource
allocation, and institutional renewal processes
The Closed Loop
• Institutions must plan effectively in order to
be effective.
• Where that is the case, the accreditation
process is an affirmation of the evidence
of that effectiveness.
Assessment
Plan
“Institutional planning is
a disciplined,
coordinated, systematic,
and sustained effort . . ..”
Resource
Allocation
Goals
Strategies
Wants
Pressures
Terrific Ideas
Assessment
Plan
Resource
Allocation
Goals
Strategies
• Standard 1: Mission and Goals
– The institution’s mission clearly defines its
purpose within the context of higher education
– And indicates who the institution serves
– And what it intends to accomplish.
• Standard 1: Mission and Goals
– The stated goals, consistent with the
aspiration and expectations of higher
education, clearly specify how the institution
will fulfill its mission.
• Standard 1: Mission and Goals
– The mission and goals are developed and
recognized by the institution with the
participation of its members and its governing
body
– and are utilized to develop and shape its
programs and practices
– and to evaluate its effectiveness.
Mission Statement
• A good mission statement is a carefully reasoned analysis of
what an institution aspires to be, and the core values that it
embraces.
• It avoids cliché language, e.g., “Students and faculty will interact
in a rich intellectual environment in which each individual has the
opportunity to achieve their full potential.” Noble sentiment, but
says nothing about the institution’s purposes and priorities.
• Mission must speak to central institutional issues, e.g., desired
balance between undergraduate and graduate education; relative
emphasis on teaching, research, and service, respectively; and
so on.
Mission Statement
• Mission statements are characterized by a sense of vision that,
while not immutable, nonetheless represents a long-term
statement of institutional values and direction around which
human and fiscal resource allocation decisions can be made.
• While cognizant of the institution’s ever changing external
environment, mission statements are not whimsical, morphing
with each new market trend that emerges.
• The mission statement provides a clear sense of direction around
which action-oriented goal statements and measurable planning
objectives can be developed.
How will we actualize our
mission?
Planning Goals
• Goal statements are derived from the institutional mission, and
help to define policy.
• They describe how to achieve the mission.
• The “how” are specific, action-oriented planning goals aimed at
moving the institution toward a fuller realization of its mission.
Planning Goals
• Mission Statement Segment: The University reaffirms its historic
mission to provide the highest quality education for its
undergraduate students, while maintaining excellence in
selected graduate programs.
• Possible related goal: “The University will continue to attract and
retain the most academically talented and diverse
undergraduate students, and support their intellectual, cultural,
and ethical development as citizens and scholars.”
• Action verbs such as “attract,” “retain,” and “support” elevate the
goal statement to policy level. How do we know that policy is being
carried out? Measurable planning objectives.
Planning Objectives
• What will we look like when we get there?
Planning Objectives
Planning objectives provide empirical evidence of the extent to
which planning goals are being achieved.
Planning Goal: to attract, retain, and support academically talented
and diverse students.
Related Objectives:
– Retain a freshman admissions target of 3,200 to 3,400 students annually,
with an admissions profile for academic year 2007 of 23,000 applications,
a 40 percent admit rate,&a yield rate in excess of 35 percent.
– Maintain a freshman-to-sophomore retention rate above the national
average for highly selective institutions, and seek to achieve a consistent
rate of 90 percent or higher.
– Increase minority and international enrollment, with retention and
graduation rates for those populations consistent with the university-wide
averages for all students.
Planning Objectives
• The defining characteristic for any good planning objective is that
it must be measurable. In other words, assessable.
• What does this mean operationally?
– Colleges or universities embarking on any planning process – long range,
strategic, tactical – require a systematic institutional research capability.
– While smaller institutions may not have an office of institutional research,
per se, they must nonetheless have the capability of quantitatively and
qualitatively assessing the extent to which planning objectives are being
implemented, planning goals are being achieved, and the institutions
mission is being realized.
Central Threads Running Through All
Accreditation Requirements:
• Planning must be systematic
• Planning must be rooted in an institution’s mission
• Planning must be predicated on analytical and
evaluative information
• Planning must be used for institutional decisions,
including resource allocation
Cornerstones of the Planning Process
Institutional Mission
ActionOriented
Goals
ActionOriented
Goals
ActionOriented
Goals
Measurable
Objectives
Measurable
Objectives
Measurable
Objectives
Breakout Session
To what degree do YOUR mission and
goal statements and the samples provided
meet the characteristics described in
MSCHE’s fundamental elements for
developing a mission and goals?
Our Lens
Linking Planning and Assessment
Two Case Studies at
The Same Institution
The First Cut: Creating a Culture of Planning
University of Delaware
Strategic Planning: 1990 - 2000
University of Delaware Mission Statement
“The University reaffirms its historic mission to provide the highest quality education for its undergraduate
students, while maintaining excellence in selected graduate programs. The faculty are responsible for
helping students learn to reason critically and independently, gain knowledge of the diverse culture and
environment in which they live, and develop into well-informed citizens and leaders. To accomplish
these goals, the University provides a learning setting enriched by undergraduate student research,
experiential learning, and study-abroad programs. The University places high priority on the education of
qualified Delaware residents and provides opportunity for a diverse group of citizens to participate in
postsecondary education. Since the University is located in a state with a small population, providing
programs of quality and diversity requires a community of student-scholars that reaches beyond the
boundaries of the state, one that reflects the nation’s racial and cultural diversity.
“The University of Delaware also aspires to excellence in graduate education, the heart of which is
scholarship and research. The creation, application, and communication of knowledge is a primary goal
of the institution and of every faculty member, providing the substance for creative, informed teaching.
Research is typically based on cooperation between faculty and students, whereby faculty mentors teach
students to conduct independent research and to master problem-solving techniques. Through
involvement of undergraduates in faculty research, the University creates a special bond between its
undergraduate and graduate programs.
“The University is also committed to providing service to society, especially in Delaware and the
neighboring region. Public service is a responsibility of every academic unit. In addition, each faculty
member is responsible for service to the University community and to his or her profession. The
University emphasizes practical research, provides extension services, and works to solve problems
confronting the community.
(University of Delaware, Middle States Self-Study Report, April 2001)
Case Study – University of Delaware
• In 1987, the University embarked on a comprehensive, longrange planning process, termed Project Vision. Over a period of
18 months, the campus developed a planning document with a
broad spectrum of planning goals and measurable objectives
embracing all aspects of University operations.
• In Fall of 1988, the President who initiated Project Vision
suddenly resigned. At the same time, the Delaware economy
along with that of the entire mid-Atlantic region was plunging into
deep recession
Case Study – University of Delaware
• Rather than let 18 months of planning activity go for naught –
even though resources would be scarce for the foreseeable
future – a panel of distinguished senior faculty was assembled to
review the Project Vision planning document and to cull out those
goals and objectives that were clearly consistent with, and
essential to furthering the University mission.
• The resulting document, Focused Vision, was economical when
compared with its progenitor, both in terms of words and
resource requirements. However, the economy would still clearly
preclude anything even remotely approaching implementation.
• In 1990, the University hired its 25th President, David P. Roselle.
Case Study – University of Delaware
• In order to maintain planning momentum, the President
consulted with his senior staff, the faculty, and appropriate
constituencies across campus to determine those areas that
required immediate attention.
• From these consultations, the President articulated four strategic
initiatives that would constitute the focus of decision-making and
resource allocation activity in the immediate future. Those
initiatives were competitive compensation for faculty and staff;
enhanced access to the University for undergraduates through
expanded availability of financial aid; a more student-centered
campus environment; and renovation and rehabilitation of
campus facilities.
Case Study – University of Delaware
These priorities were not a “wish list.” They grew out of a careful
examination of empirical data provided by the University’s Office
of Institutional Research and Planning and other data sources.
Consider the following:
• When compared with the 24 Category Doctoral I universities in
the states contiguous to Delaware, and the District of Columbia,
in 1991 the average salary for all three major faculty ranks at the
University of Delaware ranked near the bottom of the list.
• The Student College Selection Survey indicated that students
were receiving offers of more aid from admissions competitors,
and that the aid packages had more grants and fewer loans than
University aid packages. Not surprising, the University was at a
competitive disadvantage for academically talented students.
Case Study – University of Delaware
• University scores on the ACT Student Opinion Survey suggested
that the institution had considerable room for improvement with
respect to student satisfaction with programs and services, and
with a number of areas in student life.
• The University was looking at in excess of $200 million in
deferred maintenance to its buildings and grounds.
Case Study – University of Delaware
• A critical factor in moving forward with these initiatives was
getting the campus to understand that the economy was in
recession and that there would be no immediate or massive
infusions of new resources.
• Colleges and universities have multiple revenue streams –
tuition, state appropriation in the case of public institutions,
contracts and grants, gifts, etc. While growing revenue streams
is an important strategic initiative, so too is the commitment to
not balance budgets on the backs of students through
inordinately large tuition increases.
• Resource reallocation would be the primary source of funding the
four strategic initiatives, and it was critical that the campus
understand from where funds were reallocated, and why.
The University went on public record in 1991:
• Average total compensation for faculty at each academic rank
would be at or above the median within five years for the 24
Category I Doctoral Universities identified as salary peers.
• Total undergraduate financial aid from all sources would increase
by 100 percent within five years.
• Student satisfaction with programs and services at the University,
as measured through the ACT Student Opinion Survey would
demonstrate significant gains within five years.
• The University would commit itself to a policy of annually setting
aside at least 2 percent of the replacement value of the physical
plant, to be used for facilities renovation and rehabilitation.
Results - Salaries
Total Compensation, 1989-90 Compared with 1999-2000
FULL PROFESSOR
Institution
1989-90
1999-2000
Princeton University
91,800
University of Pennsylvania
151,900
University of Pennsylvania
91,500
Princeton University
134,100
Georgetown University
88,700
Georgetown University
126,100
Carnegie-Mellon University
85,800
Carnegie-Mellon University
123,200
University of Virginia
85,100
University of Virginia
122,900
George Mason University
84,400
UNIVERSITY OF DELAWARE
116,200
Johns Hopkins University
83,800
Temple University
114,300
New Jersey Institute of Technology
83,100
George Mason University
113,900
Rutgers University - New Brunswick
82,000
College of William and Mary
113,400
Lehigh University
80,800
Lehigh University
111,300
University of Maryland - College Park
80,500
American University
111,100
Virginia Polytechnic Inst. & State University
76,700
Johns Hopkins University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Maryland - Baltimore County
76,700
George Washington University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Pittsburgh - Main Campus
76,500
New Jersey Institute of Technology
109,200
George Washington University
76,500
Rutgers State University-N.Brswck
109,000
American University
76,300
Pennsylvania State University
108,600
Pennsylvania State University
75,300
University of Maryland - College Park
106,800
UNIVERSITY OF DELAWARE
74,700
University of Pittsburgh - Main Campus
105,600
College of William and Mary
74,400
Virginia Polytechnic Inst. & State University
103,800
Virginia Commonwealth University
72,600
Drexel University
103,700
Drexel University
71,000
Virginia Commonwealth University
103,300
Old Dominion University
70,400
University of Maryland - Baltimore County
96,200
Temple University
68,400
Howard University
93,300
Catholic University of America
64,800
Catholic University of America
Howard University
64,000
Old Dominion University
86,800
No Report
Results – Financial Aid
Growth In Undergraduate Financial Aid at the
University of Delaware, FY 1990 Compared with
FY 2000
FY 1990
FY 2000
% Increase
University Administered Funds
$4,458,640
$28,036,660
528.8
State Grant Funds
$3,869,000
$6,643,500
71.7
Other Fund Sources
$2,169,602
$4,394,180
102.5
$10,497,242
$39,074,340
272.2
TOTAL
Results – Student Satisfaction
Comparison of 1995 University of Delaware Scores on ACT
Student Opinion Survey with 1990 Scores and with National
Norms
1990 Scores
Compared with
National Norms
1995 Scores
Compared With
1990 Scores
1995 Scores
Compared with
National Norms
University is Ahead
21
44
39
University is Tied
14
11
15
University is Behind
27
7
8
Results - Facilities
• By 2000, the University had renovated every classroom in its
entire building inventory, retrofitting most with state-of-the-art
teaching technology.
• An aggressive program of fundraising enabled not only the
aforementioned renovation and rehabilitation, but also the
construction of several new classroom and student services
buildings.
• The University is now on a cycle of planned maintenance, as
opposed to deferred maintenance.
Results – From an Accreditation Perspective
“ The
University of Delaware has every reason to take enormous pride in
what it has accomplished over the past 10 years. A decade ago, it was
coming out of a period of considerable turmoil. Today, the University is
seen as a national model for the integration of information technology in
every aspect of university life: teaching and learning, research and
service, academic support, and campus administration. It has created
a physical plant that has few, if any, peers among public universities
and would be the envy of most private colleges. These substantial
achievements could not have happened without extraordinary
leadership from the senior administration.”
“ Better than almost any university we are familiar with, Delaware has a
clear sense of what it wants to be, namely, a university that offers high
quality undergraduate education with targeted areas of excellence in
graduate education and research.”
" The review team was enormously impressed by the high level of morale
that pervades the faculty, staff, and students. Almost without exception,
the people we spoke to take great pride in being part of the University.”
Middle States Evaluation Team, 2001
• The University submitted its Periodic Review Report to
the Commission on Higher Education in May 2006,
outlining its strategies for incorporating systematic
assessment of student learning outcomes into all
facets of campus planning.
• The Commission thanked the University for its Report,
commended the University on its content, and
reaffirmed accreditation.
What Was Not Apparent to MSCHE…
• Despite an exemplary self study and PRR process,
planning was not yet institutionalized into a formal,
iterative process at the institution.
• The four strategic initiatives articulated in 1990
continued to be the benchmarks against which the
University measured success well into the 2000
decade.
• The University needed to move forward with a new
strategic planning effort that would more substantively
incorporate planning into the organizational fiber of the
institution.
The Second Cut: Institutionalizing Planning
University of Delaware
Strategic Planning: 2007 - ???
The Path to Prominence
• On July 1, 2007, Patrick T. Harker succeeded David
Roselle as the 26th President of the University.
• As former Dean of the Wharton School at the
University of Pennsylvania, President Harker was
intimately familiar with the concept of strategic
planning.
• He immediately appointed a Strategic Planning
Committee led by two distinguished senior faculty and
comprised largely of individuals from the academic as
opposed to administrative side of the institution.
The Path to Prominence
• During the 2007-08 academic year, the Strategic
Planning Committee met with over 100 constituent
groups from within, and external to the University.
These information-gathering sessions were part of a
comprehensive SWOT analysis.
• In April 2008, the Report of the Strategic Planning
Committee was delivered to the President.
• In July, 2008, the President presented the Path to
Prominence, the strategic planning document that
would guide institutional activity for the next several
years.
The Path to Prominence: The Strategic Initiatives
• A Diverse and Stimulating Undergraduate Academic
Environment
• A Premier Research and Graduate University
• Excellence in Professional Education
• The Initiative for the Planet
• The Global Initiative
• An Engaged University
The Path to Prominence: The Strategic Initiatives
• Each of the six initiatives was operationalized with
measurable goals and planning objectives.
• The Path to Prominence document and supporting
materials were broadly circulated via the University
website: http://www.udel.edu/prominence/ for maximum
transparency and accountability with respect to what was
being accomplished.
• In Fall 2008, the U.S. and Delaware economies began to
unravel, placing severe financial constraints on full
implementation of the strategic initiatives.
The Path to Prominence: The Strategic Initiatives
• While each of the six initiatives remain viable and will receive
attention regardless of economic circumstances, hard decisions
needed to be made with respect to where resources would be
allocated.
• It was determined that three initiatives would be the primary
(but not exclusive) focus of resource allocation:
- A Diverse and Stimulating Undergraduate Academic Environment
- A Premier Research and Graduate University
- Excellence in Professional Education
The Path to Prominence: Moving Forward
• The University decided to take advantage of the MSCHE
decennial accreditation reaffirmation process, and utilize the
“Special Topics Model” of self study as a means of
documenting progress in implementing these three key
strategic initiatives.
• The scheduled 2011 site visit by an MSCHE Evaluation Team
affords the University not only the opportunity to demonstrate
compliance with the 14 standards in Characteristics of
Excellence; it provides us with a team of expert consultants in
the topical areas on which we have elected to focus. Their
feedback on the content of the Special Topics Self Study will
prove invaluable in moving the institution forward.
The Path to Prominence: Tactical Issues
• In any strategic planning process, there needs to be a clear
linkage between human and fiscal resource allocation and the
initiatives outlined in the strategic plan.
• The University of Delaware has adopted a responsibility
centered budget management model that is designed to
financially reward activity that supports full implementation of
key strategic initiatives.
• The University has also designated an Associate Provost for
Institutional Effectiveness, tasked with overseeing assessment
activity that measures the effective implementation of The Path
to Prominence.
The Path to Prominence: Tactical Issues
• The Steering Committee for the 2011 MSCHE has tasked itself
with responsibility for design of a formal, iterative institutional
strategic planning process that will, on an ongoing basis,
continuously evaluate the implementation of existing strategic
initiatives, assess the continuing relevance of initiatives over
time, and develop new initiatives, as appropriate, in the face of
a changing external environment and/or advent of specific
strategic opportunities.
• The intent is to institutionalize the strategic planning process so
that it is truly an organizational function rather than the creation
or vision of any particular individual.
Let’s Recap
• Accreditation
Quality Assurance
Compliance
Institutional Improvement
• In the Middle States region this is captured under the
concept of “Characteristics of Excellence”
• Keys to Institutional Quality and Improvement:
– Standard 7 Institutional Effectiveness
– Standard 14 Assessment of Student Learning
– Standard 2 Strategic Planning
Guided by
Standard 1:
Mission &
Goals
Let’s Recap
• The Need to Come to a Common
Terminology
• The Need to Establish a Culture of
Planning
• The Need to Institutionalize Planning
Our Agenda Today
• Creating and Sustaining a Strategic
Planning Process
• Assessing our Success
– Some concrete examples of ways to meet the
directives of
• Standard 7 Institutional Assessment and
• Standard 14 Student Learning Outcomes
• Tying It All Together
– Where do we go from here??
Considerations When Creating a
Strategic Planning Process
Strategic Planning is not:
•
•
•
•
•
•
•
•
The personal vision of a president of board of trustees.
Something done on an annual retreat.
Done by planners.
A collection of department plans, compiled and edited.
A substitution of numbers for important intangibles.
A form of surrender to market conditions and trends.
A way to eliminate risks.
An attempt to read tea leaves and outwit the future.
--- George Keller, Academic Strategy: The Management Revolution in
American Higher Education, Johns Hopkins University Press 1983.
Strategic Planning is:
• Looking outward and focused on keeping the
institution in step with the changing environment.
• Cognizant of the competitive environment,
recognizing that the institution is subject to economic
market conditions and to incredibly strong competition.
• Focused on decisions, not on documented plans,
analyses, forecasts, and goals.
• Dependent upon college leaders understanding that
they need to be active rather than passive and
construct the institution’s place in history rather than
simply accept it.
Standard 2: Planning, Resource Allocation
and Institutional Renewal
– An institution conducts ongoing planning and
resource allocationbased on its mission and goals,
– develops objectives to achieve them,
– and utilizes the results of its assessment activities
for institutional renewal.
– Implementationand subsequent evaluation of the
success of the strategic plan and resource allocation
– support the development and change necessary to
improve and to maintain quality.
Selected Fundamental Elements for
MSCHE Standard 2
• Clearly stated goals and objectives, at both the institutional and unit
levels, that reflect conclusions drawn from assessments that
are used for planning and resource allocation at those levels
• Planning and improvement processes that are clearly
communicated, provide for constituent participation, and
incorporate the use of assessment results
• Well-defined decision-making processes and authority that
facilitates planning and renewal and the assignment of
responsibility for improvement and assurance of accountability
• Periodic assessment of the effectiveness of planning, resource
allocation, and institutional renewal processes
“Planning to Plan”
Organizing Your Institution
for the Planning Process
Organizational Structure
• The Planning Committee: Who Serves? How
Representative? What is the Scope of
Responsibilities?
• If the Planning Committee is to be effective, it must be
“blessed” by the Chief Executive Officer and the Chief
Academic Officer. The President and Provost need
not be members of the Committee, but ideally will
attend the initial meeting, and will formally receive the
Committee Report.
Organizational Structure
• Who staffs the Planning Committee?
Institutional Research
Budget Office
Others
Faculty
Staff
Planning Tools
• Surveys: Students
–
–
–
–
–
Admitted Student Questionnaire
Entering Student Needs Assessment Survey
Student Satisfaction Survey
Post-Graduation Placement Survey
Alumni Survey
• Surveys: Employees
– Employee Satisfaction Survey
– Faculty Activity Analysis
– Campus Climate Survey
Planning Tools
• Institutional Studies:
–
–
–
–
–
–
–
–
Admissions Monitoring Analyses
Faculty Workload Analyses
Budget Support Data
Persistence/Graduation Analyses
Compensation Studies: Market/Equity
Financial Ratio Analyses
Benchmarking Studies
Economic Impact Studies
Planning Tools
•
Academic Program Review
–
–
–
Programmatic Self-Examination
Internal and External Review Teams
Focus on Several Levels of Quality
1.
2.
3.
4.
Programs With National/International Distinction
Programs of Regional Distinction (Meet Local Needs)
Programs That, With an Infusion of Resources, Could Achieve
Distinction
Programs Targeted for Retrenchment or Elimination
Planning Tools
•
Environmental Scan:
a.
b.
c.
d.
e.
f.
g.
Demographic issues
Market issues
Political issues
Economic issues
Technological issues
Natural disasters
Other
Creating Common Ground
• What is external
environmental scanning?
• Why are we doing it?
• What areas will we focus
upon?
• How will we do this?
• What will we do with the
information obtained?
Planning Tools
• SWOT Analysis
– Take an honest look at your institution
– Largely Internal
• Strengths
• Weaknesses
– Largely External
• Opportunities
• Threats
SWOT Analysis
Strength
Weakness
Opportunity
X
Threat
Leveraging Situation = Opportunity + Strength
We potentially have the “means” to take advantage of something that
will further our mission.
SWOT Analysis
Strength
Weakness
Opportunity
X
X
Threat
Leveraging Situation = Opportunity + Strength
We potentially have the “means” to take advantage of something that
will further our mission.
Constraining Situation = Opportunity + Weakness
There is an appealing prospect out there, but reaching for it may
either be beyond our grasp or strain our ability to support
existing mission-driven initiatives
SWOT Analysis
Opportunity
Strength
Weakness
Threat
X
SWOT Analysis
Vulnerable Situation = Threat + Strength
We potentially have the “means” to address an issue that can
significantly impact our ability to meet mission.
SWOT Analysis
Opportunity
Strength
Weakness
Threat
X
X
Vulnerable Situation = Threat + Strength
We potentially have the “means” to address an issue that can
significantly impact our ability to meet mission.
Problematic Situation = Threat + Weakness
We do not have the “means” to address an issue that can
significantly impact our ability to meet mission.
SWOT Analysis
Opportunity
Strength
Weakness
Threat
SWOT Analysis
Opportunity
Strength
Weakness
Threat
SWOT Analysis
Opportunity
Threat
Strength
Weakness
WHAT’S IMPORTANT?
University of Delaware
Strategic Initiatives
• A Diverse and Stimulating Undergraduate Academic
Environment
• A Premier Research and Graduate University
• Excellence in Professional Education
• The Initiative for the Planet
• The Global Initiative
• An Engaged University
University of Delaware
Strategic Initiatives
• A Diverse and Stimulating Undergraduate Academic
Environment
• A Premier Research and Graduate University
• Excellence in Professional Education
• The Initiative for the Planet
• The Global Initiative
• An Engaged University
In the Final Analysis, Planning is Directed at
Answering Four Basic Questions:
1.
Who are the markets we are trying to serve?
2.
What services must be in place to fully serve those
markets?
3.
What is the institutional “branding” that will enable
our college or university to appeal to those markets?
4.
How will we know if we are successful in serving
those markets?
Who are the markets we are trying to serve?
What services must be in place to fully serve
those markets?
(Should be) Dictated by the Markets
• Academic Support Services (Library, computing, advising,
tutoring, etc.)
• Student Support Services (Residence life, counseling, health
services, student center, recreation services, etc.)
• Institutional Support Services (General administrative support,
sponsored research, extension office, alumni/governmental
relations, etc.)
What is the institutional “branding” that will enable
our college or university to appeal to those markets?
(Usually is) Market Specific
• University of Delaware:
“A Teaching University”
• Trenton State College :
“The College of New Jersey”
• University of Phoenix:
“The University for Working Adults”
• SUNY College at Brockport:
“The College at Brockport”
• SUNY at Stony Brook:
“Stony Brook University”
• University of Washington
“Uniquely Washington”
How will we know if we are successful in serving
those markets?
Middle States Assessment Standards
• Standard 7: The institution has developed and implemented
an assessment process that evaluates its overall
effectiveness in achieving its mission and goals and its
compliance with accreditation standards.
• Standard14: Assessment of student learning demonstrates
that, at graduation, or other appropriate points, the
institution’s students have knowledge, skills, and
competencies consistent with institutional and appropriate
higher education goals.
Fundamental Elements
Standard 14
• Articulated expectations for
student learning
• Written institutional strategic
plan(s)
• Documented, organized, and
sustained assessment
processes
• Documented, organized, and
sustained assessment
processes
• Evidence that student learning
assessment information is
shared&used to improve
teaching and learning
• Evidence that assessment
results are shared&used in
institutional planning, resource
allocation and renewal
• Documented use of student
• Evidence that institutional
learning assessment information
assessment findings are used
as part of
success
to improve student success
•Mission
•Planning
•Resource Allocation
•Leadership
•Governance
•Administration
•Integrity
Educational
Effectiveness
•Admissions & Retention
•Student Support Services
•Faculty
•Educational Offerings
•General Education
•Related Educational
Activities
Breakout Session
How would you respond to the four key
planning questions within the context of
your institution’s mission?
In the Final Analysis, Planning is Directed at
Answering Four Basic Questions:
1.
Who are the markets we are trying to serve?
2.
What services must be in place to fully serve those
markets?
3.
What is the institutional “branding” that will enable
our college or university to appeal to those markets?
4.
How will we know if we are successful in serving
those markets?
What Should Be Assessed Under
Standard 7?
• You do not have to measure everything.
• PRIORITIZE what is to be measured within the
context of your institution’s culture and needs.
Students
•
•
•
•
•
•
Admitted
Entering
Continuing
Non-Returning
Graduating
Alumni
Environmental Issues
•
•
•
•
Student and Faculty Engagement
Student and Staff Satisfaction
Employee Productivity
Compensation
- Market
- Equity
• Campus Climate
• Economic Impact
Some Basic Measures
A Typical Admissions Monitoring Report
Eastern Seaboard State University
Weekly Admissions Monitoring Report
Campus Summary: First-Time Freshman Applicants, Their SAT Scores and Predicted Grade Index, by Admission Status and by Residency Status for the Entering Classes in the
Fall of 2005 as of 09/15/05; Fall 2006 as of 09/18/06; and Fall 2007 as of 09/13/07.
Counts
- Resident
- Nonresident
- Total
All
Applicants
2005 2006 2007
Admission
Denied
2005 2006 2007
Admission
Offered
2005 2006 2007
Admission
Accepted
2005 2006 2007
2,340 2,332 2,088
18,984 19,209 20,133
21,324 21,541 22,221
109
148
172
7,506 5,871 5,838
7,615 6,019 6,010
1,940 1,877 7,147
7,295 8,101 8,489
9,235 9,978 15,636
1,362 1,255 1,174
2,078 2,201 2,348
3,440 3,456 3,522
SAT Verbal
- Resident
- Nonresident
- Total
557
572
571
562
579
577
569
584
582
456
541
540
444
535
532
445
536
533
567
609
600
576
615
607
584
620
614
559
592
579
564
600
587
575
604
594
SAT Math
- Resident
- Nonresident
- Total
563
596
592
567
598
595
574
605
602
454
561
560
446
551
549
452
554
551
547
636
623
582
635
625
589
643
634
566
619
598
570
620
602
580
627
611
Predicted Grade
Index
- Resident
- Nonresident
- Total
2.67
2.90
2.82
2.75
2.94
2.88
2.83
2.94
2.88
1.66
2.40
2.22
1.60
2.36
2.30
1.71
2.41
2.37
2.75
2.98
2.90
2.78
3.04
2.95
2.85
3.05
2.98
2.75
2.98
2.89
2.78
3.05
2.95
2.85
3.04
2.98
Ratio of Offers
to Total Applications
2005 2006 2007
0.83
0.38
0.43
0.80
0.42
0.46
3.42
0.42
0.70
Ratio of Accepts
to Offers (Yield)
2005 2006 2007
0.70
0.28
0.37
0.67
0.27
0.35
0.61
0.28
0.23
Drilling Down
• Why do some students to whom we extend an offer of
admission choose to attend our institution?
• Why do other students to whom we extend an offer of
admission choose to attend a different school?
• How is our institution perceived by prospective
students within the admissions marketplace?
• What sources of information do students draw upon in
shaping those perceptions?
• What is the role of financial aid in shaping the college
selection decision?
Continuing/Returning Students
•
Student Satisfaction Research
- ACT Survey of Student Opinions
- Noel-Levitz Student Satisfaction Inventory
•
ACT Survey of Student Opinions
Student use of, and satisfaction with 21 programs and services typically found
at a college or university (e.g. academic advising, library, computing, residence
life, food services, etc.)
Student satisfaction with 43 dimensions of campus environment (e.g., out-ofclassroom availability of faculty, availability of required courses, quality of
advisement information, facilities, admissions and registration procedures, etc.)
Self-estimated intellectual, personal, and social growth; Overall impressions of
the college experience
NOTE: Survey is available in four-year and two-year college versions.
What About Non-Returning Student Research?
TABLE 1: ENROLLMENT, DROPOUT RATES AND GRADUATION RATES
FOR FIRST-TIME FRESHMEN ON THE NEWARK CAMPUS (Total)
Enrollment and Dropout Rates
Entering
Fall Term
Graduation Rates
1st
Fall
2nd
Fall
3rd
Fall
4th
Fall
5th
Fall
6th
Fall
within within
3 yrs
4 yrs
within
5 yrs
1995 N
% enrollment
% dropout
3154
100.0%
0.0%
2673
84.7%
15.3%
2439
77.3%
22.7%
2355
75.3%
24.7%
599
19.0%
26.4%
113
3.6%
26.1%
21 1721 2219 2344
0.7% 54.6% 70.4% 74.3%
1996 N
% enrollment
% dropout
3290
100.0%
0.0%
2804
85.2%
14.8%
2585
78.6%
21.4%
2489
76.3%
23.7%
606
18.4%
26.1%
108
3.3%
26.7%
22 1825 2302 2427
0.7% 55.5% 70.0% 73.8%
1997 N
% enrollment
% dropout
3180
100.0%
0.0%
2766
87.0%
13.0%
2523
79.3%
20.7%
2436
77.5%
22.5%
581
18.3%
24.3%
117
3.7%
24.5%
27 1827 2284 2401
0.8% 57.5% 71.8% 75.5%
1998 N
% enrollment
% dropout
3545
100.0%
0.0%
3080
86.9%
13.1%
2830
79.8%
20.2%
2762
78.5%
21.5%
653
18.4%
22.9%
118
3.3%
22.7%
22 2079 2621 2727
0.6% 58.6% 73.9% 76.9%
1999 N
% enrollment
% dropout
3513
100.0%
0.0%
3126
89.0%
11.0%
2871
81.7%
18.3%
2757
79.4%
20.6%
526
15.0%
22.6%
83
2.4%
22.7%
31 2193 2632 2684
0.9% 62.4% 74.9% 76.4%
2000 N
% enrollment
% dropout
3128
100.0%
0.0%
2738
87.5%
12.5%
2524
80.7%
19.3%
2453
79.2%
20.8%
496
15.9%
23.9%
85
2.7%
23.8%
24 1884 2297
0.8% 60.2% 73.4%
--
2001 N
% enrollment
% dropout
3358
100.0%
0.0%
2976
88.6%
11.4%
2746
81.8%
18.2%
2674
80.6%
19.4%
472
14.1%
22.3%
0
0.0%
0.0%
31 2138
0.9% 63.7%
--
--
Total
What About Non-Returning Student Research?
Drilling Deeper…..
• Commercial instruments exist, but response rates tend to be low,
and reported reasons for leaving politically correct – personal or
financial reasons.
• For the last several years, we have administered the Survey of
Student Opinions during the Spring term to a robust sample of
students across freshman, sophomore, junior, and senior
classes.
• The following Fall, the respondent pool is disaggregated into
those who took the Survey and returned in the Fall, and those
who took the Survey, did not return in the Fall, and did not
graduate.
• Test for statistically significant differences in response patterns
between the two groups.
Student Engagement
• College Student Expectations Questionnaire (CSXQ)
• College Student Experiences Questionnaire (CSEQ)
• National Survey of Student Engagement (NSSE)
Graduating Students
EMPLOYMENT AND EDUCATIONAL STATUS OF
BACCALAUREATES BY CURRICULUM GROUP
CLASS OF 2005
Curriculum Group
Number
in
Class
Number of
Respondents¹
N
%
Full-Time
Employment
%
Part-Time
Employment
%
Pursuing
Further
Education
%
Still
Seeking
Employment
%
Agriculture & Natural
Resources
175
63
36.0
58.7
12.7
19.0
7.9
Humanities
417
145
34.8
67.6
10.3
13.1
9.0
Social Sciences
904
295
32.6
71.2
4.4
14.2
8.5
Life & Health Sciences
158
63
39.9
73.0
--
25.4
--
Physical Sciences
166
61
36.7
72.1
--
16.4
9.8
Business & Economics
571
227
39.8
89.9
3.1
2.2
4.4
Engineering
189
103
54.5
85.4
--
11.7
1.9
Health Sciences
390
144
36.9
72.9
2.8
21.5
2.8
Human Services,
Educ. & Public Policy
557
213
38.2
85.9
2.8
4.2
5.6
2005 University Total2
3,527
1,314
37.3
77.2
4.0
11.9
5.9
Arts & Sciences
NOTE: Percentages may not total to 100.0 because of rounding
Alumni Research
• Commercially prepared instruments exist. Decide if
they meet your needs or if you have to develop your
own.
• Decide early on why you are doing this research: Are
you assessing the continuing relevance of the college
experience? Are you cultivating prospects for the
Development Office? Both? Be up front if fund raising
is a component.
• Decide the which classes you need to survey; don’t go
after every living alumnus unless you are a very young
institution.
Non-Student Measures of
Institutional Effectiveness:
Teaching Productivity, Instructional Costs,
and Externally Funded Scholarship
Transparency in Assessment
www.udel.edu/ir
What About Administrative Productivity???
10 Key Strategic Indicators for
Institutional Health
1.
Revenue Structure
2.
Expenditure Structure
3.
Excess (Deficit) of Current Fund Revenues Over Current Fund
Expenditures
4.
Percentage of Freshman Applicants Accepted and Percentage
of Applicants Who Enroll
5.
Ratio of Full Time Students to Full Time Faculty
10 Key Strategic Indicators for
Institutional Health
6.
Institutional Scholarship and Fellowship Expenditures as a
Percent of Total Tuition and Fees Income
7.
Tenure Status of Full Time Faculty
8.
Percent of Total Full Time Equivalent Employees Who Are
Faculty
9.
Estimated Maintenance Backlog as a Percentage of the
Replacement Value of Plant
10. Percent of Living Alumni Who Have Given At Any Time During
The Past Five Years
Think of the Budget as Buckets of Money
Two Kinds of Buckets:
• Revenues
• Expenditures
Major Revenue Streams
•
•
•
•
•
•
•
Tuition and Fees
Governmental Appropriations
Contracts and Grants
Endowment Income
Sales and Services of Educational Activities
Auxiliary Operations
Other
Major Expenditure Categories
•
•
•
•
•
•
•
•
•
•
•
Instruction
Research
Public service
Academic Support
Student Services
Institutional Support
Operation and Maintenance of Plant
Student Aid
Mandatory Transfers
Non-Mandatory Transfers
Auxiliary Enterprises
Note: The term “Education and General Expenditures” refers to the core costs of doing
business related to the institution’s mission of teaching, research, and service. It does not
include non-mission related activities which typically fall under the umbrella of auxiliary
enterprises.
Ratio Analysis:
Revenue Contribution Ratios
Expenditure Demand Ratios
Tuition Dependency Ratio
Net Tuition
Education and General Expenditures
Other Important Revenue Contributors
• State Appropriation
• Gifts
• Endowment Income
Expenditure Demand Ratios
Instructional Demand Ratio
Instructional Expenditures
Total Operating Income
Institutional Support Demand Ratio
Institutional Support Expenditures
Total Operating Income
Useful Resource
IPEDS Peer Analysis System
http://nces.ed.gov/ipedspas/mainScreen.asp
Institutional “Dashboards” that report on key
success indicators can be an succinct means of
reporting on basic measures of institutional
effectiveness.
www.udel.edu/ir/UDashboard
Claims of institutional effectiveness are
stronger when focal institution’s measures on
important measures are compared with those
of peer institutions.
Choosing Peer Groupings
• Scientific – Cluster Analysis or Other Multivariate Tool
• Pragmatic - e.g. Compensation Peers
• “Whatever!” - e.g. Admissions Peers
We are extending the Dashboard concept to include key
variables related to the University’s new Strategic Plan
that enable us to compare our position vis-à-vis actual
peers and aspirational peers.
www.udel.edu/ir/UDashboard_peers
Dashboard Variables Being Collected
STUDENT CHARACTERISTICS
Percent of Accepted Applicants Matriculated
Total Minority Percentage in Undergraduate Student Body
Freshman to Sophomore Retention Rate
Four Year Graduation Rate
Six Year Graduation Rate
RESEARCH ACTIVITY
Total R&D Expenditures per Full Time Faculty
Total Service Expenditures per Full Time Faculty
FINANCE
University Endowment as of June 30
Alumni Annual Giving Rate
INSTRUCTION
Full Time Students per Full Time Faculty
Total Degrees Granted
Total Doctoral Degrees Granted
Percent of Faculty With Tenure
Percent of Faculty That Are Full Time
Percent of Women Among Full Time Faculty
Percent of Minorities Among Full Time Faculty
Percent of Minorities Among Full Time Staff
Total Faculty Who Are National Academy Members
What Should Be Assessed Under
Standard 14?
Assessing Student Learning Outcomes
• I’ll provide only a brief overview, as there are others
(Linda Suskie, Trudy Banta, Jeff Seybert) who are far
better versed than I am.
• That said, understand that assessment of student
learning is at the core of demonstrating overall
institutional effectiveness.
• Assessment of student learning is a direct response to
the inadequacy of student grades for describing
general student learning outcomes.
According to Paul Dressel of Michigan State
University (1983), Grades Are:
“ An inadequate report of an inaccurate judgment by
a biased and variable judge of the extent to which
a student has attained an undefined level of
mastery of an unknown proportion of an indefinite
material. ”
There is no “one size fits all” approach to
assessment of learning across the disciplines
None of these should be applied to evaluation of individual student performance for
purposes of grading and completion/graduation status.
1. Standardized Tests
2.
3.
4.
5.
• General Education or Discipline Specific
• State, Regional, or National Licensure Exams
Locally Produced Tests/Items
• “Stand Alone” or Imbedded
Portfolios/Student Artifacts
• Collections of Students’ Work
• Can Be Time Consuming, Labor Intensive, and Expensive
Final Projects
• Demonstrate Mastery of Discipline and/or General Education
Capstone Experiences/Courses
• Entire Course, Portion of a Course, or a Related Experience
(Internship, Work Placement, etc.)
Tying It All Together……
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Start from institutional Mission: The Mission Statement should
define the institution and its core values, communicate what
business(es) it will engage in, and convey a sense of its unique
character.
• Understand the Market and create a “Brand”: Market and
branding decisions are important in today’s competitive
environment, and should naturally flow from the mission. Avoid
rushing into new markets without careful research/planning.
• Long Range and Strategic Planning: these types of plans
should set the stage, articulating directions chosen to help the
institution achieve its mission. Focus here is on what to do.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Tactical and Operational Planning: These plans have a shorter
time horizon and focus on how to achieve the goals and
purposes that have been set.
• Integrate Academic, Resource, and Facilities Elements:
Academic decisions should be the driving force, but good
planning is holistic in approach.
• Set Priorities and Establish a Timetable: Decisions about
priorities and timelines should recognize the “art of the possible”
for a unique institutional environment.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Plans as Guidelines, not Monuments: The process is much
more important than the plan itself. Change is constant.
• Identify and Implement Appropriate Linkages: Results from
assessments should flow to budget decisions and into new
rounds of planning. Department/division work should link to
institutional work.
• Leadership is Imperative: none of this will happen without
direct support of campus executives.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• This is About Change Management: Respect the institution’s
culture and history; be realistic in expectations; work toward the
sustainable.
• Student Learning Outcomes Assessments and Assessment
of Institutional Effectiveness: Both types of assessment are
important in determining if the institution is achieving its goals.
• Develop a Capacity for Institutional Research: Internal
research and external benchmarking is essential in assessment,
but the function can be institutionally organized in many ways.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Judge Progress and Re-Calibrate the Plan: Consider how the
institution is doing in reaching its goals and purposes, and
whether processes are all they should be.
• Do Not Re-Invent the Wheel: Use external resources such as
the Society for College and University Planning and other topical
higher education organizations and data bases.
• Encourage Broad Participation and Wide Communication:
These characteristics provide a solid foundation for a collegial
process, and improve prospects for a successful implementation.
• Coming in Fall 2009:
Middaugh, M.F. Making Higher Education
Accountable: Creating a Culture of Evidence for
Institutional Effectiveness. San Francisco: JosseyBass Publishers, 2009.
That’s All, Folks!
• What have we missed that you would like covered?
• Other questions????
• [email protected]
• [email protected]