Transcript Integrating Higher Education Planning and Assessment

Integrating Higher Education Planning and
Assessment:
Concepts and Strategies
Michael F. Middaugh
Assistant Vice President for Institutional Research and Planning
University of Delaware
Commissioner and Member of Executive Committee
Middle States Commission on Higher Education
Andrea A. Lex
Dean of Planning and Institutional Research
Prince George’s Community College
“The nicest thing about not planning is that failure comes
as a complete surprise and is not preceded by a period of
worry and depression.”
John Preston, Boston College
“A common mistake that people make when trying to
design something completely foolproof is to underestimate
the ingenuity of complete fools.”
Douglas Adams, Hitchhikers
Guide to the Galaxy
A Brief Overview of Accreditation
What is accreditation?
• A means of providing quality assurance in
the areas of
– Student learning and achievement
– Curriculum development and support
– Faculty
– Facilities and equipment
– Finance, administration, and governance
– Integrity and compliance
Why is it done?
•
•
•
•
•
•
Voluntary self-regulation
Voluntary self-examination
Opportunity for peer review
Quality assurance
Public accountability
And, in the case of regional accreditation
– Ability to receive Federal financial aid dollars
What do we use?
Characteristics of Excellence
Institutional Mission
• Institutional Effectiveness • Educational Effectiveness
–
–
–
–
–
–
–
Planning
Resource Allocation
Leadership
Governance
Administration
Integrity
Institutional Assessment
–
–
–
–
–
–
–
Admissions & Retention
Support Services
Faculty
Offerings
General Education
Related Education Activities
Assessment of Student
Learning
What have we learned?
• That the quality of an institution’s
accreditation self study hinges on how it
addresses the standards related to
– assessing institutional effectiveness
(Standard 7)
– assessing student learning outcomes
(Standard 14)
– implementing a vital and comprehensive
strategic planning process (Standard 2).
Commission-Directed Follow-up
(2004-2009)
• 81 percent required some sort of follow-up
•
•
•
•
•
Student Learning Outcomes
Institutional Effectiveness
Strategic Planning
Sufficient Resources
Enrollment Management
(S 14)
(S 7)
(S 2)
(S 3)
(S 8)
Creating and Closing the Loop
• Institutional Mission is the Foundation of & Our
Reason for Being
Creating and Closing the Loop
Characteristics of Excellence
Institutional Mission
Educational Effectiveness
–
–
–
–
–
–
–
Admissions & Retention
Support Services
Faculty
Offerings
General Education
Related Education Activities
Assessment of Student
Learning
To what extent have students
achieved the educational and
co-curricular goals set forth by
an institution as it recruits and
retains these individuals?
-- Assessment of Student Learning
(Standard 14)
Creating and Closing the Loop
Standard 14: Student Learning
Outcomes
• Assessment of student learning demonstrates
that, at graduation, or other appropriate points,
the institution’s students have knowledge, skills,
and competencies consistent with institutional
and appropriate higher education goals.
Selected Fundamental Elements for
MSCHE Standard 14
• Articulated expectations for student learning (at
institutional, degree/program, and course levels)
• Documented, organized, and sustained assessment
processes (that may include a formal assessment
plan)
• Evidence that student learning assessment
information is shared and used to improve teaching
and learning
• Documented use of student learning assessment
information as part of institutional assessment
Creating and Closing the Loop
Characteristics of Excellence
Institutional Mission
• Institutional Effectiveness
–
–
–
–
–
–
–
Planning
Resource Allocation
Leadership
Governance
Administration
Integrity
Institutional Assessment
To what extent has the
institution acquired and
allocated human and fiscal
resources in a manner that
supports the achievement of
its goals?
-- Assessment of Institutional
Effectiveness (Standard 7)
Creating and Closing the Loop
Standard 7: Institutional Assessment
– The institution has developed and
implemented an assessment process that
evaluates its overall effectiveness in
achieving its mission and goals and its
compliance with accreditation standards.
Selected Fundamental Elements for
MSCHE Standard 7
• Documented, organized, and sustained assessment
processes to evaluate the total range of programs and
services, achievement of mission, and compliance
with accreditation standards
• Evidence that assessment results are shared and
used in institutional planning, resource allocation and
renewal
• Written institutional strategic plan(s) that reflect(s)
consideration of assessment results
• Evidence that institutional assessment findings
are used to improve student success
Creating and Closing the Loop
Characteristics of Excellence
Institutional Mission
Planning
Institutional Effectiveness
– Planning
– Resource Allocation
– Leadership
– Governance
– Administration
– Integrity
– Institutional Assessment
•
Planning
•
Planning
Educational Effectiveness
– Admissions & Retention
– Support Services
– Faculty
– Offerings
– General Education
– Related Education Activities
– Assessment of Student Learning
To what extent are an institution’s policy, decision making and resource
allocation processes shaped by a systematic, evidence-based, and
mission-guided planning process? --- Assessment of Institutional
Planning (Standard 2)
Creating and Closing the Loop
Standard 2: Planning, Resource Allocation
and Institutional Renewal
– An institution conducts ongoing planning and
resource allocation based on its mission and goals,
develops objectives to achieve them, and utilizes
the results of its assessment activities for
institutional renewal.
– Implementation and subsequent evaluation of the
success of the strategic plan and resource
allocation support the development and change
necessary to improve and to maintain quality.
Selected Fundamental Elements for
MSCHE Standard 2
• Clearly stated goals and objectives, at both the institutional and unit
levels, that reflect conclusions drawn from assessments that are
used for planning and resource allocation at those levels
• Planning and improvement processes that are clearly
communicated, provide for constituent participation, and incorporate
the use of assessment results
• Well-defined decision-making processes and authority that
facilitates planning and renewal and the assignment of responsibility
for improvement and assurance of accountability
• Periodic assessment of the effectiveness of planning, resource
allocation, and institutional renewal processes
Central Threads Running Through All
Accreditation Requirements:
• Planning must be systematic
• Planning must be rooted in an institution’s mission
• Planning must be predicated on analytical and
evaluative information
• Planning must be used for institutional decisions,
including resource allocation
The Closed Loop
• Institutions must plan effectively in order to
be effective.
• Where that is the case, the accreditation
process is an affirmation of the evidence
of that effectiveness.
Components of This Curriculum
• Best Practice in Planning
• Best Practice in Assessment
• Closing the Loop Between Planning and
Assessment
• Real Life Case Studies
End Result
• Institutions must plan effectively in order to be
effective.
• Where that is the case, the accreditation process is an
affirmation of the evidence of that effectiveness.
Components of This Curriculum
• Best Practice in Planning
• Best Practice in Assessment
• Closing the Loop Between Planning and Assessment
• Real Life Case Studies
In thinking about what good planning is, it is useful to begin the
discussion by describing what it is not:
George Keller, Academic Strategy: the Management Revolution in
American Higher Education, Johns Hopkins University Press, 1983.
Strategic Planning is not:
• The production of a blueprint.
• A set of platitudes.
• The personal vision of a president or board of trustees.
• A collection of departmental plans, compiled and edited.
Strategic Planning is not:
• Done by planners.
• A substitution of numbers for important intangibles.
• A form of surrender to market conditions and trends.
• Something done on an annual retreat.
• A way of eliminating risks.
• An attempt to read tea leaves and outwit the future.
Strategic Planning IS
•
Academic strategic decision-making means that a college or
university and its leaders are active rather than passive about
their position in history.
•
Strategic planning looks outward and is focused on keeping
the institution in step with the changing environment.
•
Academic strategy making is competitive, recognizing that
higher education is subject to economic market conditions and
to incredibly strong competition.
•
Strategic planning concentrates on decisions, not on
documented plans, analyses, forecasts, and goals.
Strategic Planning IS
•
Strategy making is a blend of rational and economic analysis,
political maneuvering, and psychological interplay. It is
therefore participatory and highly tolerant of controversy.
•
Strategic planning concentrates on the fate of the institution
above everything else.
Cornerstones of the Planning Process
Institutional Mission
ActionOriented
Goals
ActionOriented
Goals
ActionOriented
Goals
Measurable
Objectives
Measurable
Objectives
Measurable
Objectives
Let’s begin at the beginning . .
.
Why are we here?
Mission Statement
• A good mission statement is a carefully reasoned analysis of
what an institution aspires to be, and the core values that it
embraces.
• It avoids cliché language, e.g., “Students and faculty will interact
in a rich intellectual environment in which each individual has the
opportunity to achieve their full potential.” Noble sentiment, but
says nothing about the institution’s purposes and priorities.
• Mission must speak to central institutional issues, e.g., desired
balance between undergraduate and graduate education; relative
emphasis on teaching, research, and service, respectively; and
so on.
Mission Statement
• Mission statements are characterized by a sense of vision that,
while not immutable, nonetheless represents a long-term
statement of institutional values and direction around which
human and fiscal resource allocation decisions can be made.
• While cognizant of the institution’s ever changing external
environment, mission statements are not whimsical, morphing
with each new market trend that emerges.
• The mission statement provides a clear sense of direction around
which action-oriented goal statements and measurable planning
objectives can be developed.
Planning Goals
• Goal statements are derived from the institutional mission, and
help to define policy.
• For example, the mission statement might say that “The
University affirms its historic mission of providing the highest
quality education for its undergraduate students, while
maintaining excellence in selected graduate programs.”
• The mission statement is underscoring the primacy of
undergraduate instruction in the curriculum. The question for
planners is how to provide that high quality undergraduate
instruction.
How will we achieve our
mission?
Planning Goals
• The “how” translates into specific, action-oriented planning goals
aimed at moving the institution toward a fuller realization of its
mission.
• Possible goal related to the undergraduate education mission
statement: “The University will continue to attract and retain the
most academically talented and diverse undergraduate students,
and support their intellectual, cultural, and ethical development
as citizens and scholars.”
• Action verbs such as “attract,” “retain,” and “support” elevate the
goal statement to policy level. How do we know that policy is
being carried out? Measurable planning objectives.
Meeting Goals
• How will we know when we get there?
• What will we look like when we get there?
Planning Objectives
Planning objectives provide empirical evidence of the extent to
which planning goals are being achieved. Consider the following
planning objectives as they relate to our goal to attract, retain,
and support academically talented and diverse students.
• Retain a freshman admissions target of 3200 to 3400 students
annually, with an admissions profile for academic year 2007 of
23,000 applications, a 40 percent admit rate, and a yield rate in
excess of 35 percent.
• Improve the alignment of undergraduate enrollment distribution
and instructional resource distribution across the disciplines,
especially with respect to faculty.
Planning Objectives Continued
• Maintain a freshman-to-sophomore retention rate above the
national average for highly selective institutions, and seek to
achieve a consistent rate of 90 percent or higher.
• Maintain a graduation rate above the national average for highly
selective institutions, and seek to achieve a consistent six-year
rate of 75 percent or higher.
• Increase minority and international enrollment, with retention and
graduation rates for those populations consistent with the
university-wide averages for all students.
How are we doing?
• How close are we to getting there?
• How are we doing along the way?
Planning Objectives
• The defining characteristic for any good planning objective is that
it must be measurable. In other words, assessable.
• Colleges or universities embarking on any planning process –
long range, strategic, tactical – require a systematic institutional
research capability.
• While smaller institutions may not have an office of institutional
research, per se, they must nonetheless have the capability of
quantitatively and qualitatively assessing the extent to which
planning objectives are being implemented, planning goals are
being achieved, and the institutions mission is being realized.
Breakout Session
Using your own Institutional Mission
Statement, create at least five planning
goals and associated measurable planning
objectives that will help assess the extent
to which that mission is being realized.
Linking Planning and Assessment
Two Case Studies at
The Same Institution
The First Cut: Creating a Culture of Planning
University of Delaware
Strategic Planning: 1990 - 2000
University of Delaware Mission Statement
“The University reaffirms its historic mission to provide the highest quality education for its undergraduate
students, while maintaining excellence in selected graduate programs. The faculty are responsible for
helping students learn to reason critically and independently, gain knowledge of the diverse culture and
environment in which they live, and develop into well-informed citizens and leaders. To accomplish
these goals, the University provides a learning setting enriched by undergraduate student research,
experiential learning, and study-abroad programs. The University places high priority on the education of
qualified Delaware residents and provides opportunity for a diverse group of citizens to participate in
postsecondary education. Since the University is located in a state with a small population, providing
programs of quality and diversity requires a community of student-scholars that reaches beyond the
boundaries of the state, one that reflects the nation’s racial and cultural diversity.
“The University of Delaware also aspires to excellence in graduate education, the heart of which is
scholarship and research. The creation, application, and communication of knowledge is a primary goal
of the institution and of every faculty member, providing the substance for creative, informed teaching.
Research is typically based on cooperation between faculty and students, whereby faculty mentors teach
students to conduct independent research and to master problem-solving techniques. Through
involvement of undergraduates in faculty research, the University creates a special bond between its
undergraduate and graduate programs.
“The University is also committed to providing service to society, especially in Delaware and the
neighboring region. Public service is a responsibility of every academic unit. In addition, each faculty
member is responsible for service to the University community and to his or her profession. The
University emphasizes practical research, provides extension services, and works to solve problems
confronting the community.
(University of Delaware, Middle States Self-Study Report, April 2001)
Case Study – University of Delaware
• In 1987, the University embarked on a comprehensive, longrange planning process, termed Project Vision. Over a period of
18 months, the campus developed a planning document with a
broad spectrum of planning goals and measurable objectives
embracing all aspects of University operations.
• In Fall of 1988, the President who initiated Project Vision
suddenly resigned. At the same time, the Delaware economy
along with that of the entire mid-Atlantic region was plunging into
deep recession
Case Study – University of Delaware
• Rather than let 18 months of planning activity go for naught –
even though resources would be scarce for the foreseeable
future – a panel of distinguished senior faculty was assembled to
review the Project Vision planning document and to cull out those
goals and objectives that were clearly consistent with, and
essential to furthering the University mission.
• The resulting document, Focused Vision, was economical when
compared with its progenitor, both in terms of words and
resource requirements. However, the economy would still clearly
preclude anything even remotely approaching implementation.
• In 1990, the University hired its 25th President, David P. Roselle.
Case Study – University of Delaware
• In order to maintain planning momentum, the President
consulted with his senior staff, the faculty, and appropriate
constituencies across campus to determine those areas that
required immediate attention.
• From these consultations, the President articulated four strategic
initiatives that would constitute the focus of decision-making and
resource allocation activity in the immediate future. Those
initiatives were competitive compensation for faculty and staff;
enhanced access to the University for undergraduates through
expanded availability of financial aid; a more student-centered
campus environment; and renovation and rehabilitation of
campus facilities.
Case Study – University of Delaware
These priorities were not a “wish list.” They grew out of a careful
examination of empirical data provided by the University’s Office
of Institutional Research and Planning and other data sources.
Consider the following:
• When compared with the 24 Category Doctoral I universities in
the states contiguous to Delaware, and the District of Columbia,
in 1991 the average salary for all three major faculty ranks at the
University of Delaware ranked near the bottom of the list.
• The Student College Selection Survey indicated that students
were receiving offers of more aid from admissions competitors,
and that the aid packages had more grants and fewer loans than
University aid packages. Not surprising, the University was at a
competitive disadvantage for academically talented students.
Case Study – University of Delaware
• University scores on the ACT Student Opinion Survey suggested
that the institution had considerable room for improvement with
respect to student satisfaction with programs and services, and
with a number of areas in student life.
• The University was looking at in excess of $200 million in
deferred maintenance to its buildings and grounds.
Case Study – University of Delaware
• A critical factor in moving forward with these initiatives was
getting the campus to understand that the economy was in
recession and that there would be no immediate or massive
infusions of new resources.
• Colleges and universities have multiple revenue streams –
tuition, state appropriation in the case of public institutions,
contracts and grants, gifts, etc. While growing revenue streams
is an important strategic initiative, so too is the commitment to
not balance budgets on the backs of students through
inordinately large tuition increases.
• Resource reallocation would be the primary source of funding the
four strategic initiatives, and it was critical that the campus
understand from where funds were reallocated, and why.
The University went on public record in 1991:
• Average total compensation for faculty at each academic rank
would be at or above the median within five years for the 24
Category I Doctoral Universities identified as salary peers.
• Total undergraduate financial aid from all sources would increase
by 100 percent within five years.
• Student satisfaction with programs and services at the University,
as measured through the ACT Student Opinion Survey would
demonstrate significant gains within five years.
• The University would commit itself to a policy of annually setting
aside at least 2 percent of the replacement value of the physical
plant, to be used for facilities renovation and rehabilitation.
Results - Salaries
Total Compensation, 1989-90 Compared with 1999-2000
FULL PROFESSOR
Institution
1989-90
1999-2000
Princeton University
91,800
University of Pennsylvania
151,900
University of Pennsylvania
91,500
Princeton University
134,100
Georgetown University
88,700
Georgetown University
126,100
Carnegie-Mellon University
85,800
Carnegie-Mellon University
123,200
University of Virginia
85,100
University of Virginia
122,900
George Mason University
84,400
UNIVERSITY OF DELAWARE
116,200
Johns Hopkins University
83,800
Temple University
114,300
New Jersey Institute of Technology
83,100
George Mason University
113,900
Rutgers University - New Brunswick
82,000
College of William and Mary
113,400
Lehigh University
80,800
Lehigh University
111,300
University of Maryland - College Park
80,500
American University
111,100
Virginia Polytechnic Inst. & State University
76,700
Johns Hopkins University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Maryland - Baltimore County
76,700
George Washington University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Pittsburgh - Main Campus
76,500
New Jersey Institute of Technology
109,200
George Washington University
76,500
Rutgers State University-N.Brswck
109,000
American University
76,300
Pennsylvania State University
108,600
Pennsylvania State University
75,300
University of Maryland - College Park
106,800
UNIVERSITY OF DELAWARE
74,700
University of Pittsburgh - Main Campus
105,600
College of William and Mary
74,400
Virginia Polytechnic Inst. & State University
103,800
Virginia Commonwealth University
72,600
Drexel University
103,700
Drexel University
71,000
Virginia Commonwealth University
103,300
Old Dominion University
70,400
University of Maryland - Baltimore County
96,200
Temple University
68,400
Howard University
93,300
Catholic University of America
64,800
Catholic University of America
Howard University
64,000
Old Dominion University
86,800
No Report
Results – Financial Aid
Growth In Undergraduate Financial Aid at the
University of Delaware, FY 1990 Compared with
FY 2000
FY 1990
FY 2000
% Increase
University Administered Funds
$4,458,640
$28,036,660
528.8
State Grant Funds
$3,869,000
$6,643,500
71.7
Other Fund Sources
$2,169,602
$4,394,180
102.5
$10,497,242
$39,074,340
272.2
TOTAL
Results – Student Satisfaction
Comparison of 1995 University of Delaware Scores on ACT
Student Opinion Survey with 1990 Scores and with National
Norms
1990 Scores
Compared with
National Norms
1995 Scores
Compared With
1990 Scores
1995 Scores
Compared with
National Norms
University is Ahead
21
44
39
University is Tied
14
11
15
University is Behind
27
7
8
Results - Facilities
• By 2000, the University had renovated every classroom in its
entire building inventory, retrofitting most with state-of-the-art
teaching technology.
• An aggressive program of fundraising enabled not only the
aforementioned renovation and rehabilitation, but also the
construction of several new classroom and student services
buildings.
• The University is now on a cycle of planned maintenance, as
opposed to deferred maintenance.
Results – From an Accreditation Perspective
“ The
University of Delaware has every reason to take enormous pride in
what it has accomplished over the past 10 years. A decade ago, it was
coming out of a period of considerable turmoil. Today, the University is
seen as a national model for the integration of information technology in
every aspect of university life: teaching and learning, research and
service, academic support, and campus administration. It has created
a physical plant that has few, if any, peers among public universities
and would be the envy of most private colleges. These substantial
achievements could not have happened without extraordinary
leadership from the senior administration.”
“ Better than almost any university we are familiar with, Delaware has a
clear sense of what it wants to be, namely, a university that offers high
quality undergraduate education with targeted areas of excellence in
graduate education and research.”
" The review team was enormously impressed by the high level of morale
that pervades the faculty, staff, and students. Almost without exception,
the people we spoke to take great pride in being part of the University.”
Middle States Evaluation Team, 2001
• The University submitted its Periodic Review Report to
the Commission on Higher Education in May 2006,
outlining its strategies for incorporating systematic
assessment of student learning outcomes into all
facets of campus planning.
• The Commission thanked the University for its Report,
commended the University on its content, and
reaffirmed accreditation.
What Was Not Apparent to MSCHE…
• Despite an exemplary self study and PRR process,
planning was not yet institutionalized into a formal,
iterative process at the institution.
• The four strategic initiatives articulated in 1990
continued to be the benchmarks against which the
University measured success well into the 2000
decade.
• The University needed to move forward with a new
strategic planning effort that would more substantively
incorporate planning into the organizational fiber of the
institution.
The Second Cut: Institutionalizing Planning
University of Delaware
Strategic Planning: 2007 - ???
The Path to Prominence
• On July 1, 2007, Patrick T. Harker succeeded David
Roselle as the 26th President of the University.
• As former Dean of the Wharton School at the
University of Pennsylvania, President Harker was
intimately familiar with the concept of strategic
planning.
• He immediately appointed a Strategic Planning
Committee led by two distinguished senior faculty and
comprised largely of individuals from the academic as
opposed to administrative side of the institution.
The Path to Prominence
• During the 2007-08 academic year, the Strategic
Planning Committee met with over 100 constituent
groups from within, and external to the University.
These information-gathering sessions were part of a
comprehensive SWOT analysis.
• In April 2008, the Report of the Strategic Planning
Committee was delivered to the President.
• In July, 2008, the President presented the Path to
Prominence, the strategic planning document that
would guide institutional activity for the next several
years.
The Path to Prominence: The Strategic Initiatives
• A Diverse and Stimulating Undergraduate Academic
Environment
• A Premier Research and Graduate University
• Excellence in Professional Education
• The Initiative for the Planet
• The Global Initiative
• An Engaged University
The Path to Prominence: The Strategic Initiatives
• Each of the six initiatives was operationalized with
measurable goals and planning objectives.
• The Path to Prominence document and supporting
materials were broadly circulated via the University
website: http://www.udel.edu/prominence/ for maximum
transparency and accountability with respect to what was
being accomplished.
• In Fall 2008, the U.S. and Delaware economies began to
unravel, placing severe financial constraints on full
implementation of the strategic initiatives.
The Path to Prominence: The Strategic Initiatives
• While each of the six initiatives remain viable and will receive
attention regardless of economic circumstances, hard decisions
needed to be made with respect to where resources would be
allocated.
• It was determined that three initiatives would be the primary
(but not exclusive) focus of resource allocation:
- A Diverse and Stimulating Undergraduate Academic Environment
- A Premier Research and Graduate University
- Excellence in Professional Education
The Path to Prominence: Moving Forward
• The University decided to take advantage of the MSCHE
decennial accreditation reaffirmation process, and utilize the
“Special Topics Model” of self study as a means of
documenting progress in implementing these three key
strategic initiatives.
• The scheduled 2011 site visit by an MSCHE Evaluation Team
affords the University not only the opportunity to demonstrate
compliance with the 14 standards in Characteristics of
Excellence; it provides us with a team of expert consultants in
the topical areas on which we have elected to focus. Their
feedback on the content of the Special Topics Self Study will
prove invaluable in moving the institution forward.
The Path to Prominence: Tactical Issues
• In any strategic planning process, there needs to be a clear
linkage between human and fiscal resource allocation and the
initiatives outlined in the strategic plan.
• The University of Delaware has adopted a responsibility
centered budget management model that is designed to
financially reward activity that supports full implementation of
key strategic initiatives.
• The University has also designated an Associate Provost for
Institutional Effectiveness, tasked with overseeing assessment
activity that measures the effective implementation of The Path
to Prominence.
The Path to Prominence: Tactical Issues
• The Steering Committee for the 2011 MSCHE has tasked itself
with responsibility for design of a formal, iterative institutional
strategic planning process that will, on an ongoing basis,
continuously evaluate the implementation of existing strategic
initiatives, assess the continuing relevance of initiatives over
time, and develop new initiatives, as appropriate, in the face of
a changing external environment and/or advent of specific
strategic opportunities.
• The intent is to institutionalize the strategic planning process so
that it is truly an organizational function rather than the creation
or vision of any particular individual.
Considerations When Creating a
Strategic Planning Process
“Planning to Plan:”
Organizing Your Institution
for the Planning Process
Organizational Structure
• The Planning Committee: Who Serves? How
Representative? What is the Scope of
Responsibilities:
• If the Planning Committee is to be effective, it must be
“blessed” by the Chief Executive Officer and the Chief
Academic Officer. The President and Provost need
not be members of the Committee, but ideally will
attend the initial meeting, and will formally receive the
Committee Report.
Organizational Structure
• Who staffs the Planning Committee?
Institutional Research
Budget Office
Others
Faculty
Staff
Planning Tools
• Surveys: Students
–
–
–
–
–
Admitted Student Questionnaire
Entering Student Needs Assessment Survey
Student Satisfaction Survey
Post-Graduation Placement Survey
Alumni Survey
• Surveys: Employees
– Employee Satisfaction Survey
– Faculty Activity Analysis
– Campus Climate Survey
Planning Tools
• Institutional Studies:
–
–
–
–
–
–
–
–
Admissions Monitoring Analyses
Faculty Workload Analyses
Budget Support Data
Persistence/Graduation Analyses
Compensation Studies: Market/Equity
Financial Ratio Analyses
Benchmarking Studies
Economic Impact Studies
Planning Tools
•
Academic Program Review
–
–
–
Programmatic Self-Examination
Internal and External Review Teams
Focus on Several Levels of Quality
1.
2.
3.
4.
Programs With National/International Distinction
Programs of Regional Distinction (Meet Local Needs)
Programs That, With an Infusion of Resources, Could Achieve
Distinction
Programs Targeted for Retrenchment or Elimination
Planning Tools
•
Environmental Scan:
a.
b.
c.
d.
e.
f.
g.
Demographic issues
Market issues
Political issues
Economic issues
Technological issues
Natural disasters
Other
Planning Tools – SWOT Analysis
• Take an honest look at your institution within the
context of a SWOT analysis (Strengths; Weaknesses;
Opportunities; Threats)
• Leveraging Situation - Opportunity + Strength
Constraining Situation - Opportunity + Weakness
Vulnerable Situation – Threat + Strength
Problematic Situation – Threat + Weakness
In the Final Analysis, Planning is Directed at
Answering Four Basic Questions:
1.
Who are the markets we are trying to serve?
2.
What services must be in place to fully serve those
markets?
3.
What is the institutional “branding” that will enable
our college or university to appeal to those markets?
4.
How will we know if we are successful in serving
those markets?
Who are the markets we are trying to serve?
Largely Dictated by Mission
• Baccalaureate Colleges: Primarily undergraduate students (what
kinds?)
• Master’s Institutions: Undergraduate students, selected graduate
students, targeted state/community partnerships
• Doctoral Universities: Undergraduate students, graduate
students, research/public service contractors/grantors, broad
range of regional, national, international partnerships
• Community Colleges: Matriculated undergraduate students,
occasional students with specific training needs, general interest
students, local businesses, county government
What services must be in place to fully serve those
markets?
Dictated by Markets
• Academic Support Services (Library, computing, advising,
tutoring, etc.)
• Student Support Services (Residence life, counseling, health
services, student center, recreation services, etc.)
• Institutional Support Services (General administrative support,
sponsored research, extension office, alumni/governmental
relations, etc.)
What is the institutional “branding” that will enable our
college or university to appeal to those markets?
Market Specific
• University of Delaware:
“A Teaching University”
• Trenton State College :
“The College of New Jersey”
• University of Phoenix:
“The University for Working Adults”
How will we know if we are successful in serving
those markets?
Middle States Assessment Standards
• Standard 7: The institution has developed and implemented
an assessment process that evaluates its overall
effectiveness in achieving its mission and goals and its
compliance with accreditation standards.
• Standard14: Assessment of student learning demonstrates
that, at graduation, or other appropriate points, the
institution’s students have knowledge, skills, and
competencies consistent with institutional and appropriate
higher education goals.
Breakout Session
How would you respond to the four key
planning questions within the context of
your institution’s mission?
In the Final Analysis, Planning is Directed at
Answering Four Basic Questions:
1.
Who are the markets we are trying to serve?
2.
What services must be in place to fully serve those
markets?
3.
What is the institutional “branding” that will enable
our college or university to appeal to those markets?
4.
How will we know if we are successful in serving
those markets?
What Should Be Assessed Under
Standard 7?
• You do not have to measure everything.
• PRIORITIZE what is to be measured within the
context of your institution’s culture and needs.
Students
•
•
•
•
•
•
Admitted
Entering
Continuing
Non-Returning
Graduating
Alumni
Environmental Issues
•
•
•
•
Student and Faculty Engagement
Student and Staff Satisfaction
Employee Productivity
Compensation
- Market
- Equity
• Campus Climate
• Economic Impact
Some Basic Measures
A Typical Admissions Monitoring Report
Eastern Seaboard State University
Weekly Admissions Monitoring Report
Campus Summary: First-Time Freshman Applicants, Their SAT Scores and Predicted Grade Index, by Admission Status and by Residency Status for the Entering Classes in the
Fall of 2005 as of 09/15/05; Fall 2006 as of 09/18/06; and Fall 2007 as of 09/13/07.
Counts
- Resident
- Nonresident
- Total
All
Applicants
2005 2006 2007
Admission
Denied
2005 2006 2007
Admission
Offered
2005 2006 2007
Admission
Accepted
2005 2006 2007
2,340 2,332 2,088
18,984 19,209 20,133
21,324 21,541 22,221
109
148
172
7,506 5,871 5,838
7,615 6,019 6,010
1,940 1,877 7,147
7,295 8,101 8,489
9,235 9,978 15,636
1,362 1,255 1,174
2,078 2,201 2,348
3,440 3,456 3,522
SAT Verbal
- Resident
- Nonresident
- Total
557
572
571
562
579
577
569
584
582
456
541
540
444
535
532
445
536
533
567
609
600
576
615
607
584
620
614
559
592
579
564
600
587
575
604
594
SAT Math
- Resident
- Nonresident
- Total
563
596
592
567
598
595
574
605
602
454
561
560
446
551
549
452
554
551
547
636
623
582
635
625
589
643
634
566
619
598
570
620
602
580
627
611
Predicted Grade
Index
- Resident
- Nonresident
- Total
2.67
2.90
2.82
2.75
2.94
2.88
2.83
2.94
2.88
1.66
2.40
2.22
1.60
2.36
2.30
1.71
2.41
2.37
2.75
2.98
2.90
2.78
3.04
2.95
2.85
3.05
2.98
2.75
2.98
2.89
2.78
3.05
2.95
2.85
3.04
2.98
Ratio of Offers
to Total Applications
2005 2006 2007
0.83
0.38
0.43
0.80
0.42
0.46
3.42
0.42
0.70
Ratio of Accepts
to Offers (Yield)
2005 2006 2007
0.70
0.28
0.37
0.67
0.27
0.35
0.61
0.28
0.23
Drilling Down
• Why do some students to whom we extend an offer of
admission choose to attend our institution?
• Why do other students to whom we extend an offer of
admission choose to attend a different school?
• How is our institution perceived by prospective
students within the admissions marketplace?
• What sources of information do students draw upon in
shaping those perceptions?
• What is the role of financial aid in shaping the college
selection decision?
Continuing/Returning Students
•
Student Satisfaction Research
- ACT Survey of Student Opinions
- Noel-Levitz Student Satisfaction Inventory
•
ACT Survey of Student Opinions
Student use of, and satisfaction with 21 programs and services typically found
at a college or university (e.g. academic advising, library, computing, residence
life, food services, etc.)
Student satisfaction with 43 dimensions of campus environment (e.g., out-ofclassroom availability of faculty, availability of required courses, quality of
advisement information, facilities, admissions and registration procedures, etc.)
Self-estimated intellectual, personal, and social growth; Overall impressions of
the college experience
NOTE: Survey is available in four-year and two-year college versions.
What About Non-Returning Student Research?
TABLE 1: ENROLLMENT, DROPOUT RATES AND GRADUATION RATES
FOR FIRST-TIME FRESHMEN ON THE NEWARK CAMPUS (Total)
Enrollment and Dropout Rates
Entering
Fall Term
Graduation Rates
1st
Fall
2nd
Fall
3rd
Fall
4th
Fall
5th
Fall
6th
Fall
within within
3 yrs
4 yrs
within
5 yrs
1995 N
% enrollment
% dropout
3154
100.0%
0.0%
2673
84.7%
15.3%
2439
77.3%
22.7%
2355
75.3%
24.7%
599
19.0%
26.4%
113
3.6%
26.1%
21 1721 2219 2344
0.7% 54.6% 70.4% 74.3%
1996 N
% enrollment
% dropout
3290
100.0%
0.0%
2804
85.2%
14.8%
2585
78.6%
21.4%
2489
76.3%
23.7%
606
18.4%
26.1%
108
3.3%
26.7%
22 1825 2302 2427
0.7% 55.5% 70.0% 73.8%
1997 N
% enrollment
% dropout
3180
100.0%
0.0%
2766
87.0%
13.0%
2523
79.3%
20.7%
2436
77.5%
22.5%
581
18.3%
24.3%
117
3.7%
24.5%
27 1827 2284 2401
0.8% 57.5% 71.8% 75.5%
1998 N
% enrollment
% dropout
3545
100.0%
0.0%
3080
86.9%
13.1%
2830
79.8%
20.2%
2762
78.5%
21.5%
653
18.4%
22.9%
118
3.3%
22.7%
22 2079 2621 2727
0.6% 58.6% 73.9% 76.9%
1999 N
% enrollment
% dropout
3513
100.0%
0.0%
3126
89.0%
11.0%
2871
81.7%
18.3%
2757
79.4%
20.6%
526
15.0%
22.6%
83
2.4%
22.7%
31 2193 2632 2684
0.9% 62.4% 74.9% 76.4%
2000 N
% enrollment
% dropout
3128
100.0%
0.0%
2738
87.5%
12.5%
2524
80.7%
19.3%
2453
79.2%
20.8%
496
15.9%
23.9%
85
2.7%
23.8%
24 1884 2297
0.8% 60.2% 73.4%
--
2001 N
% enrollment
% dropout
3358
100.0%
0.0%
2976
88.6%
11.4%
2746
81.8%
18.2%
2674
80.6%
19.4%
472
14.1%
22.3%
0
0.0%
0.0%
31 2138
0.9% 63.7%
--
--
Total
What About Non-Returning Student Research?
Drilling Deeper…..
• Commercial instruments exist, but response rates tend to be low,
and reported reasons for leaving politically correct – personal or
financial reasons.
• For the last several years, we have administered the Survey of
Student Opinions during the Spring term to a robust sample of
students across freshman, sophomore, junior, and senior
classes.
• The following Fall, the respondent pool is disaggregated into
those who took the Survey and returned in the Fall, and those
who took the Survey, did not return in the Fall, and did not
graduate.
• Test for statistically significant differences in response patterns
between the two groups.
Student Engagement
• College Student Expectations Questionnaire (CSXQ)
• College Student Experiences Questionnaire (CSEQ)
• National Survey of Student Engagement (NSSE)
Graduating Students
EMPLOYMENT AND EDUCATIONAL STATUS OF
BACCALAUREATES BY CURRICULUM GROUP
CLASS OF 2005
Curriculum Group
Number
in
Class
Number of
Respondents¹
N
%
Full-Time
Employment
%
Part-Time
Employment
%
Pursuing
Further
Education
%
Still
Seeking
Employment
%
Agriculture & Natural
Resources
175
63
36.0
58.7
12.7
19.0
7.9
Humanities
417
145
34.8
67.6
10.3
13.1
9.0
Social Sciences
904
295
32.6
71.2
4.4
14.2
8.5
Life & Health Sciences
158
63
39.9
73.0
--
25.4
--
Physical Sciences
166
61
36.7
72.1
--
16.4
9.8
Business & Economics
571
227
39.8
89.9
3.1
2.2
4.4
Engineering
189
103
54.5
85.4
--
11.7
1.9
Health Sciences
390
144
36.9
72.9
2.8
21.5
2.8
Human Services,
Educ. & Public Policy
557
213
38.2
85.9
2.8
4.2
5.6
2005 University Total2
3,527
1,314
37.3
77.2
4.0
11.9
5.9
Arts & Sciences
NOTE: Percentages may not total to 100.0 because of rounding
Alumni Research
• Commercially prepared instruments exist. Decide if
they meet your needs or if you have to develop your
own.
• Decide early on why you are doing this research: Are
you assessing the continuing relevance of the college
experience? Are you cultivating prospects for the
Development Office? Both? Be up front if fund raising
is a component.
• Decide the which classes you need to survey; don’t go
after every living alumnus unless you are a very young
institution.
Non-Student Measures of
Institutional Effectiveness:
Teaching Productivity, Instructional Costs,
and Externally Funded Scholarship
Transparency in Assessment
www.udel.edu/ir
What About Administrative Productivity???
10 Key Strategic Indicators for
Institutional Health
1.
Revenue Structure
2.
Expenditure Structure
3.
Excess (Deficit) of Current Fund Revenues Over Current Fund
Expenditures
4.
Percentage of Freshman Applicants Accepted and Percentage
of Applicants Who Enroll
5.
Ratio of Full Time Students to Full Time Faculty
10 Key Strategic Indicators for
Institutional Health
6.
Institutional Scholarship and Fellowship Expenditures as a
Percent of Total Tuition and Fees Income
7.
Tenure Status of Full Time Faculty
8.
Percent of Total Full Time Equivalent Employees Who Are
Faculty
9.
Estimated Maintenance Backlog as a Percentage of the
Replacement Value of Plant
10. Percent of Living Alumni Who Have Given At Any Time During
The Past Five Years
Think of the Budget as Buckets of Money
Two Kinds of Buckets:
• Revenues
• Expenditures
Major Revenue Streams
•
•
•
•
•
•
•
Tuition and Fees
Governmental Appropriations
Contracts and Grants
Endowment Income
Sales and Services of Educational Activities
Auxiliary Operations
Other
Major Expenditure Categories
•
•
•
•
•
•
•
•
•
•
•
Instruction
Research
Public service
Academic Support
Student Services
Institutional Support
Operation and Maintenance of Plant
Student Aid
Mandatory Transfers
Non-Mandatory Transfers
Auxiliary Enterprises
Note: The term “Education and General Expenditures” refers to the core costs of doing
business related to the institution’s mission of teaching, research, and service. It does not
include non-mission related activities which typically fall under the umbrella of auxiliary
enterprises.
Ratio Analysis:
Revenue Contribution Ratios
Expenditure Demand Ratios
Tuition Dependency Ratio
Net Tuition
Education and General Expenditures
Other Important Revenue Contributors
• State Appropriation
• Gifts
• Endowment Income
Expenditure Demand Ratios
Instructional Demand Ratio
Instructional Expenditures
Total Operating Income
Institutional Support Demand Ratio
Institutional Support Expenditures
Total Operating Income
Useful Resource
IPEDS Peer Analysis System
http://nces.ed.gov/ipedspas/mainScreen.asp
Institutional “Dashboards” that report on key
success indicators can be an succinct means of
reporting on basic measures of institutional
effectiveness.
www.udel.edu/ir/UDashboard
Claims of institutional effectiveness are
stronger when focal institution’s measures on
important measures are compared with those
of peer institutions.
Choosing Peer Groupings
• Scientific – Cluster Analysis or Other Multivariate Tool
• Pragmatic - e.g. Compensation Peers
• “Whatever!” - e.g. Admissions Peers
We are extending the Dashboard concept to include key
variables related to the University’s new Strategic Plan
that enable us to compare our position vis-à-vis actual
peers and aspirational peers.
www.udel.edu/ir/UDashboard_peers
Dashboard Variables Being Collected
STUDENT CHARACTERISTICS
Percent of Accepted Applicants Matriculated
Total Minority Percentage in Undergraduate Student Body
Freshman to Sophomore Retention Rate
Four Year Graduation Rate
Six Year Graduation Rate
RESEARCH ACTIVITY
Total R&D Expenditures per Full Time Faculty
Total Service Expenditures per Full Time Faculty
FINANCE
University Endowment as of June 30
Alumni Annual Giving Rate
INSTRUCTION
Full Time Students per Full Time Faculty
Total Degrees Granted
Total Doctoral Degrees Granted
Percent of Faculty With Tenure
Percent of Faculty That Are Full Time
Percent of Women Among Full Time Faculty
Percent of Minorities Among Full Time Faculty
Percent of Minorities Among Full Time Staff
Total Faculty Who Are National Academy Members
What Should Be Assessed Under
Standard 14?
Assessing Student Learning Outcomes
• I’ll provide only a brief overview, as there are others
(Linda Suskie, Trudy Banta, Jeff Seybert) who are far
better versed than I am.
• That said, understand that assessment of student
learning is at the core of demonstrating overall
institutional effectiveness.
• Assessment of student learning is a direct response to
the inadequacy of student grades for describing
general student learning outcomes.
According to Paul Dressel of Michigan State
University (1983), Grades Are:
“ An inadequate report of an inaccurate judgment by
a biased and variable judge of the extent to which
a student has attained an undefined level of
mastery of an unknown proportion of an indefinite
material. ”
There is no “one size fits all” approach to
assessment of learning across the disciplines
None of these should be applied to evaluation of individual student performance
for purposes of grading and completion/graduation status.
1. Standardized Tests
2.
3.
4.
5.
• General Education or Discipline Specific
• State, Regional, or National Licensure Exams
Locally Produced Tests/Items
• “Stand Alone” or Imbedded
Portfolios/Student Artifacts
• Collections of Students’ Work
• Can Be Time Consuming, Labor Intensive, and Expensive
Final Projects
• Demonstrate Mastery of Discipline and/or General Education
Capstone Experiences/Courses
• Entire Course, Portion of a Course, or a Related Experience
(Internship, Work Placement, etc.)
Tying It All Together……
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Start from institutional Mission: The Mission Statement should
define the institution and its core values, communicate what
business(es) it will engage in, and convey a sense of its unique
character.
• Understand the Market and create a “Brand”: Market and
branding decisions are important in today’s competitive
environment, and should naturally flow from the mission. Avoid
rushing into new markets without careful research/planning.
• Long Range and Strategic Planning: these types of plans
should set the stage, articulating directions chosen to help the
institution achieve its mission. Focus here is on what to do.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Tactical and operational Planning: These plans have a shorter
time horizon and focus on how to achieve the goals and
purposes that have been set.
• Integrate Academic, Resource, and Facilities Elements:
Academic decisions should be the driving force, but good
planning is holistic in approach.
• Set Priorities and Establish a Timetable: Decisions about
priorities and timelines should recognize the “art of the possible”
for a unique institutional environment.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Plans as Guidelines, not Monuments: The process is much
more important than the plan itself. Change is constant.
• Identify and Implement Appropriate Linkages: Results from
assessments should flow to budget decisions and into new
rounds of planning. Department/division work should link to
institutional work.
• Leadership is Imperative: none of this will happen without
direct support of campus executives.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• This is About Change Management: Respect the institution’s
culture and history; be realistic in expectations; work toward the
sustainable.
• Encourage Broad Participation and Wide Communication:
These characteristics provide a solid foundation for a collegial
process, and improve prospects for a successful implementation.
• Student Learning Outcomes Assessments and assessment
of Institutional Effectiveness: Both types of assessment are
important in determining if the institution is achieving its goals.
Helpful Hints for Successfully Integrating
Planning, Assessment, and Other Processes
• Develop a Capacity for Institutional Research: Internal
research and external benchmarking is essential in assessment,
but the function can be institutionally organized in many ways.
• Judge Progress and Re-Calibrate the Plan: Consider how the
institution is doing in reaching its goals and purposes, and
whether processes are all they should be.
• Do Not Re-Invent the Wheel: Use external resources such as
the Society for College and University Planning and other topical
higher education organizations and data bases.