Transcript REPORT OF THE STETSON UNIVERSITY KEY INDICATORS …

REPORT OF THE STETSON
UNIVERSITY KEY INDICATORS
ACADEMIC WORK GROUP
(KIG)
February 2, 2010
Revised March 12, 2010
The KIG Committee
John K. Schorr, Senior Professor of Sociology, Chair
Erich Friedman, Associate Professor of Mathematics
and Computer Science
Laura Glander, Administrative Assistant II,
Department of Physics
W. Tandy Grubbs, Professor of Chemistry
W. Daniel Hale, Professor of Psychology
Janis B. Kindred, Professor of Music
Susan Ryan, Professor and Associate Director of the
duPont-Ball Library
Theodore J. Surynt, Professor of Decision and
Information Sciences
John M. Tichenor, Director, Institutional Research Office,
Assistant Professor of Decision and Information Sciences
Charge to the Committee
“Define key indicators and analyses that can be
used by academic leaders in the College,
Schools, Departments, and Programs to study
the “economies” of our academic programs
(e.g., course and program enrollment
distributions, student-faculty ratios, faculty
workload, major and minor counts, change of
major paths, program costs, net revenue)”
Source: SU Planning Document
www.stetson.edu/administration/academics/planning.pdf
Clarification of the Charge
Once formed, the committee sought clarification from the provost
about the scope of their work. After discussion between the provost
and the committee, it was agreed that:
• The work of KIG is to determine and recommend indicators for
academic planning at Stetson. These indicators should be suitable
for the support of academic decision making and to serve as guides
for academic planning.
• The focus of KIG is the academic program (i.e., data for the
indicators would be collected at the departmental or program level).
• Indicators should apply to all undergraduate and graduate programs
on the DeLand and Celebration campuses. The College of Law may
also use some of the indicators recommended by KIG, but the
College of Law should not be a specific focus for the current KIG
effort.
Clarification of the Charge (continued)
• KIG’s task is to define the indicators needed and direct, where
necessary, how they should be gathered in order to build a useful
academic program review system. KIG is not asked to gather these
data; nor is it asked to set university targets for the indicators
recommended.
• KIG recommendations should assume that data for each indicator
recommended would be collected annually by the appropriate
academic or administrative unit and data entry, storage, and access
would be coordinated by the Institutional Research Office.
• The general goal of KIG is to recommend indicators for a system of
academic decision making and planning that is more data-driven,
principle-driven, and transparent. This system will help to determine
what constellation of academically excellent programs can be
offered at Stetson University in a way that is financially sustainable.
Committee Organization
After agreement on the charge and the scope of KIG’s
work, the committee determined that the following
themes were central to the development of a system of
indicators for academic programs at Stetson (committee
members responsible in parentheses):
• Measures of program cost, program revenue, and
program net revenue (Grubbs, Ryan, Surynt).
• Measures of program student demand (Hale, Schorr).
• Measures of program faculty load (Glander, Kindred).
• Measures of program quality/reputation (Friedman).
Findings and Recommendations
After two months of research and discussion, and after
considering hundreds of possible indicators, the KIG
committee agreed to recommend 28 indicators. We
recommend that each of these indicators be based
initially on a 3-year average rather than a one-year
result. It was felt that use of a 3-year average would
reduce the effects of one-year programmatic variations
that might not be typical (e.g., sabbatical leaves, large
expenditures for equipment, an exceptionally large
graduating class, etc.). As the data base envisioned by
KIG grows over the next few years, we also recommend
examination of the possible use of 5-year averages; this
would further reduce the effect of one-year fluctuations in
program results.
KIG Topic Group 1: Net Revenue and Cost
1. Program/Department Net Revenue ( Model 1 = sum of tuition & fees for
each departmental major or program participant
- unfunded discount / Departmental or Program FTE Faculty). (Source:
Finance Office)
2. Program/Department Net Revenue ( Model 2 = Departmental major or
program annual CHG / sum of all SU annual CHG
X Annual SU Net Tuition & Fee Revenue / Departmental or Program FTE
Faculty). (Source: Finance Office)
3. Program/Department External Funding ( Revenue from external grants,
contributions, earnings, etc.). (Source: Finance Office)
4. Program/Departmental Labor Costs (All faculty & staff salary and benefits
/dept. for A&S & SoBA & school for SOM) (Source: Finance Office).
5. Program/Departmental Direct Costs (All non-salary expenditures/dept).
(Source: at College & School level Finance Office, Dept. Level TBD).
6. Program/Departmental Net Revenue (Total Net Revenue Model 1 or 2 +
External Funding - Labor and Direct Costs) (Source: TBD)
KIG Topic Group 2: Student Demand
7. Course Demand (Total CHG by Department or Program) (Source:
IRO)
8. Course Demand per FTE Faculty (Total CHG by Department or
Program / Departmental or Program FTE Faculty)(Source: IRO)
9. Academic Major and Minor Demand (Total Majors and Minors by
Department or Program)(Source:IRO)
10. Academic Major Demand per FTE Faculty (Total Majors in
Department or Program / Department or Program FTE Faculty)
(Source: IRO)
11. Graduating Majors (Total Annual Graduates by Department or
Program) (Source: IRO)
12. Graduating Majors / Faculty FTE (Total Annual Graduates by
Department or Program/ Department or Program Faculty FTE)
(Source: IRO)
13. Prospective Student Demand (# of prospective students
expressing interest in a department. or program) (Source: IRO)
14. Geographic Student Demand (# of prospective students
expressing interest in a department or program by geographic
region) (Source: IRO)
KIG Topic Group 3: Faculty Workload
15. Standard Format Course CHG (lecture, discussion, seminar
courses, etc.) (Source: course designation TBD)
16. Intensive Format Course CHG (writing intensive, theater/music
production, lecture w/ lab, ensemble, etc.) (Source: course
designation source TBD)
17. Independent Format Course CHG (seminar, thesis, internships,
advanced tutorial, independent study, lab-format course, studio,
etc.)(Source: course designation source TBD)
18. Departmental/Program Scholarly Load (papers, publications,
performances, exhibits, etc.) (Source: IRO, Deans, Chairs)
19. Departmental/Program Service Load (committee/professional
memberships, chairs, administration; community & public service,
equipment maintenance, etc) (Source: IRO, Deans, Chairs)
20. Departmental/Program Student Services Load (recruiting,
advising, outreach, alumni support, etc.) (Source: IRO, Deans,
Chairs)
21. Departmental/Program Academic Contact Hours/ Faculty FTE
(Source: Contact hours formulation TBD)
KIG Topic Group 4: Quality
22. Departmental/Program Student Admissions Quality Points
(Source: IRO)
23. Departmental/Program Student Quality SU Jr./ Sr. Non-Major GPA
(Source: IRO)
24. Departmental/Program Alumni Quality Post-SU Activity
(Career progress, graduate/professional school, etc)
(Source: Career Services for job placement rates, Student
Clearinghouse through IRO for schools attended after Stetson)
25. Departmental/Program Faculty Scholarship Quality
(Scholarly production, awards, grants, etc.)
(Source: IRO, Deans, Chairs)
26. Departmental/Program Faculty Teaching Quality
(Course evaluations, peer reviews, etc.)
(Source: IRO, Deans, Chairs)
27. Departmental/Program Alumni Perception of Program Quality
(Senior surveys, alumni surveys) (Source: IRO)
28. Departmental/Program Learning Outcomes Assessments (Source:
TBD)
Committee Concerns and Possible
Shortcomings
The KIG recognizes that Key Indicators are essential in
determining the viability of our current and future
programs, and we were happy to be able to participate in
building a healthier institution. We hope that the KIs we
have presented will help to develop a University that is
both financially stable and academically excellent.
We also recognize that a system of indicators
summarizes the rich complexity of the academic
program at Stetson, and therefore much detail may be
lost in the process. As we developed our system of
indicators, we sought to keep in mind possible
exceptions and shortcomings in the system. We list
these here for the consideration of the academic leaders
who will employ our system in the academic planning
process.
Committee Concerns and Possible
Shortcomings
(continued)
• Is the School of Music a single program or many programs for the
purposes of this system?
• How will we define Discovery for the purposes of the system
recommended here?
• How will the movement to the course unit system affect our
measures, especially those employing three-year rolling averages?
• Will administrative units be able to provide the required data input in
a timely fashion and at the level of specificity required?
• Will measuring these indicators and not others affect our academic
program in the way U.S. News rankings have affected college and
university behavior and reporting?
Committee Concerns and Possible
Shortcomings
(continued)
• Will looking at unfunded scholarships as departmental
expenditures penalize departments with large numbers
of students with large unfunded merit aid packages?
• Can we measure program quality without looking at
student academic work in that program? Should we
include a portfolio of academic work as an indicator of
program quality?
• Finally, we recognize that indicators of net revenue and
student demand are the easiest to quantify and employ.
However, our recommended system of KIs integrates
faculty effort and academic quality measures into the
program evaluation process. While more difficult to
measure, the KIG believes that measures of faculty effort
and academic quality are essential components of the
program review process.
Questions, Suggestions, and
Comments