Transcript Middle States Expectations with Respect to Institutional

Middle States Expectations with Respect
to Institutional Self-Study
Michael F. Middaugh
University of Delaware
Vice Chair
Middle States Commission on Higher education
Middle States Accreditation Standards
Expectations: Assessment & Planning
It is the Commission’s intent, through the self-study
process, to prompt institutions to reflect on those
assessment activities currently in place (both for
institutional effectiveness and student learning), to
consider how these assessment activities inform
institutional planning, and to determine how to improve
the effectiveness and integration of planning and
assessment.
MSCHE Accreditation Standards
Characteristics of Excellence
Institutional Mission
• Institutional Effectiveness
–
–
–
–
–
–
–
Planning
Resource Allocation
Leadership
Governance
Administration
Integrity
Institutional Assessment
• Educational Effectiveness
–
–
–
–
–
–
–
Admissions & Retention
Support Services
Faculty
Offerings
General Education
Related Education Activities
Assessment of Student Learning
Not All Standards are Created Equal
• There are 14 accreditation standards within Characteristics of
Excellence.
• Experience suggests that the quality of an institution’s
accreditation self study hinges on how it addresses the
standards related to assessing student learning outcomes
(Standard 14), assessing institutional effectiveness (Standard 7),
and implementing a vital and comprehensive strategic planning
process (Standard 2).
• Annual institutional follow-up activity typically centers on these
three standards.
MSCHE Linked Accreditation Standards:
Standard 14: Student Learning Outcomes
Assessment of student learning demonstrates that, at
graduation, or other appropriate points, the institution’s
students have knowledge, skills, and competencies
consistent with institutional and appropriate higher
education goals.
Selected Fundamental Elements for
MSCHE Standard 14
• Articulated expectations for student learning (at
institutional, degree/program, and course levels)
• Documented, organized, and sustained assessment
processes (that may include a formal assessment plan)
• Evidence that student learning assessment information is
shared and used to improve teaching and learning
• Documented use of student learning assessment
information as part of institutional assessment
MSCHE Linked Accreditation Standards:
Standard 7: Institutional Assessment
The institution has developed and implemented an
assessment process that evaluates its overall
effectiveness in achieving its mission and goals and its
compliance with accreditation standards.
Selected Fundamental Elements for
MSCHE Standard 7
• Documented, organized, and sustained assessment
processes to evaluate the total range of programs and
services, achievement of mission, and compliance with
accreditation standards
• Evidence that assessment results are shared and used in
institutional planning, resource allocation and renewal.
• Written institutional strategic plan(s) that reflect(s)
consideration of assessment results
MSCHE Linked Accreditation Standards:
Standard 2: Planning, Resource Allocation
and Institutional Renewal
An institution conducts ongoing planning and resource
allocation based on its mission and goals, develops
objectives to achieve them, and utilizes the results of its
assessment activities for institutional renewal.
Implementation and subsequent evaluation of the
success of the strategic plan and resource allocation
support the development and change necessary to
improve and to maintain quality.
Selected Fundamental Elements for
MSCHE Standard 2
• Clearly stated goals and objectives that reflect conclusions
drawn from assessments that are used for planning and
resource allocation at the institutional and unit levels
• Planning and improvement processes that are clearly
communicated, provide for constituent participation, and
incorporate the use of assessment results
• Assignment of responsibility for improvement and
assurance of accountability
Some Words of Wisdom in Approaching the Standards
“The nicest thing about not planning is that failure comes as a
complete surprise and is not preceded by a period of worry
and depression.”
John Preston, Boston College
“A common mistake that people make when trying to design
something completely foolproof is to underestimate the
ingenuity of complete fools.”
Douglas Adams, Hitchhikers
Guide to the Galaxy
Assessing Student Learning Outcomes
• I’ll provide only a brief overview, as there are others (Linda
Suskie, Trudy Banta, Jeff Seybert) who are far better versed
than I am.
• That said, understand that assessment of student learning
is at the core of demonstrating overall institutional
effectiveness.
• Assessment of student learning is a direct response to the
inadequacy of student grades for describing general
student learning outcomes.
There is no “one size fits all” approach to
assessment of learning across the disciplines
None of these should be applied to evaluation of individual student performance for
purposes of grading and completion/graduation status.
1. Standardized Tests
2.
3.
4.
5.
• General Education or Discipline Specific
• State, Regional, or National Licensure Exams
Locally Produced Tests/Items
• “Stand Alone” or Imbedded
Portfolios/Student Artifacts
• Collections of Students’ Work
• Can Be Time Consuming, Labor Intensive, and Expensive
Final Projects
• Demonstrate Mastery of Discipline and/or General Education
Capstone Experiences/Courses
• Entire Course, Portion of a Course, or a Related Experience
(Internship, Work Placement, etc.)
Assessing Institutional Effectiveness
• Is the institution making the most effective and efficient
use of its human and fiscal resources to support
teaching/learning activities within the context of its
institutional mission?
• Important to assess effectiveness with respect to students
(admissions, engagement, satisfaction, post-graduation
follow-up); faculty and staff productivity; financial
efficiency; facilities efficiency.
Students
• Entering Student Needs Assessment
• Admissions
- Monitoring Activity
- Admitted Student Questionnaire or Similar Instrument
• Student Engagement/Satisfaction
- Student Attrition Persistence
- NSSE/FSSE
- Student Satisfaction Research (ACT/Noel-Levitz)
• Alumni Research
A Typical Admissions Monitoring Report
Eastern Seaboard State University
Weekly Admissions Monitoring Report
Campus Summary: First-Time Freshman Applicants, Their SAT Scores and Predicted Grade Index, by Admission Status and by Residency Status for the Entering Classes in the
Fall of 2005 as of 09/15/05; Fall 2006 as of 09/18/06; and Fall 2007 as of 09/13/07.
All
Applicants
2005 2006 2007
Counts
- Resident
- Nonresident
- Total
2,340 2,332 2,088
18,984 19,209 20,133
21,324 21,541 22,221
Admission
Denied
2005 2006 2007
Admission
Offered
2005 2006 2007
Admission
Accepted
2005 2006 2007
109
7,506
7,615
148
5,871
6,019
172
5,838
6,010
1,940
7,295
9,235
1,362
2,078
3,440
1,255
2,201
3,456
1,174
2,348
3,522
1,877 2,147
8,101 8,489
9,978 10,636
SAT Verbal
- Resident
- Nonresident
- Total
557
572
571
562
579
577
569
584
582
456
541
540
444
535
532
445
536
533
567
609
600
576
615
607
584
620
614
559
592
579
564
600
587
575
604
594
SAT Math
- Resident
- Nonresident
- Total
563
596
592
567
598
595
574
605
602
454
561
560
446
551
549
452
554
551
547
636
623
582
635
625
589
643
634
566
619
598
570
620
602
580
627
611
Predicted Grade
Index
- Resident
- Nonresident
- Total
2.67
2.90
2.82
2.75
2.94
2.88
2.83
2.94
2.88
1.66
2.40
2.22
1.60
2.36
2.30
1.71
2.41
2.37
2.75
2.98
2.90
2.78
3.04
2.95
2.85
3.05
2.98
2.75
2.98
2.89
2.78
3.05
2.95
2.85
3.04
2.98
Ratio of Offers
to Total Applications
2005 2006 2007
0.83
0.38
0.43
0.80
0.42
0.46
1.03
0.42
0.48
Ratio of Accepts
to Offers (Yield)
2005 2006 2007
0.70
0.28
0.37
0.67
0.27
0.35
0.61
0.28
0.33
Drilling Down
• Why do some students to whom we extend an offer of
admission choose to attend our institution?
• Why do other students to whom we extend an offer of
admission choose to attend a different school?
• How is our institution perceived by prospective students
within the admissions marketplace?
• What sources of information do students draw upon in
shaping those perceptions?
• What is the role of financial aid in shaping the college
selection decision?
What About Non-Returning Student Research?
TABLE 1: ENROLLMENT, DROPOUT RATES AND GRADUATION RATES
FOR FIRST-TIME FRESHMEN ON THE NEWARK CAMPUS (Total)
Enrollment and Dropout Rates
Entering
Fall Term
Graduation Rates
1st
Fall
2nd
Fall
3rd
Fall
4th
Fall
5th
Fall
6th
Fall
1995 N
% enrollment
% dropout
3154
100.0%
0.0%
2673
84.7%
15.3%
2439
77.3%
22.7%
2355
75.3%
24.7%
599
19.0%
26.4%
1996 N
% enrollment
% dropout
3290
100.0%
0.0%
2804
85.2%
14.8%
2585
78.6%
21.4%
2489
76.3%
23.7%
1997 N
% enrollment
% dropout
3180
100.0%
0.0%
2766
87.0%
13.0%
2523
79.3%
20.7%
1998 N
% enrollment
% dropout
3545
100.0%
0.0%
3080
86.9%
13.1%
1999 N
% enrollment
% dropout
3513
100.0%
0.0%
2000 N
% enrollment
% dropout
within
3 yrs
within
4 yrs
within
5 yrs
Total
113
3.6%
26.1%
21
1721
0.7% 54.6%
2219
70.4%
2344
74.3%
606
18.4%
26.1%
108
3.3%
26.7%
22
1825
0.7% 55.5%
2302
70.0%
2427
73.8%
2436
77.5%
22.5%
581
18.3%
24.3%
117
3.7%
24.5%
27
1827
0.8% 57.5%
2284
71.8%
2401
75.5%
2830
79.8%
20.2%
2762
78.5%
21.5%
653
18.4%
22.9%
118
3.3%
22.7%
22
2079
0.6% 58.6%
2621
73.9%
2727
76.9%
3126
89.0%
11.0%
2871
81.7%
18.3%
2757
79.4%
20.6%
526
15.0%
22.6%
83
2.4%
22.7%
31
2193
0.9% 62.4%
2632
74.9%
2684
76.4%
3128
100.0%
0.0%
2738
87.5%
12.5%
2524
80.7%
19.3%
2453
79.2%
20.8%
496
15.9%
23.9%
85
2.7%
23.8%
24
1884
0.8% 60.2%
2297
73.4%
--
2001 N
% enrollment
% dropout
3358
100.0%
0.0%
2976
88.6%
11.4%
2746
81.8%
18.2%
2674
80.6%
19.4%
472
14.1%
22.3%
0
0.0%
0.0%
31
2138
0.9% 63.7%
--
--
2002 N
% enrollment
% dropout
3399
100.0%
0.0%
3055
89.9%
10.1%
2866
84.3%
15.7%
2787
83.2%
16.8%
0
0.0%
0.0%
0
0.0%
0.0%
42
1.2%
--
--
--
2003 N
% enrollment
% dropout
3433
100.0%
0.0%
3035
88.4%
11.6%
2808
81.8%
18.2%
0
0.0%
0.0%
0
0.0%
0.0%
0
0.0%
0.0%
--
--
--
--
2004 N
% enrollment
% dropout
3442
100.0%
0.0%
3064
89.0%
11.0%
0
0.0%
0.0%
0
0.0%
0.0%
0
0.0%
0.0%
0
0.0%
0.0%
--
--
--
--
What About Non-Returning Student Research?
Drilling Deeper…..
• Commercial instruments exist, but response rates tend to be low, and
reported reasons for leaving politically correct – personal or financial
reasons.
• For the last several years, we have administered the Survey of Student
Opinions during the Spring term to a robust sample of students across
freshman, sophomore, junior, and senior classes.
• The following Fall, the respondent pool is disaggregated into those
who took the Survey and returned in the Fall, and those who took the
Survey, did not return in the Fall, and did not graduate.
• Test for statistically significant differences in response patterns
between the two groups.
Faculty and Staff Productivity
• Budget Support Metrics
• Delaware Study of Instructional Costs and Productivity
• Academic and Administrative Program Reviews
• Strategic Ratio Indicators
• Satisfaction Studies
BUDGET SUPPORT DATA
College of Arts and Science
1996-97 Through 1998-99
Department X
A. TEACHING WORKLOAD DATA
FALL
FALL
FALL
SPRING
SPRING
SPRING
1996
1997
1998
1997
1998
1999
FTE MAJORS
Undergraduate
38
31
39
38
40
39
Graduate
0
0
0
0
0
0
Total
38
31
39
38
40
39
DEGREES GRANTED
Bachelor's
-----
-----
-----
20
19
19
Master's
-----
-----
-----
0
0
0
Doctorate
-----
-----
-----
0
0
0
TOTAL
-----
-----
-----
20
19
19
STUDENT CREDIT HOURS
Low er Division
6,246
5,472
5,448
4,518
6,156
5,478
Upper Division
726
638
869
1,159
951
966
Graduate
Total
% Credit Hours Taught by
183
153
129
195
276
135
7,155
6,263
6,446
5,872
7,383
6,579
77%
81%
77%
82%
91%
82%
23%
19%
23%
18%
9%
18%
98%
97%
98%
96%
98%
97%
Faculty on Appointm ent
% Credit Hours Taught by
Supplem ental Faculty
% Credit Hours Consum ed by Non-Majors
BUDGET SUPPORT DATA
College of Arts and Science
1996-97 Through 1998-99
Department X
A. TEACHING WORKLOAD DATA
FALL
FALL
FALL
SPRING
SPRING
SPRING
1996
1997
1998
1997
1998
1999
FTE STUDENTS TAUGHT
Low er Division
416
365
363
301
410
365
Upper Division
48
43
58
77
63
64
Graduate
20
17
14
22
31
15
Total
485
424
435
400
504
445
Departm ent Chair
1.0
1.0
1.0
1.0
1.0
1.0
Faculty on Appointm ent
15.0
16.0
15.0
15.0
15.0
15.0
Supplem ental Faculty
1.5
1.0
1.3
1.0
0.8
1.5
Total
17.5
18.0
17.3
17.0
16.8
17.5
Student Credit Hrs./FTE Faculty
408.9
347.9
373.7
345.4
440.8
375.9
FTE Students Taught/FTE Faculty
27.7
23.6
25.2
23.5
30.1
25.4
FTE FACULTY
WORKLOAD RATIOS
BUDGET SUPPORT DATA
College of Arts and Science
1996-97 Through 1998-99
Department X
B. FISCAL DATA
FY 1996
FY 1997
FY 1998
($)
($)
($)
RESEARCH AND SERVICE
Research Expenditures
0
5,151
499
Public Service Expenditures
0
0
0
Total Sponsored Research/Service
0
5,151
499
Sponsored Funds/FTE Fac. On Appointment
0
312
31
1,068,946
1,141,927
1,144,585
Direct Expense/Student Credit Hour
81
84
88
Direct Expense/FTE Student Taught
1,198
1,229
1,301
3,960,208
4,366,720
4,311,275
3.73
3.82
3.77
COST OF INSTRUCTION
Direct Instructional Expenditures
REVENUE MEASURES
Earned Income from Instruction
Earned Income/Direct Instructional Expense
Science Department
Undergraduate S tudent Credit Hours
Taught per FTE T/TT Faculty
Total S tudent Credit Hours Taught per FTE
T/TT Faculty
300
250
200
UD
150
Nat'l
100
50
0
1994
1996
1997
1998
350
300
250
200
150
100
50
0
Total Class S ections Taught per
FTE T/TT Faculty
Nat 'l
1996
1997
1998
1996
1997
1998
1999
Total S tudent Credit Hours Taught per FTE
Faculty (All Categories)
UD
1994
Nat'l
1994
1999
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
UD
1999
260
250
240
230
220
210
200
190
180
UD
Nat 'l
1994
Direct Instructional Expenditures per
S tudent Credit Hour
1996
1997
1998
1999
S eparately Budgeted Research and S ervice
Expenditures per FTE T/TT Faculty
$250
$120,000
$200
$100,000
$150
UD
$100
Nat 'l
$80,000
UD
$60,000
Nat 'l
$40,000
$50
$20,000
$0
$0
1994
1996
1997
1998
1999
FY95
FY97
FY98
FY99
FY00
Financial and Facilities Indicators
• Financial Ratio Analysis – Trend Data
• Facilities
- Space Utilization Studies
- Deferred Maintenance
Strategic Planning: Closing the Loop
• Most institutions in the Middle States Region do a
reasonable job of assessing student learning and
institutional effectiveness.
• Where trouble most frequently occurs is in providing
demonstrable evidence that assessment information is
actually being used to inform strategic planning, decisionmaking, and resource allocation at the institutoion.
From the University of Delaware Mission Statement:
“The University affirms its historic mission of providing
the highest quality education for its undergraduate
students, while maintaining excellence in selected
graduate programs…. The University will continue to
attract and retain the most academically talented and
diverse undergraduate students, and support their
intellectual, cultural, and ethical development as
citizens and scholars.”
The University went on public record in 1991:
• Average total compensation for faculty at each academic rank
would be at or above the median within five years for the 24
Category I Doctoral Universities identified as salary peers.
• Total undergraduate financial aid from all sources would increase
by 100 percent within five years.
• Student satisfaction with programs and services at the University,
as measured through the ACT Student Opinion Survey would
demonstrate significant gains within five years.
• The University would commit itself to a policy of annually setting
aside at least 2 percent of the replacement value of the physical
plant, to be used for facilities renovation and rehabilitation.
These priorities were not a “wish list.” They grew out of a careful
examination of empirical data provided by the University’s Office
of Institutional Research and Planning and other data sources.
Consider the following:
• When compared with the 24 Category Doctoral I universities in
the states contiguous to Delaware, and the District of Columbia,
in 1991 the average salary for all three major faculty ranks at the
University of Delaware ranked near the bottom of the list.
• The Student College Selection Survey indicated that students
were receiving offers of more aid from admissions competitors,
and that the aid packages had more grants and fewer loans than
University aid packages. Not surprising, the University was at a
competitive disadvantage for academically talented students.
• University scores on the ACT Student Opinion Survey suggested
that the institution had considerable room for improvement with
respect to student satisfaction with programs and services, and
with a number of areas in student life.
• The University was looking at in excess of $200 million in
deferred maintenance to its buildings and grounds.
Results - Salaries
Total Compensation, 1989-90 Compared with 1999-2000
FULL PROFESSOR
Institution
1989-90
1999-2000
Princeton University
91,800
University of Pennsylvania
151,900
University of Pennsylvania
91,500
Princeton University
134,100
Georgetown University
88,700
Georgetown University
126,100
Carnegie-Mellon University
85,800
Carnegie-Mellon University
123,200
University of Virginia
85,100
University of Virginia
122,900
George Mason University
84,400
UNIVERSITY OF DELAWARE
116,200
Johns Hopkins University
83,800
Temple University
114,300
New Jersey Institute of Technology
83,100
George Mason University
113,900
Rutgers University - New Brunswick
82,000
College of William and Mary
113,400
Lehigh University
80,800
Lehigh University
111,300
University of Maryland - College Park
80,500
American University
111,100
Virginia Polytechnic Inst. & State University
76,700
Johns Hopkins University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Maryland - Baltimore County
76,700
George Washington University
110,800
------------------------------------------------------------------------------------------------------------------------------------------------------University of Pittsburgh - Main Campus
76,500
New Jersey Institute of Technology
109,200
George Washington University
76,500
Rutgers State University-N.Brswck
109,000
American University
76,300
Pennsylvania State University
108,600
Pennsylvania State University
75,300
University of Maryland - College Park
106,800
UNIVERSITY OF DELAWARE
74,700
University of Pittsburgh - Main Campus
105,600
College of William and Mary
74,400
Virginia Polytechnic Inst. & State University
103,800
Virginia Commonwealth University
72,600
Drexel University
103,700
Drexel University
71,000
Virginia Commonwealth University
103,300
Old Dominion University
70,400
University of Maryland - Baltimore County
96,200
Temple University
68,400
Howard University
93,300
Catholic University of America
64,800
Catholic University of America
Howard University
64,000
Old Dominion University
86,800
No Report
Results – Financial Aid
Growth In Undergraduate Financial Aid at the
University of Delaware, FY 1990 Compared with
FY 2000
FY 1990
FY 2000
% Increase
University Administered Funds
$4,458,640
$28,036,660
528.8
State Grant Funds
$3,869,000
$6,643,500
71.7
Other Fund Sources
$2,169,602
$4,394,180
102.5
$10,497,242
$39,074,340
272.2
TOTAL
Results – Student Satisfaction
Comparison of 1995 University of Delaware Scores on ACT
Student Opinion Survey with 1990 Scores and with National
Norms
1990 Scores
Compared with
National Norms
1995 Scores
Compared With
1990 Scores
1995 Scores
Compared with
National Norms
University is Ahead
21
44
39
University is Tied
14
11
15
University is Behind
27
7
8
Results - Facilities
• By 2000, the University had renovated every classroom in its
entire building inventory, retrofitting most with state-of-the-art
teaching technology.
• An aggressive program of fundraising enabled not only the
aforementioned renovation and rehabilitation, but also the
construction of several new classroom and student services
buildings.
• The University is now on a cycle of planned maintenance, as
opposed to deferred maintenance.
Results – From an Accreditation Perspective
“ The
University of Delaware has every reason to take enormous pride in
what it has accomplished over the past 10 years. A decade ago, it was
coming out of a period of considerable turmoil. Today, the University is
seen as a national model for the integration of information technology in
every aspect of university life: teaching and learning, research and
service, academic support, and campus administration. It has created
a physical plant that has few, if any, peers among public universities
and would be the envy of most private colleges. These substantial
achievements could not have happened without extraordinary
leadership from the senior administration.”
“ Better than almost any university we are familiar with, Delaware has a
clear sense of what it wants to be, namely, a university that offers high
quality undergraduate education with targeted areas of excellence in
graduate education and research.”
" The review team was enormously impressed by the high level of morale
that pervades the faculty, staff, and students. Almost without exception,
the people we spoke to take great pride in being part of the University.”
Middle States Evaluation Team, 2001
End Result
• Institutions must plan effectively in order to be effective.
• Where that is the case, the accreditation process is an
affirmation of the evidence of that effectiveness.