Transcript President’s Advisory Group Meeting October 6, 2006

U S News & World Report
Undergraduate College Rankings
Ruth Kallio, Associate Director for Institutional Research
Office of Budget and Planning
November 14, 2007
8:30 – 10:30 a.m., Michigan Room
Office of Budget & Planning
0
USN&WR Undergraduate College
Rankings Background

What do they rank?

Undergraduate programs overall


Graduate and professional programs overall


Some specialized undergraduate programs
Specialized graduate & professional programmatic areas
When did they begin ranking programs?


1983 undergraduate
1987 graduate and professional
1
USN&WR Undergraduate College Rankings
Background

How many undergraduate colleges and
universities are ranked?

Institutions are grouped according to Carnegie
classifications and for some categories, geographic
region





262 national universities
266 liberal arts colleges
574 master’s universities in 4 geographic regions
320 baccalaureate colleges in 4 geographic regions
1,422 institutions in total
2
USN&WR Undergraduate College
Rankings Background

Who is U-M ranked against?

U-M is in the “national universities” category
which includes:



Doctoral/Research Universities -- Extensive
Doctoral Research Universities -- Intensive
USN&WR publishes two sets of national
universities rankings


All national universities (N=262)
 59 of which we commonly use as “peers”
All public national universities (N=163)
 33 of which we commonly use as “peers”
3
USN&WR Undergraduate College Rankings
Background

Where does U-M rank overall?
Prior to 1988: 7th or 8th
 Since 1988: consistently between 21st and 25th
-- Currently: 25th (in a tie with UCLA)


Where does U-M rank among public
universities?


Consistently 2nd or 3rd
Currently: 3rd (in a tie with UCLA and behind
California Berkeley and Virginia)
4
Ten-Year History of
UM-Ann Arbor Rankings
Year
Overall Score Rank
1998-99
87
25th (tied w/ UCLA)
1999-00
73
25th (tied w/ UCLA)
2000-01
78
25th (tied w/ UCLA & UNC-Chapel Hill)
2001-02
77
25th
2002-03
72
25th (tied w/ UCLA & Wake Forest)
2003-04
75
25th
2004-05
76
22nd (tied w/ Carnegie Mellon & Univ. of Virginia)
2005-06
75
25th (tied w/ UCLA)
2006-07
75
24th (tied w/ Univ. of Virginia)
2007-08
73
25th (tied w/ UCLA)
5
USN&WR Data Collection Procedures


Surveys of colleges and universities
 Data are supposed to conform to national
standards for formatting and definitions developed
by college guide publishers in conjunction with
representatives from the higher education
community (aka, “Common Data Set”)
 92.4% of the institutions surveyed in 2007
responded
 USN&WR will estimate data for any institution that
cannot or will not provide them with some or all of
the necessary data.
Survey of presidents, provosts, and deans of
admissions (51% response rate in 2007)
6
How does USN&WR arrive at the overall score
and ranking for each national university?

Three types of components
 Reputation
 Measures of Educational Success
 Resources
7
Breakdown of the Reputational Component
Weight in
Overall Score

Peer Assessment Ranking


U-M
Rank
25.0% 12th
Avg. academic reputation score
(survey of presidents, provosts, deans
of admission)
Student Selectivity



15.0% 23rd
Avg. SAT/ACT scores of enrollees
7.5%
% of enrollees in top 10% of high school class 6.0% 15th
% of applicants who are admitted
1.5% 50th
8
Breakdown of the Educational Success Component
Weight in
Overall Score

Graduation & Retention



25.0%
% of entering class still enrolled one year later 4.0%
% of entering class who graduated in 6 years
or less
16.0%
Graduation Rate Performance
(Difference between a school’s actual and
predicted 6-year graduation rate for an
entering class)
5.0%
U-M
Rank
26th
21st
29th
33rd
9
Breakdown of the Resources Component

Faculty Resources







Financial resources


Avg. faculty salary + benefits adjusted for
regional differences in cost of living
% of undergrad class sections fewer than
20 students
% of undergrad class sections 50 or more
students
Student/faculty ratio
% of faculty with highest degree in their fields
% of faculty who are full-time
Expenditures per student
Alumni Giving

% of undergraduate alumni who donated money
to their school
Weight in
Overall Score
U-M
Rank
20.0%
69th
7.0%
6.0%
69th
2.0%
1.0%
3.0%
1.0%
103rd
73rd
10.0%
29th
5.0%
83rd
83rd
10
U-M’s Strengths and Weaknesses
in the Rankings


U-M scores well on measures of
reputation and educational success
U-M’s scale causes it to rank less well
on per capita resource measures
11
USN&WR Supplementary Information


Lists of leading institutions based on
other information not included in the
rankings calculations
Quantitative measures



Racial diversity
Economic diversity (U-M ranked 6th)
International Students
12
USN&WR Supplementary Information

Qualitative information








Internships
Senior Capstone
First-Year Experience*
Undergraduate Research/Creative Projects*
Learning Communities*
Study Abroad
Service Learning*
Writing in the Disciplines
*U-M listed as a leading institution
13
What affects the rankings from one
year to the next?




A change in the USN&WR methodology
A real change in an institution’s data
A real change in the data for other
institutions
A change in how many institutions
USN&WR chooses to publish on a given
ranking
14
Issues/Concerns regarding
USN&WR rankings







Affect of rankings on students’ decisions
Subjectiveness of the reputational component
Subjectiveness of the assigned weight for each
component
Component items may not measure what they are
assumed to
Consistency and definition problems found in data
collection
Ability of institutions to manipulate their data
Rankings measures are biased in favor of private
institutions
15
Issues/Concerns regarding
USN&WR rankings


Rankings falsely imply a precision in the numbers and
a significant difference between one rank and the
next
Rankings do not capture some of the most important
components of a student’s college experience -especially learning outcomes
16
Alternatives to USN&WR Rankings

Undergraduate

Media sponsored rankings


Other


Media sponsored rankings


Business Week, Wall Street Journal, etc.
Other

NRC, Gourman Report
For more examples of other rankings see:


Gourman Report, Princeton Review
Graduate


Washington Monthly, Business Week, Kiplinger’s, Wall Street
Journal
http://www.library.uiuc.edu/edx/rankings.htm
Many of these share the same problems and
weaknesses of USN&WR rankings--if not more!
17
Movements Away From Rankings

Current protest by liberal arts colleges on the
value of USN&WR rankings


40 institutions have pledged not to fill out the
USN&WR reputational survey and not to use the
rankings in promotional efforts
The national “Accountability” movement

Voluntary System Accountability (VSA)


NASULGC, AASCU
Annapolis Group, NAICU, The Education
Conservancy
18
How OBP Uses USN&WR Rankings

In general not used to define our peers




“Official” peers (19 institutions)
Big Ten peers
AAU peers (61 institutions)
Publish but do not endorse the rankings
on our website

(http://sitemaker.umich.edu/obpinfo/home)
19
How OBP Uses USN&WR Rankings


The rankings provide a quick source of
selected comparative information on our
“peers”
Mainly use on an ad hoc basis


To identify other peers that may not be in our
traditional peer groups (e.g., Notre Dame for
undergraduate tuition comparisons)
To narrow down our official peers (e.g.,
institutions with the most top ranked graduate
programs in engineering and the sciences for the
Advanced Project)
20