Transcript University and Higher Education Rankings – What Relevance

University and Higher Education Rankings –
What Relevance Do they Have?
EI Affiliates Conference in the OECD member countries
“FRAMING EDUCATION FOR THE PUBLIC GOOD”
29-30 January 2013, London, United Kingdom
Grahame McCulloch
General Secretary, NTEU (Australia)
EI Executive Board Member
BACKGROUND AND CONTEXT
• Rise of mass higher education in rich countries
(1960s-1990s) – mainly well funded public systems
(with notable Japanese and Korean exceptions) but
recognisable hierarchies and stratification (with a
premium on research intensity)
• Economic, social and labour market benefits
(including R&D, innovation, technology transfer and
human capital)
BACKGROUND AND CONTEXT
• Slowing of growth and scarcer resources – increased
managerial authority, expanded role for private effort
and markets, accountability, performance
measurement and indicators and erosion of tenure
and academic autonomy (1990s – present)
• Rapid growth in emerging and developing countries
(embryo of mass systems) and strong preoccupation
with science, technology, R&D and direct economic
role of universities (particularly in Asia)
BACKGROUND AND CONTEXT
• 177 million students, 17000 institutions and 11
million staff – expected to double in 20 years (mainly
in Asia and Latin America)
• Global trade and flows – 2.5 million international
students (around $100 billion), regional trade blocs
(US, Canada, UK, Australia and Europe) and
regional/national accreditation, qualifications and
quality assurance
BACKGROUND AND CONTEXT
• Imbalance between per capita resources and
enrolments on world scale
• Rich countries seeking larger share of world
expansion via trade, offshore and joint ventures and
research collaboration
• Emerging countries seeking domestic expansion
through national strategy and investment (including
imports of foreign capital, expertise and technical
systems)
Source: UNESCO 2010
Source: UNESCO 2010
2,000
0
4,000
6,000
8,000
10,000
12,000
14,000
13,524
12,500
12,252
12,189
10,616
9,679
9,613
9,144
8,810
7,496
7,312
7,212
6,483
6,455
6,431
6,372
6,253
6,102
5,509
4,818
4,644
4,535
4,500
3,309
2,751
1,501
18,000
16,000
Annual Public Expenditure per Student on Tertiary Education
Institutions
18,623
17,848
17,252
15,868
20,000
Public Expenditure per Pupil in Tertiary Education ($US PPP)
2005
$12,000
$10,000
$8,000
$6,000
$4,000
$2,000
$0
Philippines
Fiji
Thailand
Malaysia
New
Zealand
Australia
Japan
United Hong Kong
Kingdom
Source: NTEU Estimates derived from:
UNESCO (Public Expenditure per Tertiary Pupil as % of GDP per Capita) and Mundi Index GDP per Capita ($US PPP) (http://indexm undi.com)
United
States
Denmark
OVERVIEW OF GLOBAL MEASUREMENT
INSTRUMENTS
• Parallels between schools, vocational education and
higher education – Program for International Student
Assessment (PISA), Teaching and Learning Survey
(TALIS), Program for International Assessment of
Adult Competencies (PIAAC) and Assessment of
Higher Education Learning Outcomes (AHELO)
• Focussed on systems and/or disciplines (not
individual institutions) for cross-country comparisons
and use in national benchmarking and quality
assurance processes, high media and political
visibility
OVERVIEW OF GLOBAL MEASUREMENT
INSTRUMENTS
• Problems of measurement and misinterpretation or
misuse of data by national governments – causation,
correlation, limits of mathematical language and
dangers of simplistic international league tables and
single standardised scores
• Narrowing of domestic public policy standards with
strong instrumental focus, and less emphasis on
wider social and educational objectives.
OVERVIEW OF GLOBAL MEASUREMENT
INSTRUMENTS
• Tendency of scores and metrics to undermine
qualitative and organisational quality assurance
measures, and to encourage ‘gaming’ and
manipulation of metrics
UNIVERSITY RANKING SYSTEMS
• Siblings and first cousins of international and
national performance indicators/accountability
systems, but focussed on individual institutions and
not systems
• Typical weighted indicators include undergraduate
and postgraduate enrolments, research grants and
endowments, public and private funding,
student/staff ratios, graduation rates, research
citations and publications and prizes/awards
UNIVERSITY RANKING SYSTEMS
• Measurement of teaching and research quality uses
proxies (metrics and reputational surveys), and
league tables are based on standardisation,
aggregation into single score and ordinal scale based
on the top ranked institutions
• Developed and administered by media companies or
specialist arm of university research centres – no
direct government or intergovernmental involvement
UNIVERSITY RANKING SYSTEMS
• Multi-Ranking without league tables – University
Ranking and U-Map – the “Berlin Rankings” (CHE/die
Zeit, Germany and IREG) and U-Multirank (EU)
UNIVERSITY RANKING SYSTEMS
• National ranking league tables – Japan (Asahi
Shimbun), Canada (Macleans), Italy (La Repubblica),
US (US News and World Report)
• International ranking league tables – US News and
World Report (with QS Symonds), Times Higher
Education Supplement (with Thomson Reuters),
Academic Rank of World Universities (Shanghai Jiao
Tong University, China), Global Universities Rankings
(Lomonosov State University, Russia), Scientific
Papers for World Universities (Accreditation and
Evaluation Council, Taiwan), Leiden Research Ranking
(Leiden University, Netherlands), University Web
Ranking (CSIC Cybernetics, Spain)
MOST INFLUENTIAL – THES AND ARWU
• In rich countries used by governments in domestic
policy debate and by universities in marketing and
promotion, particularly in North and South East Asia
• In emerging and developing countries used by
governments as benchmark for development of
domestic institutions and systems
• Directly affects institutional behaviour and indirectly
high achieving student choice
MOST INFLUENTIAL – THES AND ARWU
• ARWU based solely on metrics with research
(maths and science in particular), accounting for
90% of composite scores
• THES apparently more balanced (30% teaching,
30% research volume, income and reputation,
32.5% research citations, 7.5% international and
2.5% economic innovation), but actually closer to
75% weighting for research
MOST INFLUENTIAL – THES AND ARWU
• Both rankings actually reflect the prestige, high
selectivity in student enrolments and staff
appointments, economic resources and global reach
of each university
• Are not able and do not aspire to reflect diversity of
institutions and systems (large and small, teaching
intensity, access and equality, three and four year
programs, cultural context)
• Not a guide or benchmark for national system
development
Proportion of universities covered by THES and ARWU rankings
Source: European Universities Association (EUA) 2011
Times Higher Education 2012
Academic Ranking of World Universities
1
2
California Institute of Technology
University of Oxford
USA
UK
1
2
2
Stanford University
USA
3
4
5
6
7
8
9
10
11
USA
USA
USA
UK
UK
USA
USA
USA
13
14
15
16
17
18
19
20
21
22
Harvard University
Massachusetts Institute of Technology
Princeton University
University of Cambridge
Imperial College London
University of California, Berkeley
University of Chicago
Yale University
ETH Zürich – Swiss Federal Institute of
Technology Zürich
University of California, Los Angeles
Columbia University
University of Pennsylvania
Johns Hopkins University
University College London
Cornell University
Northwestern University
University of Michigan
University of Toronto
Carnegie Mellon University
23
USA
USA
4
5
6
7
8
9
10
11
Harvard University
Stanford University
Massachusetts Institute of Technology
(MIT)
University of California, Berkeley
University of Cambridge
California Institute of Technology
Princeton University
Columbia University
University of Chicago
University of Oxford
Yale University
Switz
12
University of California, Los Angeles
USA
USA
USA
USA
USA
UK
USA
USA
USA
Can
USA
13
14
15
16
17
18
19
20
21
22
USA
USA
USA
USA
USA
USA
USA
Jap
UK
USA
Duke University
USA
23
24
University of Washington
USA
24
Cornell University
University of Pennsylvania
University of California, San Diego
University of Washington
The Johns Hopkins University
University of California, San Francisco
University of Wisconsin - Madison
The University of Tokyo
University College London
University of Michigan - Ann Arbor
Swiss Federal Institute of Technology
Zurich
The Imperial College of Science,
Technology and Medicine
25
University of Texas at Austin
USA
25
University of Illinois at Urbana-Champaign
USA
25
27
28
29
30
Georgia Institute of Technology
University of Tokyo
University of Melbourne
National University of Singapore
University of British Columbia
USA
Jap
Aust
Sing
Can
26
27
27
29
30
Kyoto University
New York University
University of Toronto
University of Minnesota, Twin Cities
Northwestern University
Jap
USA
Can
USA
USA
12
USA
USA
UK
USA
USA
USA
USA
UK
USA
Switz
UK
Time Higher Education Top 400 by Region
Region
No.
North America
133
%
33.3%
Europe
Asia
180
80
45.0%
20.0%
Africa
4
1.0%
Latin America
3
0.8%
ARWU Top 400 by Country/Region
Region / Country
USA
Canada
No.
139
19
%
34.8%
4.8%
UK
Europe
33
128
8.3%
32.0%
68
17.0%
Africa
5
1.3%
Latin America
8
2.0%
Asia
AN EI RESPONSE
• High quality information and feedback for national
and international students necessary in mass
systems, and robust quality assurance is essential
• Quality assurance and performance assessment
should reflect the characteristics, resources
social/educational objectives of each institution, and
be autonomously determined within each university
using peer review and stakeholder consultation
AN EI RESPONSE
• Academic freedom, collegial decision-making, trade
union rights and employment standards should be
part of quality assurance criteria
• The aggregation of data at national and international
level for any cross-institutional comparative purposes
should prevent the construction of league tables
• Building on EI’s strategic response to PISA, EI should
continue a critical dialogue with OECD in the
development and implementation of AHELO (noting
its discipline and national system focus). Any final
methodology should prevent the construction of
arbitrary league tables
AN EI RESPONSE
• EI should develop direct dialogue with the Berlin
rankings group (CHE/die Ziet and IREG) on the
development of University Ranking and U-Map, and
EU on U-Multirank (noting these are consciously
constructed to enable comparison without league
tables)
FURTHER READING
• Global university rankings: where to from here?, Simon Marginson, Centre
for the Study of Higher Education, University of Melbourne, Australia
• To Rank or To Be Ranked: The Impact of Global Rankings in Higher
Education, Simon Marginson and Marijk van der Wende, Journal of Studies
in International Education, Vol. 11 No 3/4
• College and University Ranking Systems - Global Perspectives and
American Challenges, Institute for Higher Education Policy, Washington
D.C., April 2007
• Global University Rankings and Their Impact, EUA Report on Rankings
2011, Andrejs Rauhvargers
• The Road to Academic Excellence – The Making of World-Class Research
Universities, Philip G. Altbach and Jamil Salmi Editors, The World Bank,
Washington D.C