Transcript Growing Research in New Universities

The Global Obsession with Rankings:
How should Ireland respond?
Ellen Hazelkorn
Director of Research and Enterprise, and Dean of the
Graduate Research School
Higher Education Policy Research Unit (HEPRU)
Dublin Institute of Technology
NUI Centennial Conference, Dublin Castle
2-3 December 2008
‘What do we need to achieve by 2013? Two universities ranked
in the top 20 worldwide’ (Cronin, 2006).
‘This is the opportunity for more of our universities to emerge
as world-class institutions. More of our universities should aim
to be within the top 100 internationally and I would like some
of our universities to aspire to the top 10’ (Bishop, 2007).
‘This strategic plan…reflects our unswerving commitment….to
transform [xxx] University, within the next 10 years, into a
world-class institution that will be ranked among the top 30
leading universities in the world.’
‘To be number two – that would be good – and to be among
the first ten universities in Germany is also a goal. We are ten
or eleven so it differs between the different rankings so that’s a
point. So we might reach number five or six, would be
possible.’
Themes
1. Globalisation and the Rise of Rankings
2. How Rankings Impact on Higher Education
3. Lessons for Ireland
4. Conclusions
1. Globalisation and the Rise of Rankings
The Policy Context (1)

Globalisation and Knowledge Society




Knowledge recognized as foundation of economic growth, social
development, and national competitiveness,
Emphasis on human capital formation and knowledge production,
dissemination and transmission,
HE now an issue of geo-political dimensions.
‘Battle for Brainpower’
‘Scramble for students’
(Matsumoto and Ono, 2008, p1) or ‘Skilled Migration’ (OECD, 2008)
(Economist, 2006),

Greying society and shortage of PhD/researchers,

Competition between HEIs for students, faculty, researchers,

Internationalisation of higher education.
The Policy Context (2)

‘New Public Management’





Shift from HE as part of social to productive economy, and
market steering mechanisms.
Competitive positioning of HE and HEIs.
Emphasis on value for money, efficiency and investor confidence
 metrics,
HEIs asked why they exist – no longer an end but a
means.
Student is savvy participant/consumer/customer as link
between HE and career/salary grows



Education as public or private good.
‘Consumer’ information for students, parents and other key
stakeholders.
Increasing desire for comparative or benchmarking data.
Rankings and the K-economy
If HE is the engine of the economy, then productivity, quality
and status of HE/HE research is vital indicator;

Global competition reflected in the rising significance and
popularity of rankings

Attempt to measure knowledge-producing and talent-catching
capacity of HEIs,

Appear to (re)order global knowledge by giving weight and
prominence to particular disciplines/fields of investigation,

Provide a framework or lens through which the global economy and
national (and supra-national) positioning can be understood by giving
a ‘plausible’ explanation of world excellence,

Measure national competitiveness as expressed by number of HEIs
in top 20, 50 or 100…

There is a gap between national/supra-national ambitions and
global performance.

Be careful what you wish for…


Rankings part of US academic system for 100 yrs, but
today increasing popularity worldwide.
But, policy role, autonomy and funding comes with a price:




Greater accountability, efficiency and value-for-money,
Reform of curriculum, organisation and governance
model,
Emphasis on academic output which is measurable and
comparable,
Quality assurance, assessment and evaluation
mechanisms.
Trends
College guides: fulfil public service role, helping and
informing domestic undergraduate students and their
parents.
Evaluation and assessment of research, and teaching &
learning or whole institutions for QA and accreditation.
Benchmarking: used to manage more strategically,
effectively and efficiently as systematic comparison of
practice and performance with peer institutions.
National rankings
 Modernisation of HE management, strategic planning and
accountability/public disclosure,
 45+ countries have a national ranking system.
Global rankings next logical step. The rising significance
and popularity of worldwide comparisons.
Obsession With Rankings

Satisfy a ‘public demand for transparency and information
that institutions and government have not been able to meet
on their own.’

(Usher & Savino, 2006, p38)
Cue to students/consumers re: monetary ‘private benefits’ of
university attainment and occupational/salary premium,

Cue to employers what they can expect from graduates,

Cue to government/policymakers re: quality, international
standards & economic credibility,

Cue to public because they are perceived as independent of the
sector or individual universities,

Cue to HEIs because they want to be able to benchmark their
performance.
Audience/User Beyond Likely Suspects

Undergraduate, domestic students

Parents

Internationally mobile students and faculty

Postgraduate students

Academic partners and academic organisations

Government/Policymakers

Employers

Sponsors and private investors

Industrial partners

The public and public opinion

Ranking agencies/organisations
Difficulties with League Tables




Technical and Methodological Difficulties

Indicators as proxies for quality?

Quality and appropriateness of the metrics
Usefulness of the results as ‘consumer’ information

Rater bias? Halo effect? Reputational ranking?

Quality and appropriateness of the information
Comparability of complex institutions

One-size-fits-all? Diversity of missions, complex organisations

Matthew effect?
Influence on higher education, policy and public opinion?

Distorting academic values or Providing transparent
information?

Setting strategic goals or encouraging HEIs to become what is
measured?
2. How Rankings Impact on Higher
Education
‘They did not tell me frankly but I could read their minds that if I
am lucky enough to graduate at this university I could not be as
highly appreciated as the one who graduated from Columbia
University.
We are ‘unlikely to consider research partnerships with a lower
ranked university unless the person or team was exceptional.’
‘I think the university needs to calm down. We’ve had two career
panic days; it’s what I call them where they’re like Communist
training sessions where everyone has to stand up and say what
they are doing to improve their career.’
… those who are looking at their institution on an international
scale are fully aware of the potential of these ratings, rankings,
evaluations to attract students, to attract faculty and so on and it
is also commented in…the newspapers, in comments in the
media and so on ….
Ranking Status
Despite methodological concerns, HEIs taking rankings very
seriously...

58% respondents unhappy with current rank;

93% and 82% respondents, respectively, want to improve
their national or international ranking.

70% of all respondents wish to be in top 10% nationally, and
71% want to be in top 25% internationally.
Despite context, mission, age or size – all HE drawn into global
marketplace.
Impact on Students (1)

Domestic undergraduate: rely on local intelligence, national
rankings and entry scores BUT mobility on the rise;

Domestic postgraduate: becoming internationally mobile and
ranking sensitive;

International undergraduate: influenced by institutional
partnerships & familial links – some rankings sensitivity;

International postgraduate: Highly receptive to global rankings

Rankings = short-listing mechanism

Rankings influence employment opportunities.
Impact on Students (2)

40% US students use newsmagazine rankings, and 11% said
rankings were important factor in choice (Mcdonagh et al 1997,
1998).

61% UK students referred to rankings before making choice,
and 70% considered they were important/very important
(Roberts, 2007, 20) .


92% int’l students considered UK rankings important/very
important to inform choice (Roberts, 2007, 5, 18-20).
60% prospective German students ‘know rankings and use
rankings as one source of information among others’ (Federkeil,
2007).

Applicant behaviour conditioned by rankings
(Ehrenberg, 2004, 26) .
Impact on Social Selectivity

Above-average students make choices based non-financial
factors, e.g. reputation

(Spies, 1973, 1978).
Full-pay students likely to attend higher ranked college
(even by a few places) but grant-aided students less
responsive.

US Universities increasing recruitment of high SAT scorers
to influence student/selectivity metric.

In binary systems, evidence suggests students migrating
out of ‘lower status’ institutions.
Impact on Employers
Employers have implicit rankings based on own experience
which is self-perpetuating



UK study shows employers favour graduates from more highly
ranked HEIs


‘Systematic’ approach by large/int’l businesses rather than
SME.
25% of graduate recruiters interviewed ‘cited league tables
as their main source of information about quality and
standards’ (University of Sussex, 2006, 87, 80, also 87-92).
Boeing to Rank Colleges by Measuring Graduates' Job Success

To show which colleges have produced workers it considers most
valuable because it wants ‘more than just subjective information’
and ‘facts and data’ (Chronicle of HE, 19 September 2008).
Impact on Academic/Industry Partners

Academic Partnerships:





40% respondents said rankings integral to decision-making about
international collaboration, academic programmes, research or
student exchanges
57% thought rankings influencing willingness of other HEIs to
partner with them.
34% respondents said rankings influencing willingness of other
HEIs to support their institution’s membership of academic or
professional organisations.
Almost all universities chosen for Deutsche Telekom
professorial chairs used rankings as evidence of research
performance (Spiewak, 2005) .
Boeing will use performance data to influence ‘choice of
partners for academic research and...decisions about which
colleges it will ask to share in the $100-million’ Boeing spends
course work and supplemental training for employees. (Chronicle
of HE, 19 September 2008).
Impact on Government
French, German and Russian governments introduced
initiatives to boost performance in rankings:


French Senate Debate, Conference and Declaration

German Excellence Initiative
Malaysian government established Royal Commission of
Inquiry to investigate why rankings of two top universities fell
by almost 100 places within a year (Salmi & Saroyan, 2007, 40) .

Governments use rankings as an indicator of ‘value-for-money’
w/ ref to scholarship for int’l study (Clarke, 2007, 43; Salmi & Saroyan

2007, 52).
Macedonia Law on HE (2008) automatically recognises top
500 Times QS, SJT or USN&WR.

Dutch immigration law (2008) targets ‘foreigners that are
relatively young and received their Bachelor, Master or PhD
degree...from a university...in the top 150’ of SJT/Times QS.

Impact on Faculty and Academic Work

Increased emphasis on academic performance/outputs

Contracts tied to metrics/performance,

New salary and tenure arrangements,

Active head-hunting of high-achievers.

Rankings used to identify under-performers.

Impact on Staff Morale.

Faculty not innocent victims:

Rankings confer social and professional capital on faculty in high-
ranked HEIs,

‘Research power’ in deregulated global division of academic
labour.
How are Institutions Responding?
63% HE leaders have taken strategic, organisational,
managerial or academic actions in response to the results.
Of those,

Overwhelming majority took either strategic or academic
decisions and actions.

Only 8% respondents indicated they had taken no action.
Mapping Institutional Actions
Specific Actions
Weightings
Research
• Relatively develop/promote bio-sciences rather than arts, humanities & social
sciences
• Allocate additional faculty to internationally ranked departments
• Reward publications in highly-cited journals
• Publish in English-language journals
• Set individual targets for faculty and departments
SJT = 40%
Times = 20%
Organisation
• Merge with another institution, or bring together discipline-complementary
departments
• Incorporate autonomous institutes into host HEI
• Establish Centres-of-Excellence & Graduate Schools
• Develop/expand English-language facilities, international student facilities,
laboratories
SJT = 40%
Times = 20%
Curriculum
•
•
•
•
•
Students
• Target high-achieving students, esp. PhD
• Offer attractive merit scholarships and other benefits
Faculty
•
•
•
•
•
Academic
Services
• Professionalise Admissions, Marketing and Public Relations
• Ensure common brand used on all publications
• Advertise in high-focus journals, e.g. Science and Nature
Harmonise with EU/US models
Discontinue programmes/activities which negatively affect performance
Grow postgraduate activity in preference to undergraduate
Favour science disciplines
Positively affect student/staff ratio (SSR)
Head-hunt international high-achieving/HiCi scholars
Create new contract/tenure arrangements
Set market-based or performance/merit based salaries
Reward high-achievers
Identify weak performers
SJT = 10%
Times = 20%
Times = 15%
SJT = 40%
Times = 25%
Times = 40%
To summarise...
1. Audience/User goes beyond the usual suspects,
2. High achievers – students and faculty – are particularly sensitive to
rankings,
3. Rankings influence decision-making, and incentivize behaviour with
positive and perverse effects,
4. HE are focusing resources on fields and activities that will positively
affect position, status and reputation.
3. Lessons for Ireland
Legacy of Rankings
Rankings = metaphor for competition and driver of HE reform


Using rankings to inform policy and restructure HE system

As a ‘market mechanism’ to drive difference,

To concentrate resources in ‘Centres of Excellence’.
Linking indicators to resource allocation and accreditation
 Shift from input  outcome/output  impact,
 Will intensify as economies/financial situation tightens.

Cross-national comparisons as indicator of HE
performance.
Indicator of Global Competitiveness?
Top 100
Times QS
SJT Ranking
2007
2008
2007
2008
US
37
37
53
54
Europe
35
36
35
34
9
8
2
3
13
14
6
5
Canada
6
5
4
4
Latin America/Africa
0
0
0
0
Switzerland
1
3
3
3
19
17
11
11
France
2
2
4
3
Germany
3
3
5
6
Japan
4
4
5
4
China (incl. HK)
5
5
0
0
Ireland
1
1
0
0
Sweden
1
2
4
4
Russia
0
0
1
1
Australia/New Zealand
Asia Pacific (incl. Israel)
UK
Another Way to View Competitiveness?
Wealth of U.S. Universities, 2007
Endowment $b
Gifts Raised
$m
SJT
Rank
Times
QS
Rank
Harvard
34.9
614
1
1
Yale
22.5
304
11
2=
Stanford
17.2
911*
3
19
Princeton
15.8
254
8
6
MIT
10.0
333
5
10
Columbia
7.2
913
7
11
U-Penn
6.6
450
15
14
Cornell
5.4
406
12
20=
Dartmouth
3.8
159
101-152
71=
Brown
2.8
126
86
32
Can we afford that ‘reputation race’?
Rankings inflate academic ‘arms race’ locking institutions and
governments into continual ‘quest for ever increasing resources’
(Ehrenberg 2004).

‘World-class University’ estimated to cost min. $1b-$1.5b-a-
year operation + $500m for medical school
(Usher 2006; Sadlak & Liu
2007).

This would require min. 600% increase for the largest Irish
HEI and diverting the entire HE budget to a single institution.
Policy Trends
2 main policy regimes
1. Create greater vertical (reputational) differentiation [neo-liberal
model] (e.g. German, Japan, France):


‘Excellence initiatives’ to concentrate research in 10/30 world-class
universities;
‘To compete globally, the government will close down some
regional and private universities and direct money to the major
universities’
2. Create greater horizontal (mission) differentiation [social-
democratic] (e.g. Australia, Norway):


‘Create diverse set of high performing, globally-focused HEIs’
‘Move towards self-declaration of mission, setting own metrics and
a corresponding funding model’

Link ‘compacts’ to mission and performance
Responding to Global Rankings
International/Europe

OECD AHELO Project,

EU Expert Group: Assessment of University-Based Research,


Declaration on Ranking of European Higher Education
Institutions,
EU Tender for a European Ranking of HE.
Ireland

National Research Platform & National Research Data Project,

IOTI: developing standardised for data reporting,

HERG – SSTI Indicators Project,

Foresight AHSS – Metrics,

Increasing attention to evaluation of research outcomes.
Global Networks

HE judged increasingly in both national and global context.

National boundaries declining in significance –

‘National pre-eminence is no longer enough’,




Worldwide comparisons more significant in the future,
Implications even for ‘elite’ HEIs which may have been
dominant within national boundaries,
Development of ‘single world market’
(Marginson, 2006) .
Growing importance of global HE networks – lessons/benefits of
research teams translated to regions,

Lisbon Agreement/EHEA and ERA,

ASEAN common higher education space
http://blog.beerkens.info/index.php/2008/11/creating-a-higher-education-commonspace-in-southeast-asia/.
Diversity of Missions
Trend of simple to complex knowledge and shift from Mode 1
to Mode 2 corresponds with blurring boundaries between
vocational and classical HE.

Top-down regulation defining mission difference no longer
capable of meeting geo-political competitive demands for RDI.

Renewed attention on valuing diversity and cohesiveness of HE
‘system’.

Greater horizontal differentiation according to mission, e.g.
civic, technological, classical, specialist:


Research intensity replaced by research/field specialisation,

Basic vs. applied replaced with applied and almost applied
2008, p9).
(LERU,
Global Cities and Higher Education


As the distribution of economic activity has gone global, cities
now compete on global terrain (Florida, 2008)
Successful cities/mega-regions:



Depend on specialised clusters of HE and research
institutes that interact with creative enterprise, exchanging
ideas and personnel (OECD, 2006),
Have HEIs that either already have, or are growing, an
international reputation and have close relationships with
businesses based on the particular specialism of the
institution (Hutton, 2006),
Because most OECD countries face talent shortages,
successful cities attract international students and
researchers (OECD, 2006, p122).
Building a World Class System

Diverse and coherent portfolio of vertically differentiated high
performing, globally-focused institutions and student
experiences:

Research base for creation of knowledge to fuel innovation
and forge/attract international links.

‘balanced’ disciplinary or comprehensive approach
or



‘focused’ disciplinary approach of developing world-class
expertise in targeted areas.
Scale and quality of graduates to provide for desired societal
and economic outcomes.
Maximising capability beyond individual capacity.
Cf. Strategies in Norway, Australia, Catalonia
Ranking World Class Systems (1)
Rank
Country
Score
1.
United States
100
2.
United Kingdom
98
3.
Australia
94
4.
Germany
92
5.
Canada
92
6.
Japan
90
7.
France
89
8.
Netherlands
86
9.
South Korea
79
10.
Sweden
79
11.
Switzerland
79
12.
Italy
77
13.
Belgium
77
14.
New Zealand
76
15.
China
75
16.
Hong Kong
72
17.
Ireland
71
18.
Finland
70
30.
South Africa
54
40.
Turkey
35




System: No. HEIs ranked
500 or higher ÷ average
position.
Access: Total FTE at top 500
HEIs ÷ population size.
Flagship: normalized score
based on performance of
leading university.
Economic: performance
relative to investment.
QS SAFE - National System Strength
Rankings
Ranking World Class Systems (2)
Overall
Rank
Country
Overall Score


Inclusiveness – participation rates
Access – Threshold of skill
aptitude required for HE graduation.
1
Australia
30.6
2
UK
31.1
3
Denmark
39.1
4
Finland
40.8
5
USA
49.0
6
Sweden
49.2
7
Ireland
49.2
8
Portugal
54.3
9
Italy
60.9
10
France
62.2
Age range – Lifelong learning
capacity as % 30-39 year olds
enrolled.
11
Poland
64.4

12
Hungary
64.5
13
Netherlands
69.6
14
Switzerland
70.3
15
Germany
72.5
16
Austria
76.4
17
Spain
79.4
Effectiveness – Value of HE to
labour market as per wage premia.

Attractiveness – Ability to attract
international students.


Responsiveness – ability of system
to reform and change – measured
by speed/effectiveness Bologna
Declaration.
University Systems Ranking. Citizens and
Society in the Age of Knowledge. Lisbon
Council, 2008.
Strategic Considerations for Small Nations
Small nations face particular difficulties seeking to build world
class universities without sacrificing other policy objectives – the
gap is very wide;

Performances of HE in small nations is uniformly strong
throughout top 500 suggesting research investment evenly
rewarded across the sector (Sheil, 2007);

A World Class HE System can be developed adapting/learning
from:


Strategies of successful mega-regions (e.g. Florida, Sassen),
Innovation clusters (e.g. Porter, Nelson, Lundvall, Etzkowitz and
Leydesdorff),


Mode 2 research networks (e.g. Gibbons, Nowotny et al),

Biodiversity (Rosen, Wilson).
4. Conclusion
Positive and Perverse Effects

Creating sense of urgency and accelerating modernisation
agenda;

Driving up institutional performance and providing some
public accountability and transparency;

Distorting the focus of HE away from research-informed
teaching towards research, in the narrowest sense;

Reshaping HE by aligning national and institutional priorities –
education and research – to indicators;

Challenging government, HEIs and the public to (re)think HE,
and how and what should be measured.
Urban Myths

Rankings provide useful comparative information about the
performance of different HEIs facilitating student choice and
benchmarking;

Indicators are ‘plausible’/meaningful measurements of research
and knowledge creation;

High ranked HEIs are better than lower ranked/not ranked
institutions;

Concentrating research in a few elite institutions or scientific
disciplines will ‘lift all boats’.
Conclusion (1)

Rankings have gained popularity because they (appear to)
gauge world class status, provide accountability and measure
national competitiveness;

But even in relation to scientific research, rankings do great
damage to the research enterprise - inducing HE and
governments to adopt simplistic solutions and skew research
agendas and policies to become what is measured.
Conclusion (2)

Because cross-national comparisons are ‘here to stay’, the
choice of metrics (and weightings) are critical.

The challenge is to:
1. Ensure Ireland’s research landscape can participate appropriately
across the spectrum of world science;
2. Develop depth and excellence wherever it occurs;
3. Mobilise and amplify the potential of the whole HE system and its
benefits to society at large.
[email protected]
http://www.oecd.org/edu/imhe/rankings