Transcript 大學評鑑的作為

AC 21 International Forum
Competition and Cooperation among Universities
in the age of Internationalization
An Analysis of Positions Mobility of Global Rankings:
The Effective Use of Global Rankings in Making
Institutional Strategic Plans and Positioning for Building
World Class Universities
Dr. Angela Yung-chi Hou
Dean of Office of Research & Development,
Higher Education Evaluation and Accreditation Council of Taiwan
Director of Faculty Development & Instructional Resources Center ,
Fu Jen Catholic University
18-21 OCT , 2010
Shanghai
1
Introduction
• Globalization in the 21st century presents
universities and states with a number of
challenges and opportunities.
• No matter whether countries are developed
or developing ones, they are immensely
eager to build at least one world class
university, but they don’t know exactly
what they look like.
2
What does a world class university look like ?
• In terminology
– world class universities are top universities striving for
“Excellence”, in other words, it means “its quality must surpass
the expectation of their various stakeholders”
• Philip Altbach
– excellence in research, top professors, academic freedom and an
atmosphere of intellectual excitement, governance, adequate
facilities and funding.
• Jamil Salmi (World Bank) based on two rankings
(Shanghai and QS)
– a high concentration of talent (faculty and students)
– abundant resources to offer a rich learning environment and
conduct advanced research
– favorable governance (features that encourage strategic
vision, innovation and flexibility, and enable institutions to
make decisions and manage resources without being
encumbered by bureaucracy)
3
Relevance between global rankings and
World Class University
• the characteristics of world class universities are inevitably
deemed to be strongly correlated to most indicators used by
global rankings.
• Many nations tend to use global rankings as a basis of
building world class universities despite their well
documented methodological flaws.
• Many top administrators at leading universities are learning
to use global rankings wisely in order to achieve the
institutional short term and long term strategic plans, not just
to boycott them.
– Minnesota’s initiative to become one of the top three
research institutions in the world
– Taiwan National University announced the initiative of
“Moving into the top 100” at its 80th anniversary
– Baylor University put the vision on making the institution
one of the U.S. News Top 50 by 2012.
4
Characteristics of 4 Major Global rankings and
their methodological limitations
ARWU
QS (THE)*
Webmetrics
HEEACT
Established year
2003
2004
2004
2007
Institution
Academic
institution
Mass
media/Private
Education
consulting firm
Governmental
research unit
QA Agency
Goal
Academic
competition
Profit making
Academic sharing
Benchmarking
Number of
indicators
6
6
4
8
Indicator
category
Research
output/
learning input
Research output /
reputation/
learning input
Web size/
research output/
reputation
Research output
Data sources
Database
Survey/ database/
institution
database
Database
Outcomes
Presentation
Only Top 100 of
500 institutions
are shown in
numerical orders
Top 400 are shown
in numerical orders
Top 1000 in
numerical order
Top 500 in
numerical order
Transparency
Highly medium
Medium
Highly medium
Highly medium
5
Methodological limitations of global rankings
•
•
•
•
•
Reductionism / Simplicity
Research focus
Unfair for humanities, arts and social science
fields
English domination
Arbitrary selection of indicators and
weightings
6
Popular use of global rankings by
stakeholders
• Students are using ranking tables in their decision-making
about where to study.
• Governments are taking advantage of rankings to know
where to invest
• Scientists use them to know where to work
• Institutions use rankings to know where they stand and
whom they can partner with.
– OECD survey in 2007 showed:
• over 50 % of respondents regarded rankings as a positive impact on
the institution’s reputation and helping its development, such as
student recruitment, academic partnerships and collaborations and
staff morale.
• Majority of the institutions were found to incorporate the outcomes
of rankings into their strategic planning processes at all levels of the
organization and to take policy actions based on them.
• 70 % wanted to be in the top 25 internationally
7
Research design and method
• The main purpose is to explore the leading
factors in 4 major global rankings which
will most affect the rank mobility of an
institution in terms of standard deviation
and K mean of cluster analysis.
• a sophisticated model of strategic
institutional framework for becoming a
world class university is proposed
8
Major Findings
• Statistical analysis on the major
indicators in 4 global rankings by
correlation coefficients
• Rank differences and moving up in 4
global rankings
9
Statistical analysis on the major indicators
in 4 global rankings by correlation
coefficients
10
Correlation coefficients among indicators by cluster
in ARWU ranking
Rank
1~30
31~70
71~10
0
90~11
0
1~100
Score on
Alumni
Score on
Award
Score on
HiCi
Score on
N&S
Score on
PUB
Score on
PCP
0.812**
0.875**
0.860**
0.900**
0.319
0.728**
0.000
0.000
0.000
0.000
0.086
0.000
-0.151
0.250
0.440**
0.741**
0.129
0.010
0.351
0.120
0.004
0.000
0.426
0.952
0.171
0.064
0.061
0.100
0.426*
0.235
0.366
0.738
0.747
0.599
0.019
0.211
-0.075
0.170
0.041
0.184
0.110
-0.090
0.739
0.449
0.856
0.413
0.627
0.692
0.761**
0.838**
0.871**
0.930**
0.636**
0.783**
0.000
0.000
0.000
0.000
0.000
0.000
11
Correlation coefficients among indicators by cluster
in QS ranking
Rank
Academic
Peer
Review
Employer
Review
Faculty
Student
Citations per
Faculty
International
Faculty
International
Student
s
0.452*
0.201
0.629**
0.627**
0.059
0.278
0.012
0.286
0.000
0.000
0.758
0.137
0.318*
0.486**
0.224
0.135
-0.006
0.210
0.043
0.001
0.159
0.401
0.969
0.187
0.214
-0.047
-0.158
0.221
0.051
0.031
0.266
0.810
0.413
0.249
0.792
0.874
-0.123
0.281
0.206
-0.024
-0.002
0.144
0.584
0.205
0.357
0.915
0.995
0.522
0.700**
0.523**
0.565**
0.363**
0.140
0.341**
0.000
0.000
0.000
0.000
0.165
0.001
1~30
31~70
71~100
90~110
1~100
12
Correlation coefficients among indicators by cluster
in HEEACT ranking
Ranks
1~30
31~70
71~100
90~110
1~100
Number
of
articles
in
the last
11
years
Number
Of
articles
in
the
Current
years
Number of
citations in
the last 11
years
Number of
citations in
the last 2
years
Number
Of
citations
in the
last
11 years
H-index
Number of
Highly
Cited
papers
articles in
High
\impact
journals
in
the
current
year
0.825**
0.881**
0.987**
0.991**
0.482**
0.903**
0.974**
0.989**
0.000
0.000
0.000
0.000
0.007
0.000
0.000
0.000
0.414**
0.422**
0.679**
0.694**
0.031
0.525**
0.662**
0.495**
0.008
0.007
0.000
0.000
0.847
0.000
0.000
0.001
-0.017
0.091
0.349
0.577**
0.238
0.405*
0.177
0.312
0.929
0.632
0.059
0.001
0.205
0.026
0.350
0.094
0.231
0.041
0.363
0.286
0.141
0.022
0.338
0.108
0.314
0.859
0.106
0.209
0.541
0.926
0.134
0.642
0.854**
0.834**
0.984**
0.988**
0.439**
0.920**
0.971**
0.977**
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
13
Correlation coefficients among indicators by
cluster in Webmetrics ranking
Rank
1~30
31~70
71~100
90~110
1~100
SIZE
VISIBILITY
RICH
SCHOLAR
0.807**
0.946**
0.606**
0.756**
0.000
0.000
0.000
0.000
0.449**
0.797**
0.595**
0.531**
0.004
0.000
0.000
0.000
0.473*
0.331
0.170
0.361
0.011
0.085
0.388
0.059
-0.330
0.578**
-0.285
-0.004
0.143
0.006
0.210
0.987
0.845**
0.949**
0.835**
0.822**
0.000
0.000
0.000
0.000
14
Rank differences and moving up
in 4 global rankings
15
ARWU-Numbers of positions moving up by
clusters
Average
Positions
improved
(mean)
Clusters
Numbers of
Positions
moving up
Cluster one
1-17
156
71.6%
6.51
4.65
Cluster two
18-50
55
25.2%
29.33
10.88
Cluster three
0ver 50
7
3.2%
74.71
15.76
218
100.0%
total
Highest
moving up
positions
No of
institutions
%
SD (No.)
94
16
ARWU-Numbers of positions moving up
and down by indicators
increased positions in 2009
100
90
80
70
60
50
40
30
20
10
0
0
100
200
300
400
500
Rank in 2008
17
QS RANKINGS Numbers of positions
moving up by clusters
Clusters
Numbers of
ranks
moving up
Cluster one
1-30
Cluster two
over 30
total number of moving ups
Highest
moving up
positions
No of
institutions
%
Average
Ranks
improved
(mean)
Standard
deviation
144
84.7%
13.15
10.11
26
15.3%
62.84
19.69
170
100.0%
125
18
WEBOMETRICS: Numbers of positions
moving up by clusters
Clusters
Numbers of
positions
moving up
Cluster one 1-39
No of
institutions
%
Average
positions
improved
(mean)
SD (No)
156
64.5%
16.21
10.90
Cluster
two
40-99
76
31.4%
61.45
16.49
Cluster
three
Over 100
10
4.1%
137.40
34.03
242
100.0
%
total number of moving ups
Highest
moving up
positions
212
19
HEEACT Numbers of positions moving up
by clusters
Clusters
Numbers of
Positions
moving up
No of
institutions
%
Average
positions
increased
(mean)
SD (No. )
Cluster one
1-19
153
66.2%
8.24
5.34
Cluster two
20-45
61
26.4%
30.23
7.11
Cluster three
Over 46
17
7.4%
60.18
10.49
total number of moving ups
231
100.0%
Highest moving up positions
82
20
Comparison among 4 global rankings by
positions rising
ARWU
QS
Webmetrics
HEEACT
Cluster one
1-17
1-30
1-39
1-19
Cluster two
20-45
Over 30
40-99
20-45
Cluster three
Over 46
X
Over 100
Over 46
242 (500)
231(500)
212
82
total number of positions
moving ups
170 (400)
218(500)
Highest ranks moving up 94
125
21
Major factors for positions mobility
•
•
•
Staying on the top 30:
–
Award of “Nobel Prize” is the most influential indicators to be on top 30 in
ARWU
–
“Academic peer review” in QS rankings,
–
‘Internet visibility’ in Webometrics,
–
“Citations in the last 2 years” in HEEACT ranking.
Moving into top 100:
–
HiCi, N& S and PUB are the most influential indicators in ARWU,
–
“Academic peer review” in QS rankings,
–
‘Size’ in Webometrics,
–
“Citations in the last 2 years and papers” and “H-Index” in HEEACT
ranking.
Moving up positions:
–
PCP, N& S and PUB are the key factors in ARWU
–
“Academic peer review” in QS rankings
–
“Visibility’ in Webometrics
–
H-index in HEEACT rankings
22
Flow Chart of Implication of 4 Global Ranking on
Making Institutional Strategic Plans
Technology/Internet International Reputation
Excellence
Short term(3-5 years)
Webometrics Ranking
Mid-term 5-15 years
QS Rankings
Academic
Long-term(15~30years)
ARWU/Shanghai
Ranking
HEEACT Ranking:
Used to inspect the quality and quantity of FACUTLY publications annually
23
Summary
• The proposal of the strategic planning model above is
completely based on the 4 global rankings, so leading
factors in the 3 categories are definitely relevant to the
research outputs of an institution.
• Some of these indicators will take longer time to improve,
such as Nobel Laureates and Academic peer review.
• If all institutions follow the model, it’s highly likely that
not all of them will actually move into the ranking in the
spots they expect to be.
• it is necessary to note that these are only guidelines and
not meant to be used as a rigid cause and effects.
• Academics should not to rely on a single model only to
implement in terms of educational policy.
24
Conclusion
• To achieve a good rankings is becoming
more and more important
• Global rankings are increasingly being used
as a tool for building world class
universities
25
Final question raised by the paper
• To what extent can a world class university
be replicated by using the factors
highlighted in a ranking model and how
can it be done?
• The answer is both yes it can be replicated
and no it can’t be.
26
Yes and No Answers
• The model, based on statistical analysis, can only
provide very rough guidance and clues to
institutions on which road to take to achieve
academic excellence.
• a clear vision, institutional features, favourable
governance, and sufficient resources which were
not taken into consideration in the above model
(or in the 4 global rankings themselves) are all
very crucial if a university is to rise and stay top
in the rankings.
27
“THERE IS NO SINGLE ROAD TO
EXCELLENCE”
by Jamil Salmi (2010)
28
Thank you for your attention
Question and Comments
Fu Jen Catholic University
Higher Education Evaluation & Accreditation
Council of Taiwan
29