Transcript 大學評鑑的作為 - utaipei.edu.tw

Angela Yung Chi Hou , 19 Dec, 2012
Fu Jen Catholic University
INTERNATIONAL TREND ON
UNIVERSITY RANKINGS AND ITS
IMPACT ON HIGHER EDUCATION
1
Presentation Outline
I.
II.
III.
IV.
V.
VI.
VII.
Introduction
Development of Rankings
Limitations of Rankings
Impact of Rankings
Outcomes of Ranking
Use of Rankings
Conclusion
2
I. Introduction
1. Three major concerns
2. Meeting challenges to higher
education in the 21st century
3. Recipe for higher education
4. World class university
5. Global rankings and world class
university
6. Rational of college ranking
3
1. Three Major Concerns for Development
of Higher Education in Global Times
Accessibility
Higher education enrollment rate has been
increasing in the past decade
Affordability
Finance need is not an impediment for eligible
students to attend a college
Accountability
Enormous resources and talent are available in
higher education institutions
Help students be ready for college, and be
equipped to graduate from college
4
2. Challenges to Higher Education
in 21st Century
Over expansion of higher education
2305 post secondary higher education institutions in
China
726 universities and 488 junior colleges in Japan
201 universities in Korea
160 universities in Taiwan
15 in Hong Kong (9/6)
12 in Macau(4/8)
Private universities outnumber public universities (except
China)
Declining birth rate
three are below 1.5
Declining financial governmental support
5
3. Recipe for Higher Education
- quality assurance and global competitiveness
Set up Internal and External Quality Assurance
System
Establishment of National Accrediting agency
Establish Quality Culture on Campus
Compulsory audit
Enhance international academic competitiveness
Launching several Excellence Programs
Brain21 (Korea)
Center for Excellence COE (Japan)
5 year-50 Billion Program (Taiwan)
China 985 project
Aiming at establishing world class universities
6
4. What does a world class university look
like?
In terminology
world class universities: top universities striving
for “excellence”
quality must surpass the expectation of stakeholders
Philip Altbach
excellence in research, top professors, academic
freedom and an atmosphere of intellectual
excitement, governance, adequate facilities and
funding
7
4. What does a world class university look
like?
Jamil Salmi (World Bank) based on two
rankings (Shanghai and QS)
a high concentration of talent (faculty and
students)
abundant resources to offer a rich learning
environment and conduct advanced research
favorable governance
features that encourage strategic vision, innovation
and flexibility, and enable institutions to make
decisions and manage resources without being
encumbered by bureaucracy
8
5. Relevance between Global Rankings and
World Class University
characteristics of world class universities are
strongly correlated to most indicators used by
global rankings
nations use global rankings as a basis of
building world class universities despite
methodological flaws
9
5. Relevance between Global Rankings and
World Class University
top administrators at leading universities use
global rankings to achieve the short term and
long term strategic plans, not just to boycott
them
Minnesota’s initiative  become one of the top
three research institutions in the world
Taiwan National University  “Moving into the
top 100” at its 80th anniversary
Baylor University  one of the U.S. News Top 50
by 2012
10
6. Rational of College Ranking
Higher education expansion
Resources allocation
Accountability
Benchmarking
Marketization in higher education
11
II. Development of Rankings
1.
2.
3.
4.
5.
6.
National College Ranking
Global Ranking
Types of Rankings
Scoring Methods
Five Global Rankings
Criteria & Indicators of Rankings
12
1. National College Ranking
U.S. News & World Report
The most influential college ranking – “American
Best Colleges” published by U.S. News & world
Report in 1983
Maclean’s , The Times, CHE, etc.
13
2. Global Ranking
intense international competitions
global college rankings have drawn
international attention
14
3. Types of Ranking
By region
by country/continent/worldwide
By field / program
such as Engineering/social science/humanities
By subject
i.e., Biology/ Math
College guide
Princeton review
15
4. Scoring Methods
The indicators were weighted at a certain
ratio and the scores were aggregated to rank
each college.
The top one university received highest points
while the scores for the remaining schools
descended accordingly.
16
5. Five Global Rankings
Academic Ranking of World Universities
Shanghai Jiao Tong University, 2003
World University Ranking
QS, 2004
Webometrics Rankings of World Universities
Spanish National Research Council, 2004
Performance ranking of scientific papers of
world class universities
HEEACT, 2007
World University Ranking
The Times Higher Education, 2010
17
6. Criteria & Indicators of Ranking
2011 ARWU Ranking
2011 QS Ranking
2011 Webometrics Ranking
2011 THE Ranking
2011 HEEACT Ranking
18
6. 2011 ARWU Ranking
分 類指標
分 項指標
權重
操 作型定 義
分類指標
分項指標
權重
操作型定義
教育品質
10%
獲得諾貝爾及菲爾茲獎校友人數。得分高低依畢業
年度，每往前推十年，降低10%權重。如2001年以
後每人以1人計算；1991-2000年每人以0.9人計算 ，
依次遞減。
獲得重要獎項人數
(Award)
20%
獲得諾貝爾及菲爾茲獎教師人數。得分高低如上。
如2001以後獲獎者每人以1人計算；1991-2000年每
人以0.9人計算，依次遞減。獲獎人同時署名兩單位，
各計.05人。諾貝爾共同得獎者，以獎金比例分配權
重。
論文高度被引用研究人數(HiCi)
20%
在21個科學領域中，論文名列ISIHighlyCited.com高
度引用之教師人數。
《自然》與《科學》兩期刊論
文發表篇數(N&S)
20%
2006至2010年間，一所大學在《自然》與《科學》
兩期刊論文發表篇數，不包含評論與快訊。
SCI與SSCI論文數(PUB)
20%
2010年，一所大學發表論文被SCI及SSCI收錄的數量。
機構規模（專任教師數）(PCP)
10%
以上五項總分/專任教師數
畢業校友
(Alumni)
教師品質
研究成果
機構規模
6. 2011 QS Ranking
評比內容
權重
分項指標
學術聲譽
(Academic Reputation)
研究
60%
(Research)
教師平均論文被引用數
(Citations per Faculty)
就業力
(Employability)
就業聲譽
10%
教學
(Teaching)
(Employer Reputation)
師生比
20%
(Faculty/Student Ratio)
國際教師比例
國際化
(Internationalization)
(International Faculty Ratio)
10%
國際學生比例
(International Student Ratio)
6. 2011 Webometrics Ranking
Indicators
Definition
IMPACT
Number of backlines Current
Number of back
domains
Majestic
SEO and
ahrefs
50%
Presence
Number of web
p ages
Number of papers
(pdf, doc, docx, ppt)
Current
Google
20%
2007-2010 Google
scholar
15%
10% most cited
papers in their
respective scientific
fields
2003-2010 Scimago
15%
openness
Excellence
Covergae Sources
Weight
21
6. 2011 THE Ranking
評比內容
權重
教學－
學習環境
(Teaching -the
30%
learning environment)
研究－
數量、收入與聲譽
論文引用－研究影響
30%
30%
國際化－職員與學生
(International outlook
7.5%
-staff and students)
業界收入－創新
(Industry income innovation)
分項指標
分項指
標權重
教學聲譽(Reputational survey-Teaching)
15%
教師平均指導博士畢業生數(PhD awards per academic)
6%
教師平均教授大學生數(Undergraduates admitted per academic)
4.5%
授予博、學士學位比(PhD-to-bachelor’s ratio)
2.25%
教師平均分配院校收入(Income per academic)
2.25%
研究聲譽(Reputational survey-Research)
18%
研究收入(Research Income)
6%
教師與研究員平均論文篇數(Papers per academic and research staff)
6%
平均論文引用數(Average citations per paper)
30%
國際對本地生比(Ratio of international to domestic students)
2.5%
國際對本地教職員比(Ratio of international to domestic staff)
2.5%
國際合作論文比例(Proportion of internationally coauthored research papers)
2.5%
研究人員自業界之平均研究收入
2.5%
2.5%
(Research income from industry per academic staff)
6. 2011 HEEACT Ranking
Criteria
Research
productivity
Research
impact
Research
excellence
Indicator
Weight
Number of articles in the last 11 years (20002010)
10%
Number of articles in the current year (2010)
10%
Number of citations in the last 11 years (20002010)
10%
Number of citations in the last 2 years (20092010)
10%
Average number of citations in the last 11 years
(2000-2010)
10%
h-index of the last 2 years (2009-2010)
20%
Number of Highly Cited Papers (2000-2010)
15%
Number of articles in high-impact journals in
the current year (2010)
15%
20%
30%
50%
23
III. Limitations of Rankings
1. Characteristics of 5 major Global
Rankings
2. Methodological Limitations
24
1. Characteristics of 5 Major Global Rankings
ARWU
QS
Webometrics
HEEACT
THE
Established
year
2003
2004
2004
2007
2009
Institution
Academic
institution
Mass
media/Private
Education
consulting firm
Governmental
research unit
QA Agency
Mass
media
Goal
Academic
competition
Profit making
Academic
sharing
Benchmarking
Academic
Competition
Number of
indicators
6
6
4
8
13
Indicator
category
Research
output/
learning input
Research output / Web size/
Reputation survey research
/
output/
learning input
reputation
Research
output
Teaching/
Research
Output
/survey
Data sources
Database
Survey/
database/
institution
database
Database
Survey/
database/
institution
Outcomes
Presentation
Only Top 100 of
500 institutions
are shown in
numerical orders
Top 400 are
shown
in numerical
orders
Top 1000 in
numerical
order
Top 500 in
numerical
order
Top 200 are
Shown in
numerical
orders
Transparency
Highly medium
Medium
Highly
medium
Highly medium
Medium
25
2. Methodological Limitations of Global
Rankings
Reductionism / Simplicity
Research focus
Unfair for humanities, arts and social science
fields
English domination
Arbitrary selection of indicators and weightings
Data quality
26
IV. Impact of Rankings
1.
2.
3.
4.
Outcomes of Rankings
Popular Use by Stakeholders
Two Major Reports
Performance in Asia
27
1. Outcomes of Rankings
US and UK institutions are on the top
Asia is on rise, particularly those with
Excellence policy
28
2. Popular Use of Global Rankings by
Stakeholders
Students
to decide where to study
Governments
to know where to invest
Scientists
to know where to work
Institutions
to know where they stand and whom they can
partner with
29
3. Two Major Reports
OECD survey in 2007 showed:
over 50 % of respondents: rankings has a positive
impact on the institution’s reputation (student
recruitment, academic partnerships and
collaborations and staff morale)
majority of the institutions incorporated the
outcomes of rankings into strategic planning
processes at all levels
70 % wanted to be in the top 25 internationally
30
4. Two Major Reports
OECD survey in 2007 showed:
an on–line UK study focused on English
Universities ‘ attitudes toward rankings
rankings reflect the views of what properties a
good university should develop that influenced
the institutional and governmental polices
a high level of agreement that the reputation of
an institution might be affected by rankings
many institutions further down in the rankings
do not care too much about global rankings
31
5. Average Number of top 500 Universities of Japan, China,
South Korea and Taiwan by three rankings
Country
ARWU
HEEACT
QS
Ave No
(2004~2012) (2007~2011) (2007~2012)
Japan
29.56
29.60
25.67
28.27
China
16.44
14.20
12.33
14.33
South Korea
9.00
9.20
11.50
9.90
Taiwan
6.22
5.40
8.50
6.71
Hong Kong
5.00
5.00
5.67
5.22
India
2.00
2.00
8.00
4.00
32
V. Ranking Outcomes
1. ARWU / QS/ HEEACT/ THE Ranking
2. Number of papers and
internationalization in China, Korea,
Taiwan and Japan
33
Top 20 Universities in the
Four Rankings (2/2)
HEEACT
(2011)
15
16
ARWU
(2012)
15
20
THE
(2012)
-----
QS
(2012)
-----
17
---
17
4
18
19
--11
--11
20
7
Imperial College London
20
---
8
6
California Institute of Technology
Princeton University
University of Chicago
Cornell University
University of Wisconsin - Madison
ETH Zürich - Swiss Federal Institute of
Technology Zürich
McGill University
-----------
6
7
9
13
19
1
5
9
20
---
10
9
8
14
---
---
---
15
13
---
---
---
18
University Name
University of California - San Diego
The University of Tokyo
University of London - University
College London
Duke University
Yale University
34
Top 20 Universities in the
Four Rankings (2/2)
HEEACT
(2011)
15
16
ARWU
(2012)
15
20
THE
(2012)
-----
QS
(2012)
-----
17
---
17
4
18
19
--11
--11
20
7
Imperial College London
20
---
8
6
California Institute of Technology
Princeton University
University of Chicago
Cornell University
University of Wisconsin - Madison
ETH Zürich - Swiss Federal Institute of
Technology Zürich
McGill University
-----------
6
7
9
13
19
1
5
9
20
---
10
9
8
14
---
---
---
15
13
---
---
---
18
University Name
University of California - San Diego
The University of Tokyo
University of London - University
College London
Duke University
Yale University
35
Figure 1: The number of the paper published on SCI and SSCI journals
in China, Taiwan, Japan, South Korea
36
Figure 2: Number of papers in Nature & Science in seven years
in China, Taiwan, Japan, South Korea
37
Figure 3: Number of international students
in China, Taiwan , Japan, South Korea
38
VI. Use of Rankings
1. Findings in rank Mobility
2. Rank Differences and moving UP in 4
Global Rankings
3. Berlin Principles and Ranking Audit
4. Future Development
5. CHE Excellence Ranking and Research
Ranking
6. College Navigator in Taiwan
39
1. Some Findings in Rank Mobility of
Global Rankings
Hou, Yung-chi, & M. Robert (2011) .
An Analysis of Positions Mobility in Global
Rankings: Making Institutional Strategic Plans and
Positioning for Building World Class Universities.
Higher Education Research & Development (SSCI).
To explore the major factors of rank mobility
in 4 major rankings.
40
2. Rank Differences and moving Up in
4 Global Rankings
Comparison among 4 Global Rankings by
positions rising
Implication of 4 Global Ranking on making
institutional strategic plans
41
2. Comparison among 4 Global Rankings
by Positions Rising
ARWU
Cluster one
1-17
Cluster two
20-45
Cluster three
Over 46
QS
1-30
Over 30
X
Webmetrics
HEEACT
1-39
1-19
40-99
20-45
Over 100
Over 46
242 (500)
231(500)
212
82
170 (400)
total number of positions
moving ups
218(500)
Highest ranks moving up
94
125
42
2. Flow Chart of Implication of 4 Global Ranking
on Making Institutional Strategic Plans
Technology/Internet International Reputation
Short term(3-5 years)
Webometrics Ranking
Mid-term 5-15 years
QS Rankings
Academic Excellence
Long-term(15~30years)
ARWU/Shanghai
Ranking
HEEACT Ranking:
Used to inspect the quality and quantity of FACUTLY publications annually
43
3. Berlin Principles and Ranking Audit
International Ranking Expert Group (IREG)
founded in 2004
by the UNESCO European Centre for Higher
Education (UNESCO-CEPES) in Bucharest and the
Institute for Higher Education Policy in
Washington, DC.
44
3. Berlin Principles and Ranking Audit
It is upon this initiative that IREG’s second
meeting (Berlin, 18 to 20 May, 2006) has been
convened to consider a set of principles of
quality and good practice in HEI rankings—the
Berlin Principles on Ranking of Higher
Education Institutions.
IREG-5 in Oct, 5-7, 2010 proposed “Ranking
Audit”
45
3. Berlin Principles: What Rankings and League
Tables should Consider
To ensure the quality of rankings: research
methods, indicators, data quality,
transparency, varying user’s interests, etc.
4 major principles
Purposes and Goals of Rankings
Design and Weighting of Indicators
Collection and Processing of Data
Presentation of Ranking Results
19 criteria of audit published by IREG
46
4. Future Development for Rankings
Field/subject based ranking
Varying ranking providers
More interactive, multi-dimensional,
personalized
CHE European Excellence Ranking
College Navigator in Taiwan
Web-based
Benefit student mobility
Student survey
47
5. CHE Excellence Ranking and Research
Ranking
Multi-dimensional global ranking that will be
based on the CHE ranking approach.
Results of the feasibility study will be available
in mid-2011.
One aspect will be the development of a
concept to introduce a web-based tool for
personalized rankings for particular target
groups on a global scale.
48
49
College Navigator in Taiwan
Published in 2009
Goal
lead to a match between the student and the institution or the
program that they’re most interested
Selection of Institutions
69 4-year colleges and universities evaluated by HEEACT
from 2006 to 2010.
77 University of Technology and Technical College
50 50
 3 professors/ one international consultant / one full time staff/ one
IT designer / 7 graduate students
Research Team
51
6. College Navigator in Taiwan-Home
Click here to
start
52
College Navigator in Taiwan- Step 1: Indicators
Step 1:
Choose the
indicators
53
College Navigator in Taiwan- Step 1: Indicators
Step 1: Choose
the indicators
54
College Navigator in Taiwan- Step 2: Weighting
Step 2: Give
each indicator
a weight
55
College Navigator in Taiwan- Step 3: Preference
Step 3:
Decide the
Universities
you want to
compare
Way 1: Narrow down your
choices
Way 2:Choose Universities
56
College Navigator in Taiwan- Step 4: Result
The indicators you choose
Our system will analyze
the rank of the
universities according
to the indicators and
weights you decide
The performance of
each university
57
College Navigator in Taiwan- Step 5: General l information
58
59
Hou, Angela Yung-chi, Morse, R., & Shao, Y.
J. E.（2012）. Is There a Gap between
Students’ Preference and University Presidents’
Concern over College Ranking Indicators? : A
Case Study of “College Navigator in Taiwan”,
Higher Education ( in press) (SSCI) (2010
Impact Factor 0.823). ISSN: 0018-1560
60
Figure 2: Scatter Plot of level of Importance
over Criteria and Indicators
Table 6 Mean and STD by items
Categories
Mea
n
ST
D
95% Confidence
Interval
Upper
Lower
Institutional policy making
3.95
0.6
1
4.14
3.76
Staff and faculty recruitment
3.93
0.6
7
4.14
3.73
Research output
4.06
0.7
6
4.30
3.83
Resources allocation
3.82
0.6
9
4.04
3.61
Student services and learning
environment
3.86
0.7
2
4.09
3.62
System operation
3.93
0.7
5
4.16
3.69
61
Table 8 Comparison of rank of importance of the indicators among focus students, general users
and institutions
Focus group’ opinions
Graduate rate
Total amount of equipment per student
Expenditure per student
Number of international academic awards
earned by students within last 3 years
students within last 3 years
Total library holdings per student/
Total grant from collaborations between of
University and industry
Most popular indictors
Academic survey * (higher)
Expenditure per student
Enrollment rate
Faculty-student ratio* (higher)
Graduation rate* (lower)
Number of national academic
awards by students (higher)
Total NSC Research grants
Proportion of faculty members with Ph.D.
Total library holdings per
student** (lower)
Total NSC grants per faculty**
(lower)
Institutions’ attitude
Institutional policy making (
rank 2)
Resources allocation
(Rank 6)
Student services and learning
environment (Rank 5)
Student services and learning
environment(Rank 5)
Student services and learning
environment
(Rank 5)
Student services and learning
environment
(Rank 5)
Resources allocation (Rank
6)
Research output (Rank 1)
Number of national academic awards earned
Proportion of full-time faculty
by students within last 3 years
Faculty Resource (Rank 3)
Average piece of collaborations between of
62
University and industry per full-time faculty Proportion of professors with Ph.D.
VII. Conclusion
1.
2.
3.
4.
Conclusion
Final Questions
Yes and No Answers
How Ecosystem Influences Top Research
Universities
63
1. Conclusion
To achieve a good rankings is becoming more
and more important.
Global rankings are increasingly being used as
a tool for building world class universities and
pursuing academic excellence .
64
2. Final question raised by the presentation
To what extent can a world class university be
replicated by using the factors highlighted in a
ranking model?
How can it be done?
The answer is both “yes,” it can be replicated, and
“no,” it can’t be.
65
3. Yes and No Answers
Rankings can only provide very rough
guidance and clues to institutions on which
road to take to achieve academic excellence.
Not taken into account, but very crucial:
a clear vision
institutional features
favourable governance
sufficient resources
66
4. Understanding How the Ecosystem Influences the
Performance of Top Research Universities by Jamil Salmi
67
“THERE IS NO SINGLE ROAD TO EXCELLENCE”
by Jamil Salmi (2010)
Coordinator,
Tertiary Higher Education, World Bank
68
Thank you for your attention!
Questions and Comments
69
References
*Hou, Angela Yung-chi (2012). Mutual Recognition of Quality Assurance Decisions on Higher
Education Institutions in Three Regions-A Lesson for Asia. Higher Education, 64:911-926. (SSCI) (2011
Impact Factor 1.016). ISSN: 0018-1560
*Hou, Yung-chi, Morse, R., & Shao, Y. J. E.（2012）. Is There a Gap between Students’ Preference and
University Presidents’ Concern over College Ranking Indicators? : A Case Study of “College Navigator
in Taiwan”, Higher Education, 64:767–787. (2011 Impact Factor 1.016)
Hou, Angela Yung-chi (2012 Quality in Cross-Border Higher Education and Challenges for the
Internationalization of National Quality Assurance Agencies in the Asia-Pacific Region –Taiwan
Experience. Studies in Higher Education. (online) (SSCI)
*Hou, Yung-chi., Ince, M., & Chiang, C.L. (2012). A Reassessment of Asian Excellence Programs in
Higher Education – the Taiwan Experience. Scientometrics. 92, 23-42 (SSCI). Print ISSN: 0138-9130.
(2011 Impact Factor 1.966)
*Hou, Yung-chi.（2011）. An Analysis of Positions Mobility in Global Rankings:
Making Institutional Strategic Plans and Positioning for Building World Class Universities. Higher
Education Research & Development (SSCI）(online) ISSN: 0729-4360(2011 Impact Factor 0.901)
*Hou, Yung-chi.（2011）. Impact of excellence programs on Taiwan higher education in terms of
quality assurance and academic excellence, examining the conflicting role of Taiwan’s accrediting
agencies. Asian Pacific Educational Review, 13, 77-88 (SSCI). ISSN: 1598-1037
*Hou, Yung-chi.（2011）. Quality Assurance at a Distance : International Accreditation in Taiwan
Higher Education, Higher Education, 61(2), 179–191 (SSCI)