1

AC 21 International Forum

Competition and Cooperation among Universities in the age of Internationalization

An Analysis of Positions Mobility of Global Rankings:The Effective Use of Global Rankings in Making

Institutional Strategic Plans and Positioning for Building

World Class Universities

Dr. Angela Yung-chi HouDean of Office of Research & Development,

Higher Education Evaluation and Accreditation Council of TaiwanDirector of Faculty Development & Instructional Resources Center ,

Fu Jen Catholic University

18-21 OCT , 2010Shanghai

2

Introduction

• Globalization in the 21st century presents universities and states with a number of challenges and opportunities.

• No matter whether countries are developed or developing ones, they are immensely eager to build at least one world class university, but they don’t know exactly what they look like.

3

What does a world class university look like ?

• In terminology– world class universities are top universities striving for

“Excellence”, in other words, it means “its quality must surpass the expectation of their various stakeholders”

• Philip Altbach– excellence in research, top professors, academic freedom and an

atmosphere of intellectual excitement, governance, adequate facilities and funding.

• Jamil Salmi (World Bank) based on two rankings (Shanghai and QS) – a high concentration of talent (faculty and students)– abundant resources to offer a rich learning environment and

conduct advanced research – favorable governance (features that encourage strategic

vision, innovation and flexibility, and enable institutions to make decisions and manage resources without being encumbered by bureaucracy)

4

Relevance between global rankings and World Class University

• the characteristics of world class universities are inevitably deemed to be strongly correlated to most indicators used by global rankings.

• Many nations tend to use global rankings as a basis of building world class universities despite their well documented methodological flaws.

• Many top administrators at leading universities are learning to use global rankings wisely in order to achieve the institutional short term and long term strategic plans, not just to boycott them. – Minnesota’s initiative to become one of the top three

research institutions in the world– Taiwan National University announced the initiative of

“Moving into the top 100” at its 80th anniversary– Baylor University put the vision on making the institution

one of the U.S. News Top 50 by 2012.

5

Characteristics of 4 Major Global rankings and their methodological limitations

ARWU QS (THE)* Webmetrics HEEACT

Established year 2003 2004 2004 2007

Institution Academicinstitution

Massmedia/PrivateEducationconsulting firm

Governmental research unit

QA Agency

Goal Academiccompetition

Profit making Academic sharing Benchmarking

Number ofindicators

6 6 4 8

Indicatorcategory

Researchoutput/ learning input

Research output /reputation/learning input

Web size/research output/reputation

Research output

Data sources Database Survey/ database/institution

database Database

Outcomes Presentation

Only Top 100 of500 institutions are shown innumerical orders

Top 400 are shownin numerical orders

Top 1000 innumerical order

Top 500 innumerical order

Transparency Highly medium Medium Highly medium Highly medium

6

Methodological limitations of global rankings

• Reductionism / Simplicity

• Research focus

• Unfair for humanities, arts and social science fields

• English domination

• Arbitrary selection of indicators and weightings

7

Popular use of global rankings by stakeholders

• Students are using ranking tables in their decision-making about where to study.

• Governments are taking advantage of rankings to know where to invest

• Scientists use them to know where to work• Institutions use rankings to know where they stand and

whom they can partner with. – OECD survey in 2007 showed:

• over 50 % of respondents regarded rankings as a positive impact on the institution’s reputation and helping its development, such as student recruitment, academic partnerships and collaborations and staff morale.

• Majority of the institutions were found to incorporate the outcomes of rankings into their strategic planning processes at all levels of the organization and to take policy actions based on them.

• 70 % wanted to be in the top 25 internationally

8

Research design and method

• The main purpose is to explore the leading factors in 4 major global rankings which will most affect the rank mobility of an institution in terms of standard deviation and K mean of cluster analysis.

• a sophisticated model of strategic institutional framework for becoming a world class university is proposed

9

Major Findings

• Statistical analysis on the major indicators in 4 global rankings by correlation coefficients

• Rank differences and moving up in 4 global rankings

10

Statistical analysis on the major indicators in 4 global rankings by correlation

coefficients

11

Correlation coefficients among indicators by cluster in ARWU ranking

RankScore onAlumni

Score onAward

Score onHiCi

Score onN&S

Score on PUB

Score onPCP

1~300.812** 0.875** 0.860** 0.900** 0.319 0.728**

0.000 0.000 0.000 0.000 0.086 0.000

31~70-0.151 0.250 0.440** 0.741** 0.129 0.010

0.351 0.120 0.004 0.000 0.426 0.952

71~100

0.171 0.064 0.061 0.100 0.426* 0.235

0.366 0.738 0.747 0.599 0.019 0.211

90~110

-0.075 0.170 0.041 0.184 0.110 -0.090

0.739 0.449 0.856 0.413 0.627 0.692

1~100 0.761** 0.838** 0.871** 0.930** 0.636** 0.783**

0.000 0.000 0.000 0.000 0.000 0.000

12

Correlation coefficients among indicators by cluster in QS ranking

RankAcademic

Peer Review

Employer Review

Faculty Student

Citations per

Faculty

International Faculty

International Student

s

1~300.452* 0.201 0.629** 0.627** 0.059 0.278

0.012 0.286 0.000 0.000 0.758 0.137

31~700.318* 0.486** 0.224 0.135 -0.006 0.210

0.043 0.001 0.159 0.401 0.969 0.187

71~1000.214 -0.047 -0.158 0.221 0.051 0.031

0.266 0.810 0.413 0.249 0.792 0.874

90~110-0.123 0.281 0.206 -0.024 -0.002 0.144

0.584 0.205 0.357 0.915 0.995 0.522

1~1000.700** 0.523** 0.565** 0.363** 0.140 0.341**

0.000 0.000 0.000 0.000 0.165 0.001

13

Correlation coefficients among indicators by cluster in HEEACT ranking

Ranks

Number of

articles in

the last 11

years

Number Of

articles inthe

Currentyears

Number ofcitations inthe last 11years

Number ofcitations inthe last 2

years

NumberOf

citations in the

last11 years

H-index

Number of HighlyCitedpapers

articles in High

\impact journals

inthe

current year

1~300.825** 0.881** 0.987** 0.991** 0.482** 0.903** 0.974** 0.989**

0.000 0.000 0.000 0.000 0.007 0.000 0.000 0.000

31~700.414** 0.422** 0.679** 0.694** 0.031 0.525** 0.662** 0.495**

0.008 0.007 0.000 0.000 0.847 0.000 0.000 0.001

71~100-0.017 0.091 0.349 0.577** 0.238 0.405* 0.177 0.312

0.929 0.632 0.059 0.001 0.205 0.026 0.350 0.094

90~1100.231 0.041 0.363 0.286 0.141 0.022 0.338 0.108

0.314 0.859 0.106 0.209 0.541 0.926 0.134 0.642

1~1000.854** 0.834** 0.984** 0.988** 0.439** 0.920** 0.971** 0.977**

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

14

Correlation coefficients among indicators by cluster in Webmetrics ranking

Rank SIZE VISIBILITY RICH SCHOLAR

1~300.807** 0.946** 0.606** 0.756**

0.000 0.000 0.000 0.000

31~700.449** 0.797** 0.595** 0.531**

0.004 0.000 0.000 0.000

71~1000.473* 0.331 0.170 0.361

0.011 0.085 0.388 0.059

90~110-0.330 0.578** -0.285 -0.004

0.143 0.006 0.210 0.987

1~1000.845** 0.949** 0.835** 0.822**

0.000 0.000 0.000 0.000

15

Rank differences and moving up

in 4 global rankings

16

ARWU-Numbers of positions moving up by clusters

ClustersNumbers of

Positionsmoving up

No ofinstitutions

%

AveragePositionsimproved

(mean)

SD (No.)

Cluster one 1-17 156 71.6% 6.51 4.65

Cluster two 18-50 55 25.2% 29.33 10.88

Cluster three 0ver 50 7 3.2% 74.71 15.76

total 218 100.0%

Highestmoving uppositions

94

17

ARWU-Numbers of positions moving up and down by indicators

0

10

20

30

40

50

60

70

80

90

100

0 100 200 300 400 500

Rank in 2008

incre

ased p

ositio

ns in 2

009

18

QS RANKINGS Numbers of positions moving up by clusters

ClustersNumbers of

ranksmoving up

No ofinstitutions

%

AverageRanks

improved(mean)

Standard deviation

Cluster one 1-30 144 84.7% 13.15 10.11

Cluster two over 30 26 15.3% 62.84 19.69

total number of moving ups 170 100.0%

Highestmoving uppositions

125

19

WEBOMETRICS: Numbers of positions moving up by clusters

ClustersNumbers of

positions moving up

No ofinstitutions

%

Average positions improved

(mean)

SD (No)

Cluster one 1-39 156 64.5% 16.21 10.90

Cluster two 40-99 76 31.4% 61.45 16.49

Cluster three

Over 100 10 4.1% 137.40 34.03

total number of moving ups 242100.0

%

Highestmoving uppositions

212

20

HEEACT Numbers of positions moving up by clusters

ClustersNumbers of

Positionsmoving up

No of institutions

%

Averagepositions increased(mean)

SD (No. )

Cluster one 1-19 153 66.2% 8.24 5.34

Cluster two 20-45 61 26.4% 30.23 7.11

Cluster three Over 46 17 7.4% 60.18 10.49

total number of moving ups 231 100.0%

Highest moving up positions 82

21

Comparison among 4 global rankings by positions rising

ARWUQS Webmetrics HEEACT

Cluster one 1-17 1-30 1-39 1-19

Cluster two 20-45 Over 30 40-99 20-45

Cluster three Over 46 X Over 100 Over 46

total number of positionsmoving ups

218(500)170 (400)

242 (500) 231(500)

Highest ranks moving up 94 125 212 82

22

Major factors for positions mobility

• Staying on the top 30: – Award of “Nobel Prize” is the most influential indicators to be on top 30 in

ARWU– “Academic peer review” in QS rankings, – ‘Internet visibility’ in Webometrics,– “Citations in the last 2 years” in HEEACT ranking.

• Moving into top 100: – HiCi, N& S and PUB are the most influential indicators in ARWU, – “Academic peer review” in QS rankings, – ‘Size’ in Webometrics, – “Citations in the last 2 years and papers” and “H-Index” in HEEACT

ranking. • Moving up positions:

– PCP, N& S and PUB are the key factors in ARWU– “Academic peer review” in QS rankings – “Visibility’ in Webometrics– H-index in HEEACT rankings

23

Flow Chart of Implication of 4 Global Ranking on Making Institutional Strategic Plans

Webometrics Ranking QS Rankings ARWU/Shanghai Ranking

HEEACT Ranking: Used to inspect the quality and quantity of FACUTLY publications annually

Short term(3-5 years) Mid-term 5-15 years Long-term(15~30years)

Technology/Internet International Reputation Academic Excellence

24

Summary

• The proposal of the strategic planning model above is completely based on the 4 global rankings, so leading factors in the 3 categories are definitely relevant to the research outputs of an institution.

• Some of these indicators will take longer time to improve, such as Nobel Laureates and Academic peer review.

• If all institutions follow the model, it’s highly likely that not all of them will actually move into the ranking in the spots they expect to be.

• it is necessary to note that these are only guidelines and not meant to be used as a rigid cause and effects.

• Academics should not to rely on a single model only to implement in terms of educational policy.

25

Conclusion

• To achieve a good rankings is becoming more and more important

• Global rankings are increasingly being used as a tool for building world class universities

26

Final question raised by the paper

• To what extent can a world class university be replicated by using the factors highlighted in a ranking model and how can it be done?

• The answer is both yes it can be replicated and no it can’t be.

27

Yes and No Answers

• The model, based on statistical analysis, can only provide very rough guidance and clues to institutions on which road to take to achieve academic excellence.

• a clear vision, institutional features, favourable governance, and sufficient resources which were not taken into consideration in the above model (or in the 4 global rankings themselves) are all very crucial if a university is to rise and stay top in the rankings.

28

“THERE IS NO SINGLE ROAD TO EXCELLENCE”

by Jamil Salmi (2010)

29

Thank you for your attention

Question and Comments

Fu Jen Catholic University

Higher Education Evaluation & Accreditation Council of Taiwan