The role of national university

rankings in an international context

The case of the I-UGR Rankings for Spanish universities

Authors: N Robinson-Garcia, JG Moreno-Torres,

D Torres-Salinas, E Delgado López-Cózar, F Herrera

0. Rationale

University rankings have gained an undisputable position in

the research managers ‘toolkit’ for measuring the state of

health of higher education institutions

… however, they present many shortcomings that prevent

against a careless use…

Main limitations

I. Technical errors and difficulties on the recollection of reliable

and standardized data

II. Criteria for ranking universities not scientifically supported

III. Multidimensional nature of universities

IV. Neglecting the disciplinary focus of universities

V. International rankings do not reflect national university

systems

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Main limitations

I. Technical errors and difficulties on the recollection of reliable

and standardized data

II. Criteria for ranking universities not scientifically supported

III. Multidimensional nature of universities

IV. Neglecting the disciplinary focus of universities

V. International rankings do not reflect national university

systems

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

International rankings either include only world-class

universities, not covering national systems…

or

… ignore the muldisciplinary focus

of universities

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

1. Rankings I-UGR

Why a national ranking

Spain is poorly represented in rankings

-> 11 univs in the Shanghai Ranking – 16 in the Leiden Ranking <-

Existing rankings at the moment were unsuitable

-> Lack of continuity – disregard for disciplinary focus - … <-

~ First edition dates from 2010

~ Measures the research performance of universities -> Web

of Science -> Main database considered for research

policy

~ Offers 12 broad fields and 37 disciplines based on

aggregates from the TS subject categories

~ Offers a five years-publication window and a ten years-

publication window

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Characteristics

Quantitative dimension

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

The indicator -> IFQ2A- Index

Qualitative dimension

NDOC - Publications

NCIT – Citations

H – H-Index

%Q1 - Ratio of Q1 papers

ACIT – Avg cit/paper

TOPCIT– Ratio of 10% Highly

Cited Papers

𝑸𝑵𝑰𝑭 = 𝑵𝑫𝑶𝑪 × 𝑵𝑪𝑰𝑻 × 𝑯 𝑸𝑳𝑰𝑭 = %𝑸𝟏 × 𝑨𝑪𝑰𝑻 × 𝑻𝑶𝑷𝑪𝑰𝑻

The indicator -> IFQ2A- Index

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Can we combine national and international

rankings to offer a global picture of a

university system’s positioning?

2. Comparing rankings

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

World-Class universities

INTERNATIONAL RANKINGS

NATIONAL RANKINGS The rest of the universities

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Selection criteria

Based on the research dimension

Based on a reliable or consensed data source

Must offer rankings by fields

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Selected international rankings

Shanghai Ranking

QS Ranking

NTU Ranking

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Matching fields between rankings

SHANGHAI RANKING RANKINGS I-UGR RHO A

Natural Sci & Math Math ; Physics ; Chemistry * ; -0.5 ; 0.5 0/3 ; 3/3 ; 2/3

QS RANKING RANKINGS I-UGR RHO A

Chemical Engineering Chemical Engineering 0.84 4/7

NTU RANKING RANKINGS I-UGR RHO A

Computer Science Computer Science 0.75 8/13

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Matching fields between rankings

The Shanghai Ranking is the less consistent with the I-UGR

Rankings as only two fields have high correlations…

… while the NTU Ranking and the QS Ranking match better

with the I-UGR Rankings.

3. Conclusions

Can national rankings complement international

rankings?

National rankings are needed in state-oriented university

systems, as they provide an accurate picture on the state of

universities…

… international rankings are interesting for research policy

makers as they put universities into a global context…

0. Rationale 1. Rankings I-UGR 2. Comparing rankings 3. Conclusions

Main conclusions and demands

I. There are discrepancies between rankings due to:

a. Methodology

b. Construction of fields

TRANSPARENCY AND CLARITY ON THE METHODOLOGY AND

CONSTRUCTION OF FIELDS

II. Despite methodological differences, it is possible to use both

and combine them in a research policy context

Thank you!

The role of university rankings in an international context:

The case of the I-UGR Rankings for Spanish universities

Authors: N Robinson-Garcia, JG Moreno-Torres,

D Torres-Salinas, E Delgado López-Cózar, F Herrera

Nicolás Robinson García

elrobin@ugr.es

http://www.ugr.es/~elrobin

@nrobinsongarcia