THE RANKING EB - Digital.CSICdigital.csic.es/bitstream/10261/12923/1/Isidro_Aguillo.pdf W EBOMETRICS R ANKINGS 11 Published since 2004, inspired by the Shanghai Ranking Top 6,000 universities

www.webometrics.info

THE RANKING WEBNEW INDICATORS FOR NEWNEEDS

Isidro F. AguilloCybermetrics Lab. [email protected]

2nd International Workshop on University Web RankingsCCHS-CSIC, Madrid (Spain). April 21st 2009


2

AGENDA

Why Ranking? Why Web?

Comparison with other Rankings

Ranking Web

Future Developments

The Floor is Open


Spanish Research Council

Largest public

research body in Spain

130 institutes (mostly basic

science)

20% of the Spanish scientific output

2 Nobel Prizes

3

ABOUT US

Cybermetrics Lab Research

group at the Center of

Humanities and Social Sciences

Editors of ejournal

CybermetricsEU Projects

involvedStaff: 4

researchers, 1 technician


4

TO RANK OR NOT TO RANK

•Explosive international student mobility•Growing demand of information•Prior to 2003 there were no World Universities

RankingsGlobal market

•Trend towards increasing complexity•Not obvious results, multiple classifications,

confusing figures•Student oriented

Demise of national rankings

•Web publication reached World audiences•League tables are easy to understand•Unexpected results, rectors & politicians unaware

of the actual situation

Impact of Shanghai’s

ARWU

•Methodological problems•Mostly research based•Promoting ranking-improvement tactics

But …


5

STUDENT MOBILITY

Growth in internationalization of tertiary education (1975-2006)Source: Education at a Glance 2008: OECD

USA UK Australia Japan Germany Canada France TotalTotal from Asia 372173 152020 145338 122538 95829 75441 45780 1416263 of which, from Eastern Asia 204593 70809 62189 109594 36567 41049 21866 666024 of which, from South-central Asia 100842 35684 31548 3840 13729 19629 3462 351929 of which, from South-eastern Asia 36495 23681 43851 8691 7749 4392 6925 169195 of which, from Western Asia 30244 21846 3864 412 35532 7278 12741 215249Total from all countries, 2006 584817 330078 184710 130124 261363 148164 247510 2924679Total from all countries, 2000 475169 222936 105764 66607 187033 94401 137085 1894792

Countries of destinationCountries of origin


6

Indi

cato

rsa1. faculty/studenta2. fundinga3. papers publisheda4. citations receiveda5. degrees offereda6. prizesa7. external reviewsa8. web sizea9. intl. facultya10. foreign students…………a999. computers ratio

Wei

ghts Rank=w1*a1+w2*a2+w3*a3+…..+w999*a999

0≤wi≤1 ∑wi=1

A methodologically sound approachConsidering many indicators, not only outputs but also inputs basedApplied a multivariate non parametric analysis to estimate wi

HOW TO RANK?


7

FEASIBILITY PROBLEMS

Bibliometric biasesBiomedicine

Social & Humanities, TechnologyNorth America, Europe

Rest of the WorldEnglish

Other languages

Poor ability to discriminateRatios with low numbers Nobel Prizes

(few universities with 2 or more)

Data not comparable due to the lack of standardsProfessor/researcher/student/international student

No data available for most of the countriesEspecially in developing ones in Africa, Asia and Latin America


8

WHY THE WEB?The Web is already

the main tool for scholarly

communication

The Web could reflect all the missions of the

universities•Research•Teaching

•Third mission

Web visibility has proved its ranking

capabilities•Google PageRank

The Web provides easy access to

relevant indicators

The Web reaches and it is accessed

from every country of the World


9

THE RANKING WEBOriginal aim• Promote Open Access Initiatives• Target: “Web Publish or Perish”

“Political agenda”• Reaching developing countries• Including all the missions, not only research

Methodology• Search engines as intermediaries: Easy way to collect data• Link (quantitative) analysis as a proxy of quality assessment• Impact Factor based composite indicator

Consequences• Webometrics Rank as an indicator: High correlation with other

Rankings• Academic Digital Divide• Web (naming) bad practices: Taking the Web seriously


10

Activity (50%)

Virtual referendum

Size

Number of Webpages

20%

Impact (50%)

Rich files

Number of documents

15%

Scholar

Number of papers15%

Visibility

Number of external inlinks

AN ACADEMIC MODEL


WEBOMETRICS RANKINGS

11

Published since 2004, inspired by the Shanghai Ranking

Top 6,000 universities from a Directory with 17,000 entries

Top 2,000 research centers from a Directory with 7,000 entries

Two editions (January & July) per year

Only organizations worldwide with their own web domain or subdomain

Based on web indicators (activity and impact)


12


13

SUMMARYREGION / countries Top100 Top200 Top500 Top1000 Total

NORTH AMERICA 7 78 121 206 392 3552 USA 70 104 179 354 3343EUROPE 54 16 60 223 412 4531 United Kingdom 6 12 35 68 235 Germany 1 17 49 67 384 Italy 1 1 13 35 200 France 1 16 47 603 Spain 4 27 42 218ASIA 33 3 10 40 118 4303 Japan 2 3 10 38 670 China & HK 3 10 22 1156 Taiwan 1 2 10 23 158 Korea 1 4 12 402LATIN AMERICA 35 2 3 13 36 2891 Brazil 1 2 9 16 1529 Mexico 1 1 2 6 354OCEANIA 12 1 6 15 35 131 Australia 1 6 13 29 79ARAB WORLD 22 2 2 496AFRICA 37 1 5 321WORLD 200 16225


• MIT (1st region & world), Stanford (2nd) and Harvard (3rd) Universities• University of Toronto (24th world)

North America

• Cambridge (1st region), ETH Zurich (2nd) and Oxford (3rd) UniversitiesEurope

• Tokyo (52nd), Taiwan National (55th) and Peking (117th) UniversitiesAsia

• National University of Australia (48th)Oceania

• Universidad Nacional Autónoma de México (44th)• Universidade de Sao Paulo (87th)Latin America

• University of Cape City (359th) Africa

WR (Enero 2009)

EXPECTED RESULTS

14


MORE RESULTS

15

• Universities of Catalonia (.es, .edu., .net, .cat)• Old and new domains (Imperial, Illinois, Victoria, Northwestern, Case Western, Pontificia de Chile, …)

Bad naming practices

• Jussieu campus (Paris 6, Paris 7, CNRS)• Strasbourg Universities (3 under common domain)• University of Helsinki (shared with the City Hall)

Common domain

• Johns Hopkins University • Universidad de la República (Uruguay)• University of Zagreb• Warsaw University

Splitteddomain

• Websites mostly in local language• (Few pages in English)

Japanese, French, Italian

• Against Biomedical small size organizations• Favoring Technology oriented and Distance Learning based universities

Biases


COMPARING WORLD RANKINGS

16

Univs analysedUnivs rankedTeaching Alumni Nobel 10% Students/Staff 20%

% Foreign Students 5%% Foreign Professors 5%

Size Webpages 20% University 10%Documents 15% Nature&Science 20%Papers GS 15% Papers S&SSCI 20%

Highly Cited ResCit High IF JournalsCitations S&SSCI 30%

Academic Survey 40%Employers Survey 10%

Citations Scopus 20%

Prestige Faculty Nobel 20%

Internatio- nalisation

Research Papers S&SSCI 20%

Impact Links 50%Highly CitedResearchers

20%50%

17000 3000 3500 20006000 500 500 500+

CRITERIAWEBOMETRICS ARWU HEEACT QS-THES

Spain (research lab) China (university) Taiwan (gov agency) United Kingdom (firm)


17

ADVANTAGES & DISADVANTAGES

ARWU

•Focus on excellence

•Historical achievements

•Estability

•Only research

•Bibliometric biases

•Discriminatory capability

•Prizes assignation

•University Hospitals

QS-THES

•Perceived prestige

•Long term performance

•Internationalization

•Representation bias of the survey

•Size related

•Not stable

•Ties (beyond rank 200)

•Private company

HEEACT

•Recent performance

•Only research

•Normalization problems

•Size related problems

•No crown indicator

WEBOMETRICS

•Developing countries

•Other missions than research

•Easy to build

•Sensible to URL bad practices

•Google Scholar data quality

•Technology biased

•Impact of Open Access initiatives


Country WEBR09 ARWU08 HEEACT08 QSTHES08

USA 104 90 91 58United Kingdom 12 22 19 30Germany 17 14* 18 11Canada 17 6 9 12Netherlands 6 9 9 10Japan 3 9 9 10Australia 6 6 6 9Switzerland 3 6 7 7France 1 7 4 4Sweden 4 4 4 4Italy 1 5 7 1Israel 1 4 3 3Belgium 1 4 2 5Denmark 1 3 2 3China & HK 3 0 3 10Korea 1 1 1 3Singapore 1 1 1 2Taiwan 2 1 1 1India 0 0 0 2Thailand 0 0 0 1

COMPARING WORLD RANKINGS (TOP 200)

18


19

ACADEMIC DIGITAL DIVIDE


20

MOSTLY NATIONAL LINKS


21

IMPROVING THE RANK (WEB)

Size

Unique Domain

More Research

Distance Learning

Visibility

English

UniversitiesBacklinks

Regional!

Rich Files & Scholar

Personal Pages

Repository

Hosting Third Parties


22

IMPROVING THE RANK (MISSIONS)

Teaching

More Technology

Web 2.0

Distance Learning

Research

International Papers

International Journals

Transfer

Community Engagement

Local culture, society, economy


23

FUTURE DEVELOPMENTS

Reinforcing Ranking Mission

Facing bad practices

•Deleting entries•Combining

domains•Excluding super-

sites

New visibility indicator

•Adapted G-factor

Cooperation with Scimago

World Ranking

More info New regions

Historical data

Discipline Rankings

By Department

JULY 2009

2010..2012


24

LESSONS

The Web is a more universal, democratic, cheaper and powerful communication tool and Web indicators could be

used for wider, fairer and feasible evaluation purposes

The Web Ranking provides useful information for developed and developing countries universities that it is not Web

specific, measuring overall performance and global impact

University web sites should reflect the full set of activities and all the results produced by the institution and help to

their dissemination to improve visibility, prestige and attracting new students, excellent scholars and big funding

Research based Rankings are measuring excellence, but not every University in the World is research intensive and focusing on these results could drive to scientific and

cultural colonialism


25

THANK YOU!QUESTIONS?

ISIDRO F. AGUILLOCYBERMETRICS LABCCHS-CSICALBASANZ, 26-2828037 MADRIDSPAINE-MAIL: [email protected]

WWW.WEBOMETRICS.INFO

mailto:[email protected]�







