Methodology & Indonesias Performance World University
Rankings DGHE Indonesia Presented by: Ben Sowter, Head of Research
Date: 10 May 2008 Sponsored by Slide 2 Objectives Establish QS
background & credentials Explain research methodology New
developments Understand limitations of rankings Provide insight
into performance enhancement specific to improved ranking Collect
feedback and input Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 2 Slide 3 Session Flow Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 3 Slide 4 Session
Flow Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 4 Slide 5 Founded 1989/1990 Nunzio Quacquarelli
MBA Wharton MBA Career Guide Approximately 100 staff in 2008 Asia
Office in Singapore Asia Research Unit Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 5 Slide 6 QS
Mission to enable motivated people around the world to fulfill
their potential, by fostering international mobility, educational
achievement and career development Our Channels primary research
leading-edge editorial & publications developmental events web
solutions Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 6 Slide 7 www.topuniversities.com 7 Slide 8
www.topuniversities.com Over 1,180,000 visits in 2007 Over 6,000
visits per day in 2008 Increasingly strong prominence in Google
& Yahoo searches Ranked 42,376 in Alexa Home of the THE QS
World University Rankings Institution Profiles Detailed study
abroad information Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 8 Slide 9 Top Universities Guide New revised
and updated edition available now Contains much information not
available on web Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 9 Slide 10 Other QS Initiatives QS World Grad
School Tour QS Top Grad School Guide QS World MBA Tour QS Top MBA
Career Guide QS APPLE Korea July 2008 QS Top Apply QS Education
Trust QS Intelligence Unit Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 10 Slide 11 QS Offices & Events
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
11 Slide 12 Session Flow Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 12 Slide 13 Why rank universities?
Interest in ranking things and people Hospitals Schools Local
authorities Rich lists; Britain, world, Asian British, footballers
Universities: The Times / US News etc Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 13 Slide 14 Why
World Rankings? Higher education becoming more global Knowledge the
key driver of international competitiveness Increasing desire for
comparative information Unique positions of THE and QS as
international and independent experts in higher education Raises
awareness of all 500+ universities involved in the project
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
14 Slide 15 International Study Trends Worldwide, the demand for
international education is forecast to increase rapidly In 2003, an
estimated 3.1 million students studied internationally There are
likely to be in excess of 5.8 million international students in
2010 Two-thirds of this global demand will be Asians US domination
of international student recruitment has begun to falter tight
Visas, increased competition (IIE, NAFSA, CCE) Bologna Accord will
create an explosion in English-language based postgraduate courses
in Europe Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 15 Slide 16 Student Mobility on the Rise
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
16 Slide 17 International Study Trends Government funding of
tertiary education being cut (per capita) in most countries around
the world (IFC) Many governments targeting international students
as a source of tertiary sector funding Australia has estimated that
international students are more important to their economy than
manufacturing & mining The UK Government estimates that 270,000
students contribute $3 billion in fees & a further $3 billion
in other spending Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 17 Slide 18 International Study Trends By 2010
The European Commissions Bologna Accord will make room for over
500,000 first degree graduates to study in other EU nations for a
Masters degree By 2016 In the USA, recent Abraham Lincoln Study
Abroad legislation aims to encourage 1 million students to study
overseas By 2020 Over 3 million Asians are expected to study
outside their home country (Source: IDP British Council) Copyright
2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com) 18 Slide
19 World University Rankings can help differentiate the active,
from the dormantvolcanoes! Professor Shih, President National
University of Singapore Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 19 Slide 20 Session Flow Copyright 2008
QS Quacquarelli Symonds Limited (www.qsnetwork.com) 20 Slide 21
Academic Ranking of World Universities Operated by Shanghai Jiao
Tong University in China Focuses principally on research output
Nobel Prizes & Fields Medals Citations HiCis Strongly
correlated with educational reputation Historical dependency
Available on www.arwu.org www.arwu.org Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 21 Slide 22
Shanghai Jiao Tong Criteria CriteriaIndicatorWeight Quality of
Education Alumni of an institution winning Nobel Prizes and Fields
Medals 10% Quality of Faculty Staff of an institution winning Nobel
Prizes and Fields Medals 20% Highly cited researchers in 21 broad
subject categories 20% Research Output Articles published in Nature
and Science* 20% Articles in Science Citation Index-expanded,
Social Science Citation Index 20% Size of Institution Academic
performance with respect to the size of an institution 10%
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
22 Slide 23 Webometrics Ranking of World Universities Focuses on
web profile Research productivity using Google Scholar Search
Engine performance Site popularity Good practices in web management
Fully automated Tracks in the realm of 14,000 institutions Prone to
anomaly A university domain change can be disastrous Available on
www.webometrics.info www.webometrics.info Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 23 Slide 24
Webometrics Criteria IndicatorWeight Size Number of pages recovered
from four engines: Google, Yahoo, Live Search and Exalead. 20% Rich
Files After evaluation of relevance to academic and publication
activities and considering the volume of the different file
formats, the following were selected: Adobe Acrobat (.pdf), Adobe
PostScript (.ps), Microsoft Word (.doc) and Microsoft Powerpoint
(.ppt). These data were extracted using Google, Yahoo Search, Live
Search and Exalead. 15% Scholar Google Scholar provides the number
of papers and citations for each academic domain. These results
from the Scholar database represent papers, reports and other
academic items. 15% Visibility The total number of unique external
links received (inlinks) by a site can be only confidently obtained
from Yahoo Search, Live Search and Exalead. 50% Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 24 Slide 25 World
University Rankings Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 25 Slide 26 THE - QS World University Rankings
Began in 2004 Collaboration between Times Higher Education (THE)
& QS THE running since 1971 THE is a weekly magazine
distributed to academics in the UK and internationally Formerly
associated with The Times Published annually in the fall 26
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
Slide 27 Session Flow Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 27 Slide 28 Our Approach Four key
themes, as basis for comparison: Research Quality Graduate
Employability International Outlook Teaching Quality Copyright 2008
QS Quacquarelli Symonds Limited (www.qsnetwork.com) 28 Slide 29
Ranking Criteria & Weights CriteriaIndicatorBrief
DescriptionWeight* Research Quality Peer Review Composite score
drawn from peer review (which is divided into five subject areas).
3,703 responses. 40% Citations per Faculty Score based on research
performance factored against the size of the research body 20%
Graduate Employability Recruiter Review Score based on responses to
recruiter survey. 738 responses 10% International Outlook
International Faculty Score based on proportion of international
faculty 5% International Students Score based on proportion of
international students 5% Teaching Quality Student Faculty Score
based on student/faculty ratio 20% Copyright 2008 QS Quacquarelli
Symonds Limited (www.qsnetwork.com) 29 *All decisions regarding the
allocation of weightings are the responsibility of the Times Higher
Education Supplement Slide 30 Selection of Initial List Began in
2004 with list of the top 500 world institutions by research impact
Excludes single-faculty institutions Excludes postgraduate-only
institutions Added on recommendation from certain countries where
English publication culture is not strong (e.g. Germany) Each year
list is re-evaluated Omitted institutions are welcome to make a
case for inclusion with supporting evidence for their inclusion
relative to an included institution 566 universities considered in
2007 Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 30 Slide 31 Peer review is an effective way to
evaluate universities. It takes smart people to recognise smart
people Sir Richard Sykes Rector, Imperial College Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 31 Slide 32 Peer
Review of Research Output Peer review is the centrepiece of this
ranking 5,101 respondents International spread Weighted Results
Subject spread Arts & Humanities Engineering & IT Life
Sciences & Biomedicine Natural Sciences Social Sciences Overall
score built up from each faculty area Active academics 3 year
latest response Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 32 Slide 33 Peer Review Selection of Peers 3
year latest response Invitation to previous respondents World
Scientific Database (180,000) Singapore Headquarters
www.worldscientific.com www.worldscientific.com Mardev Database
(12,000+) Part of Reed Elsevier UK Headquarters www.mardev.com
www.mardev.com Academic Sign-up facility www.topuniversities.com
www.topuniversities.com Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 33 Slide 34 Observations: Peer Review
Harvard second to Berkeley Oxford and Cambridge Well-liked
universities all over the world Little evidence of patriotism bias
UK and North America dominate the top 20 Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 34 Slide 35
Recruiter Review Another group who have knowledge of university
quality Key relevance to graduate employability criteria Introduced
for 2005 rankings Sourced from QS database and media and university
referrals (over 120 lists received for 2007) Improved response for
2006 & 2007 Continuing development focus for 2008 Copyright
2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com) 35 Slide
36 Observations: Recruiter Review UK Strong but less so than in
previous years Some institutions with particular strengths do well
Bocconi, HEC Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 36 Slide 37 Quantitative Measures Research
Quality citations/faculty Teaching Quality faculty student ratio
International Outlook % international faculty and students Graduate
Employability ?? Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 37 Slide 38 Full Time Equivalent (FTE) All
personnel metrics based on FTE If no FTE data is available an
average is calculated based on national or regional norms FTE
numbers are a more accurate measure of an institutions commitment
to teaching and research Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 38 Slide 39 Observations: Staff/Student
Winner Caltech US, French, Swiss, Netherlands etc institutions all
well placed Both Asian and European amongst the well-placed Weak
correlation (0.21) with research but not zero FTE numbers have had
a profound effect on institutions with an atypical approach to
part-time or distance learning Copyright 2008 QS Quacquarelli
Symonds Limited (www.qsnetwork.com) 39 Slide 40 Observations:
International US shows up badly HKUST top in staff London School of
Economics top in students Caltech among few US with international
staff Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 40 Slide 41 Classic measure of research quality
Citations per staff member (not per paper) 5 year score (changed
from 10 in 2006 to increase topicality) Source: Scopus Citations
per Faculty Score 41 Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) Slide 42 Observation: Citations Medical faculty
is a big plus Or major biomedical income CalTech the winner, then
Stanford & MIT Big country effect Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 42 Slide 43
Z-Score Aggregation Method used in various domestic rankings such
as The Times (UK) Stabilise data year on year More accurate
application of weightings throughout the sample Smoothes any
anomalies that remain Essentially smoothes the curve for each
individual indicator Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 43 Slide 44 The Effect of Z-Score Application
on Rankings Indicators Copyright 2008 QS Quacquarelli Symonds
Limited (www.qsnetwork.com) 44 Slide 45 2007 Indicator Correlations
Peer Review 0.59 Recruiter Review Strong Peer Review 0.28
International Faculty Weak Peer Review 0.35 International Students
Moderate Peer Review 0.32 Student Faculty Moderate Peer Review 0.45
Faculty Citations Moderate Recruiter Review 0.33 International
Faculty Moderate Recruiter Review 0.37 International Students
Moderate Recruiter Review 0.27 Student Faculty Weak Recruiter
Review 0.12 Faculty Citations Weak International Faculty 0.66
International Students Strong International Faculty 0.15 Student
Faculty Weak International Faculty 0.15 Faculty Citations Weak
International Students 0.23 Student Faculty Weak International
Students 0.19 Faculty Citations Weak Student Faculty 0.21 Faculty
Citations Weak Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 45 Slide 46 Effects of 2007 developments
1.Online System More thorough and complete data all the way down
the list Less need for averages 2.Peer Review Countries with
disproportionately high response but a small number of institutions
lose unnatural advantage 3.FTE Data Universities with particularly
high or low numbers for part-time will be influenced 4.Scopus Less
pronounced bias towards US and English speaking world 5.Z-Score
aggregation Less advantage for the best institutions in lower
weighted indicators Increased stability year on year Copyright 2008
QS Quacquarelli Symonds Limited (www.qsnetwork.com) 46 Slide 47
World University Rankings - Issues? The ranking has limitations.
Lack of data on: Teaching quality Student satisfaction Investment
in infrastructure The ranking relies on the best comparable data
available Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 47 Slide 48 Session Flow Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 48 Slide 49
Comparing Results THE QS 1Harvard 2=Cambridge 2=Yale 2=Oxford
5Imperial 6Princeton 7=Caltech 7=Chicago 8UCL 10MIT SHANGHAI
1.Harvard 2.Stanford 3.Berkeley 4.Cambridge 5.MIT 6.Caltech
7.Columbia 8.Princeton 9.Chicago 10.Oxford Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 49 WEBOMETRICS
1.MIT 2.Stanford 3.Harvard 4.Penn State 5.Berkeley 6.Michigan
7.Wisconsin 8.Minnesota 9.Illinois 10.Cornell Slide 50 Key Findings
Harvard The US 57 in top 200 Harvards lead the smallest yet Oxford
and Cambridge also well placed But thats only part of the story
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
50 Slide 51 Key Findings The top 200 includes universities in 28
states US, UK, Canada, Australia, Netherlands Korea, China, Japan,
Singapore Thailand, Malaysia Continental Europe Developing world
small but improving (1 in 2004, 2 in 2005, only UNAM in 2006, 3 in
2007) Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 51 Slide 52 Where are the top universities? Top
20Top 50Top 100 Region/Country
200720062005200420072006200520042007200620052004 N. America 14
(11)(12)(11) 23 (25)(23) 43 (36)(35)(38) Europe 4 (5) 12 (12)(13)
35 (41)(37) Asia/Australia 3 (4)(3)(4) 15 (14) 22 (23)(28)(25) USA
13 (11)(12)(11) 20 (22)(20) 37 (33)(31)(35) UK 4 (4) 8 (8) 19
(15)(13)(14) Australia 1 (1) 6 (6) 8 (7)(12)(11) Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 52 Slide 53
Consistency of Results 2007200620052004 1Harvard 2Cambridge
MITBerkeley 3YaleOxfordCambridgeMIT 4OxfordMITOxfordCaltech
5ImperialYaleStanfordOxford 6PrincetonStanfordBerkeleyCambridge
7ChicagoCaltechYaleStanford 8CaltechBerkeleyCaltechYale
9UCLImperialPrinceton 10MITPrincetonEcole PolytechniqueETH Zurich
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
53 Slide 54 The Results 20072006NAMECOUNTRYSCORE 11HARVARD
UniversityUnited States100.0 2=2University of CAMBRIDGEUnited
Kingdom97.6 2=3University of OXFORDUnited Kingdom97.6 2=4=YALE
UniversityUnited States97.6 59Imperial College LONDONUnited
Kingdom97.5 610PRINCETON UniversityUnited States97.2 7=7CALIFORNIA
Institute of Technology (Caltech)United States96.5 7=11University
of CHICAGOUnited States96.5 925UCL (University College
LONDON)United Kingdom95.3 104=MASSACHUSETTS Institute of Technology
(MIT)United States94.6 1112COLUMBIA UniversityUnited States94.5
1221MCGILL UniversityCanada93.9 13 DUKE UniversityUnited States93.4
1426University of PENNSYLVANIAUnited States93.3 1523JOHNS HOPKINS
UniversityUnited States92.9 16 AUSTRALIAN National
UniversityAustralia91.6 1719=University of TOKYOJapan91.1
1833=University of HONG KONGHong Kong90.7 196STANFORD
UniversityUnited States90.6 20=35=CARNEGIE MELLON UniversityUnited
States90.0 Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 54 Slide 55 Top Asia Pacific* Schools RANK IN
REGION 2007 RANK 2006 RANK INSTITUTION NAMECOUNTRY PEER REVIEW
RECRUITER REVIEW FACULTY STUDENT CITATIONS PER FACULTY INT'L
FACULTY INT'L STUDENTS OVERALL 21719=University of
TOKYOJapan100929688254491.1 31833=University of HONG KONGHong
Kong959085791008990.7 42529=KYOTO UniversityJapan99898390292487.2
733=19=National University of SINGAPORESingapore100933484100 84.3
93614PEKING UniversityChina10098 53322684.2 1038=50=The CHINESE
University of Hong KongHong Kong837980 1008583.8 114028TSINGHUA
UniversityChina959210059203683.3 144670=OSAKA
UniversityJapan83758691172980.0 1651=63SEOUL National
UniversityKorea, South92548079162477.1 1753=58=HONG KONG University
of Science & TechnologyHong Kong848228921009676.9
206961=NANYANG Technological UniversitySingapore818237721009973.6
2185=116=FUDAN UniversityChina8796456831 71.6 2290=118TOKYO
Institute of TechnologyJapan67865991344270.5 23=102=108National
TAIWAN UniversityTaiwan86683979182468.0 23=102=168=TOHOKU
UniversityJapan53599684403268.0 25112=128=NAGOYA
UniversityJapan53748385253066.1 27125180NANJING
UniversityChina74724569591464.4 29132=198=KOREA Advanced Institute
of Science & TechnologyKorea, South65276485532863.7
30136128=KYUSHU UniversityJapan50688082173162.8 31149=154CITY
University of Hong KongHong Kong625137761005161.2
32151=133=HOKKAIDO UniversityJapan49697682192161.1
34155=165=University of Science and Technology of
CHINAChina75772876161160.9 35161=120KEIO
UniversityJapan52889145251659.9 36163=179SHANGHAI JIAO TONG
UniversityChina72923855351159.7 Copyright 2008 QS Quacquarelli
Symonds Limited (www.qsnetwork.com) 55 * Excludes Australia and New
Zealand Slide 56 Indonesia Results RANK IN REGION 2007 RANK2006
RANK INSTITUTION NAME PEER REVIEW RECRUITER REVIEW FACULTY STUDENT
CITATIONS PER FACULTY INT'L FACULTY INT'L STUDENTS OVERALL
90360=270= Universitas GADJAH MADA 6637282131638.3 94369=258=
BANDUNG Institute of Technology 5469323241137.8 99395250=
University of INDONESIA 4959352161234.9 147500+526= AIRLANGGA
University 1530761151226.4 170500+- BOGOR Agricultural University
1014512141117.6 176500+495= DIPONEGORO University 1814261121115.4
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
56 Slide 57 Average Positions for Asian Institutions in the Top 500
Copyright 2008 QS Quacquarelli Symonds Limited (www.qsnetwork.com)
57 Slide 58 Faculty Level Data Peer opinion Citations per paper Not
aggregated Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 58 Slide 59 Faculty Level: Top 5s Arts &
Humanities Harvard Berkeley Oxford Cambridge Yale Engineering &
IT MIT Berkeley Stanford Caltech Cambridge Life Sciences &
Biomedicine Harvard Cambridge Oxford Johns Hopkins Berkeley Natural
Sciences Berkeley MIT Cambridge Harvard Princeton Social Sciences
Harvard Berkeley LSE Yale Stanford Copyright 2008 QS Quacquarelli
Symonds Limited (www.qsnetwork.com) 59 Slide 60 Faculty Level:
Indonesia Arts & Humanities Gadjah Mada (121) U of Indonesia
(201) Bandung (242=) Engineering & IT Bandung (114) U of
Indonesia (264) Diponegoro (440) Life Sciences & Biomedicine
Gadjah Mada (108) Diponegoro (410=) Airlangga (410=) Natural
Sciences Bandung (135) Gadjah Mada (152) U of Indonesia (339)
Social Sciences U of Indonesia (150=) Bandung (322) Airlangga
(364=) Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 60 Slide 61 The Results Available at
www.topuniversities.com www.topuniversities.com Top 100 for the
following subject areas Arts & Humanities Engineering & IT
Life Sciences & BioMedicine Natural Sciences Social Sciences
Worlds oldest universities Year on year results Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 61 Slide 62
Session Flow Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 62 Slide 63 Online Database Submit a broad
range of quantitative and qualitative data (including those we need
for the rankings) about your institution using our online interface
http://schools.qsnetwork.com http://schools.qsnetwork.com Populate
and enrich your profile on www.topuniversities.com
www.topuniversities.com Educate visitors on other aspects of your
institution Copyright 2008 QS Quacquarelli Symonds Limited
(www.qsnetwork.com) 63 Slide 64 Recruiter Review Supply us with
lists of employer contacts for the recruiter review Exclusive
single use only Benefit your institution in recruiter review
Improve the overall strength of recruiter review Copyright 2008 QS
Quacquarelli Symonds Limited (www.qsnetwork.com) 64 Slide 65 Coming
Soon! Recruiter Review Breakdown Preview Access to Recruiter Survey
Academic Sign Up Facility Subject Focuses Personalised Rankings
Innovation Showcase Engagement Database Entry Requirements
Translator PhD Matching Engine Copyright 2008 QS Quacquarelli
Symonds Limited (www.qsnetwork.com) 65 Slide 66 Ben Sowter
[email protected] Presentation slides available on
www.topuniversities.com/professional/seminars/downloads
www.topuniversities.com/professional/seminars/downloads Thank
You
