Recent developments Recent developments in rankings: in rankings:

implications for implications for developing countries?developing countries?

Jamil SalmiJamil Salmi

The World BankThe World Bank

IREG-3IREG-3

Shanghai, 29-30 October Shanghai, 29-30 October 20072007

The rankings businessA ranking of league tables

September 10, 2005

Rankers,Beware

League tablesunder scrutiny

Gordon’s Brown’s disastrous retreat

How do super-jumbo changes air travel

Meet Fred Thompson

Has commercial property peaked?

Liu Nian Cai , the man and the myth

outline of the outline of the presentationpresentation

• status of rankingsstatus of rankings

• new directions in new directions in accountabilityaccountability

• prospects for rankingprospects for ranking

who prepares the who prepares the rankings?rankings?

• A = government agency (Ministry of A = government agency (Ministry of Higher Education, Higher Education Higher Education, Higher Education Commission, University Grants Commission, University Grants Council, etc.)Council, etc.)

• B = independent organization / B = independent organization / professional association / universityprofessional association / university

• C = newspaper / magazine / mediaC = newspaper / magazine / media

• D = accreditation agencyD = accreditation agency

• I = International ranking (IA, IB, IC I = International ranking (IA, IB, IC and ID linking the international and ID linking the international dimension to the type of institution dimension to the type of institution conducting the ranking)conducting the ranking)

ranking systems in 2006

RegionRegionNational and International Ranking National and International Ranking SystemSystem

Eastern Europe and Central Eastern Europe and Central AsiaAsia

Poland (C), Slovakia (B), Russia (B), Ukraine (B)Poland (C), Slovakia (B), Russia (B), Ukraine (B)

East Asia and PacificEast Asia and PacificAustralia (B), China (B, IB), Hong Kong (C), Australia (B), China (B, IB), Hong Kong (C), Japan (C), New Zealand (A), Thailand (A)Japan (C), New Zealand (A), Thailand (A)

Latin America and the Latin America and the CaribbeanCaribbean

Argentina (D)Argentina (D)

Middle East and North Middle East and North AfricaAfrica

North AmericaNorth America Canada (C), United States (C)Canada (C), United States (C)

SSouth Asiaouth Asia India (D), Pakistan (A)India (D), Pakistan (A)

Sub-Saharan AfricaSub-Saharan Africa Nigeria (A)Nigeria (A)

Western EuropeWestern EuropeGermany (B/C), Italy (C), Netherlands (A), Germany (B/C), Italy (C), Netherlands (A), Spain (B), United Kingdom (A, B, IC)Spain (B), United Kingdom (A, B, IC)

ranking systems in 2007

RegionRegionNational and International Ranking National and International Ranking SystemSystem

Eastern Europe and Central Eastern Europe and Central AsiaAsia

Kazakhstan (A, B), Poland (C), Slovakia (B), Kazakhstan (A, B), Poland (C), Slovakia (B), Romania (B/C), Russia (B), Ukraine (B/C) Romania (B/C), Russia (B), Ukraine (B/C)

East Asia and PacificEast Asia and Pacific

Australia (B), China (B, C, IB), Hong Kong (C), Australia (B), China (B, C, IB), Hong Kong (C), Japan (B, C), Japan (B, C), Korea (A), Malaysia (A), New Korea (A), Malaysia (A), New Zealand (A), Taiwan (B), Thailand (A) Zealand (A), Taiwan (B), Thailand (A)

Latin America and the Latin America and the CaribbeanCaribbean

Argentina (D), Brazil (A), Chile (C,D), Mexico (B), Argentina (D), Brazil (A), Chile (C,D), Mexico (B), Peru (B)Peru (B)

Middle East and North Middle East and North AfricaAfrica

Tunisia (A)Tunisia (A)

North AmericaNorth America Canada (B, C, B/C), United States (C, IC) Canada (B, C, B/C), United States (C, IC)

SSouth Asiaouth Asia India (C/D), Pakistan (A)India (C/D), Pakistan (A)

Sub-Saharan AfricaSub-Saharan Africa Nigeria (A)Nigeria (A)

Western EuropeWestern Europe

Germany (B/C, C), Italy (C), Netherlands (A), Germany (B/C, C), Italy (C), Netherlands (A), Portugal (C), Spain (B, C, IC), Sweden (C), Portugal (C), Spain (B, C, IC), Sweden (C), Switzerland (B/C), United Kingdom (A, B, IC) Switzerland (B/C), United Kingdom (A, B, IC)

trendstrends

• more rankingsmore rankings– except Africa and Middle Eastexcept Africa and Middle East

• who does the rankings?who does the rankings?– less from the pressless from the press

– more from independent think tanks or more from independent think tanks or governmentsgovernments

• accepted in growing number of accepted in growing number of countriescountries

• voluntary participationvoluntary participation– Austrians, Swiss, FlemishAustrians, Swiss, Flemish

outline of the outline of the presentationpresentation

• status of rankingsstatus of rankings

• new directions in new directions in accountabilityaccountability

what does what does accountability accountability

mean?mean?

• information

•achievement of results

•sanctions

trends in trends in accountabilityaccountability

•multiple stakeholders

•multiple themes

•multiple instruments

multiple multiple stakeholdersstakeholders

•government(s)

•employers

•society at large

•professors

•students

multiple themesmultiple themes

•access

•equity

•quality

• relevance

•efficiency

•sustainability

•nation-building / values

multiple multiple instrumentsinstruments

• licensing

•evaluation / accreditation

•performance-based funding

• rankings

•assessment of learning outcomes

measurement of measurement of outcomesoutcomes

• From inputs to competencies

– Bologna process

– ABET

• Spellings Commission

– ”no graduates left behind”

• OECD / PISA for tertiary education

• World Bank benchmarking toolWorld Bank benchmarking tool

country experiencescountry experiences

• Australia

• Brazil

• Canada

• Colombia

• Jordan

• Korea

• Mexico

• UK

• USA

outline of the outline of the presentationpresentation

• status of rankingsstatus of rankings

• new directions in new directions in accountabilityaccountability

• prospects for rankingprospects for ranking

usefulness of usefulness of rankings?rankings?

• for the public?for the public?

• for the institutions?for the institutions?

• for the Government?for the Government?

usefulness of usefulness of rankings: for rankings: for individualsindividuals

•choice based on information

usefulness of usefulness of rankings: for rankings: for institutionsinstitutions

•benchmarking and self-evaluation

•counter-rankings (Paris)

•voluntary accountability framework (USA)

from the from the viewpoint of viewpoint of institutionsinstitutions

• sensitive to factors that affect sensitive to factors that affect their rankings (benchmarking)their rankings (benchmarking)

• goal setting for strategic goal setting for strategic planning purposesplanning purposes

• forming strategic partnershipsforming strategic partnerships

• mergersmergers

usefulness of usefulness of rankings: for rankings: for governmentsgovernments

• rapid QA assessment in lieu of accreditation

• international benchmarking

•national benchmarking

•culture of competition

government use of government use of rankingsrankings

• Pakistan casePakistan case

– promoting a culture of promoting a culture of accurate and transparent accurate and transparent informationinformation

– promoting a culture of promoting a culture of qualityquality

new challengesnew challenges

• multiplicity of institutions (and multiplicity of institutions (and joint programs)joint programs)

• multiplicity of missionsmultiplicity of missions

• blurring of boundaries between blurring of boundaries between disciplinesdisciplines

• emerging disciplinesemerging disciplines

• new delivery modesnew delivery modes

conclusion:conclusion: divisive or helpful? divisive or helpful?

For sale: charming and peaceful residence away from

neighbors, with a wonderful view of the sea, a grand period

staircase, and lots and lots of light…”

For sale: charming and peaceful residence away from neighbors, with a wonderful view of the sea, a grand period staircase, and lots and lots of light…”

BordeauxBordeauxthe wine producers the wine producers

suing the wine suing the wine rankersrankers

divisive or helpful?divisive or helpful?

• rankings are here to stay

• useful for prospective students

• useful in the absence of an established evaluation and/or accreditation system

• useful for benchmarking, goal-setting and self-improvement purposes

• useful to conduct a healthy debate on issues and challenges

• useful to promote a culture of accountability

linking rankings linking rankings and funding?and funding?

• Arizona StateArizona State

• RAE (UK, Hong Kong, NZ, Australia)RAE (UK, Hong Kong, NZ, Australia)

• Nigeria (based on ranking done by Accreditation Nigeria (based on ranking done by Accreditation Commission)Commission)

• performance premium for academics publishing in performance premium for academics publishing in journals included in Shanghai ranking (Norway, journals included in Shanghai ranking (Norway, Australia)Australia)

• donors and home ministries making scholarship donors and home ministries making scholarship decisionsdecisions

• corporate donorscorporate donors

• strengthening the strong and weakening the weakstrengthening the strong and weakening the weak

principles of principles of appropriateappropriate

ranking instruments ranking instruments

• compare similar institutionscompare similar institutions

• focus on program rather than focus on program rather than on entire institutionon entire institution

• rank by indicator rather than rank by indicator rather than wholesalewholesale

• outcomes/outputs/results outcomes/outputs/results rather than inputs rather than inputs

• better if used for self-better if used for self-improvement purposesimprovement purposes

• make results publicly availablemake results publicly available