Ranking Tertiary Education Systems

Jamil Salmi

Astana 15 June 2009

Benchmarking Tertiary Education Systems

Jamil Salmi

Astana 15 June 2009

3

4

the power of rankings

• public debate– Malaysia

– France

– Brazil

5

institutional behavior

• positive reactions– increased data-based decision making

– improvements in teaching and learning practices

– mergers or institutional collaboration

6

institutional behavior (II)

• negative reactions– less interest for low-income and minority

students

– distorted resource allocation to favor research over teaching

– neglect of established institutional research practices

– managing to the rankings (cheating?)

7

outline of the presentation

• world-class university or world-class tertiary education system?

• rankings of university systems

• benchmarking tertiary education systems

8

the search for excellence

my university is…

more world-class than yours

9

the WCU disease

• governments

• institutions– strategic efforts

– lobbying for resource concentration

10

11

the WCU disease

• governments

• institutions

• World Bank

13

how do you recognize a world-class university?

• everyone wants one

• no one knows what it is

• no one knows how to get onePhilip G. Altbach

14

15

reality check

• well-performing countries without world-class universities– and vice-versa

16

top 50 universities

comparison of rankings

WEF WB K4D SJTU

USA Denmark US (1)

Switzerland Sweden UK (4)

Denmark Finland Japan (19)

Sweden Netherlands Switzerland (24)

Singapore Norway Canada (24)

Finland Canada France (42)

Germany Switzerland Denmark (45)

Netherlands UK Netherlands (47)

Japan USA Sweden (51)

Canada Australia Germany (55)

17

18

reality check

• well-performing countries without world-class universities

• time dimension of alignment?– emerging economies vs. mature economies

• country size– Harvard vs. Canada

19

reality check

• institutional differentiation– different types of institutions for meeting

various learning and training needs

– regional engagement

– technology transfer through human capital formation

20

21

evolution of Nokia income

22

outline of the presentation

• world-class university or world-class tertiary education system?

• rankings of university systems

23

SJTU ranking (2008)Country Ranking Country

Rank of Top University in Country

1 USA 1 2 UK 4 3 Japan 19 4 Switzerland 24 4 Canada 24 6 France 42 7 Denmark 45 8 Netherlands 47 9 Sweden 51 10 Germany 55 11 Australia 59 12 Norway 64 13 Israel 65 14 Finland 68 15 Russia 70

16-20 Belgium, Brazil, Italy, Singapore

101 – 151

21-26 Argentina, Austria, Mexico, Spain, South Korea, Taiwan

152 – 200

27-33 China, Czech Republic, Greece. Hong Kong, Ireland, New Zealand, South Africa

201 – 302

34-36 Hungary, India, Poland 303 – 401 37-40 Chile, Portugal, Slovenia,

Turkey 402 – 503

adjusting for size

24

University World News

25

SJTU

26

QS SAFE National System Strength Ranking

• US• UK• Australia• Germany• Canada• Japan• France• Netherlands• South Korea • Sweden

• number & ranking• access to top U• flagship U• investment

27

28

Lisbon Council

29

• ranking of university systems (17 OECD countries)

• beyond research• multi-dimensional (access and

completion, labor market outcomes, lifelong learning, responsiveness, ability to attract foreign students)

• pioneer work

limitations

• face same methodological questions as university rankings– focus on universities – research bias– composite index with arbitrary weights– statistical robustness– choice of indicators

30

31

outline of the presentation

• world-class university or world-class tertiary education system?

• rankings of university systems

• benchmarking tertiary education systems

32

cross-country comparisons help increase knowledge

is this flower big or small?

it depends on the size of neighboring flowers

33

multi-dimension comparisons enriches the diagnosis

how does this flower compare with the others?

wider but shorter

purpose

• improving performance– through comparisons

• competitors

• good practices

– diagnosis (identification of areas for improvement)

– definition of specific corrective interventions

34

35

purpose (II)

• need to understand the determinants of performance– no consensus on what countries should do to

improve their performance

– wide variations in system performance with similar funding levels and common country characteristics

36

Brazil and Chile

• public spending on tertiary education = .8% and .3% of GDP respectively

• enrollment rates are 24% and 38% respectively

37

why is a comprehensive benchmarking tool important?

• performance is more than– building world-class universities

– enrolling students (equity and quality agenda)

• learning & labor market outcomes linked to the totality of the education experience

38

approach

• elaborating a theoretical framework

• selecting indicators

• finding / generating the data

• analysis– diagnosis (areas for improvement)

– identification of possible solutions

39

preliminary step

• defining a system– US

– EU

– small states

40

elaborating the theoretical framework

• distinction between performance and health of system– how good are the system’s actual outcomes?

– does it operate under conditions known to lead to high performance?

• definition of outcomes / outputs / results

• identification of determinants and causality relationships– informed by empirical evidence

41

results framework

• research production

• technology transfer

• quality and relevance of education and training– labor market outcomes

– measuring learning outcomes

42

results framework

• equity

• values and behaviors

• social and cultural engagement

determinants (inputs and processes)

• access and equity (enrollment, institutional diversification, pathways)

• quality and relevance (standards, teaching and learning, research, local engagement)

43

determinants (inputs and processes)

• sustainable financing (resource mobilization, allocation, efficient use)

• capacity to improve (system and institutional governance)

44

45

indicators and data

• selecting and defining the right indicators

• finding comparable and reliable data (statistical sources and surveys)– objective

– verifiable

46

47

Benchmarking

Tertiary Education

Upgrade your knowledge –

measure, assess and compare your universities!

48

conclusions

• exploratory work

• different type of analysis– who? governments and donors

– what: system lens vs. institutional

• multi-dimensional

49

main challenges ahead

• defining and measuring multiplicity of outputs

• finding reliable data

• linking results and causes to be able to take action