EFFICIENCY in English higher education

Triple Accredited | World ranked

EFFICIENCY IN ENGLISH HIGHER EDUCATION

Jill JohnesDepartment of Economics, Lancaster University UK

http://www.mbaworld.com/

http://www.efmd.org/




Outline of talk

1. IntroductionMain developments in English higher educationComposition of the English higher education sector today

2. Efficiency and its measurementa) Stochastic frontier analysis (SFA) b) Data envelopment analysis (DEA)

3. An empirical study

4. Policy issuesa) Comparison of various efficiency measures and rankingsb) Mergers

5. Conclusions






1. IntroductionTimeline: English higher education (HE) from 1963

• 1963: Robbins Report led to expansion of HE in England• 1986: First Research Assessment Exercise (RAE)• 1992: Abolition of divide between universities and polytechnics• 1997: Quality Assurance Agency (QAA) established to assess HE teaching• 1998: Undergraduate (UG) tuition fees set at £1000• 1999: Tony Blair aims for 50% of young people in HE by 2010• 2003: Colleges of Higher Education allowed to apply for university status• 2006: Introduction of variable UG tuition fees with ceiling of £3000• 2008: Research Assessment Exercise• 2012: Ceiling on variable UG tuition fees raised to £9000• 2013: Research Excellence Framework






1. IntroductionComposition of the English higher education sector

• The current English higher education sector therefore comprises very diverse groups of HEIs:

Pre-1992 universitiesTraditional HEIs including Oxford and Cambridge, and universities established in the 1960s; they offer traditional programmes and subjects and have a research mission

Post-1992 universitiesFormer polytechnics which offer a range of programmes including vocational degrees; some also have a research mission

Former colleges of HEOften small, specialist HEIs; often do not have a research mission; allowed to obtain university status since 2004






2. Efficiency and its measurementWhy measure the efficiency of English higher education?

• Assessing the performance or efficiency of higher education institutions (HEIs) is of interest to: - potential students and their parents - HEI managers- government (since HEIs receive public funding)






2. Efficiency and its measurementOfficial performance indicators

• The Higher Education Statistics Agency (HESA) has published performance indicators since 1999

‘Performance indicators in higher education (HE) provide information on the nature and performance of the HE sector in the UK. They are intended as an objective and consistent set of measures of how a higher education institution is performing.’

The indicators currently cover the following data:•widening participation indicators•non-continuation rates•module completion rates•research output•employment of graduates






2. Efficiency and its measurementOfficial performance indicators

• Research Excellence Framework – REF (formerly Research Assessment Exercise – RAE)- evaluation of research quality in British Universities. The research of each subject is ranked by a peer review panel and the rankings are used in the allocation of government funding.

• Quality Assurance for Higher Education (QAA) assesses undergraduate teaching.






2. Efficiency and its measurementUnofficial performance indicators

• Newspapers regularly provide rankings of higher education institutions (HEIs). For example- The Times Good University Guide- RIA Novosti/HSE ranking of Russian Universities- Forbes ranking of Russian Universities- Times Higher Education World University Rankings- The QS World University Rankings






2. Efficiency and its measurementThe Times Good University Guide: 8 measures• The universities are compared on ‘8 well-established measures of

importance to the undergraduate experience’.1. Student satisfaction 2. Research quality3. Entry standards4. Student-staff ratio5. Services and facilities spending6. Completion7. Good honours8. Graduate prospects






2. Efficiency and its measurementProblems with these rankings

• HEIs operate under different conditions and in different environments.

• HEIs produce an array of outputs from a range of inputs. A ratio of one output to one input is, at best, only a partial indicator.

• What is the appropriate way to combine information across a variety of indicators?

• We need a method for measuring efficiency which takes into account the multi-dimensional nature of production in HEIs.






2. Efficiency and its measurementA multi-dimensional measure of efficiency

Inputs

‘Raw materials’

Labour

Capital

Black Box

Outputs

Teaching

Research

Social







Teaching/staff

Research/staff

Production possibility frontier (PPF)

O







Teaching/staff

Research/staff

• F

• F’

O

Efficiency = OF/OF’






2. Efficiency and its measurementOrdinary least squares regression

Teaching/staff

Research/staff

• •

•

• •

•

•

•

• •

•

•

• •

•

• F

• F’

O

Estimated efficiency = OF/OF’

Estimated inefficiency = FF’/OF’ i.e. based on the OLS residual






2. Efficiency and its measurementStochastic frontier analysis (SFA)

Aigner, Lovell and Schmidt (1977)

• The error term is split into 2 components:- a random error component (as in OLS)- a half-normally distributed component to reflect efficiency






2. Efficiency and its measurementStochastic frontier analysis

Teaching/staff

Research/staff

• •

•

• •

•

•

•

• •

•

•

• •

•

• F

• F’

O






2. Efficiency and its measurement Data envelopment analysis (DEA)

Charnes, Cooper & Rhodes(1978)

• DEA estimates a piecewise linear frontier using linear programming methods

• No error term; no equation






2. Efficiency and its measurementData envelopment analysis

Teaching/staff

Research/staff

• •

•

• •

•

•

•

• •

•

•

• •

•

• F

• F’

O






3. An empirical studyInputs and outputs

InputsPrimary inputs:

PGINPUT (x1): Numbers on postgraduate programmes

UGINQUAL (x2): Numbers on undergraduate programmes

weighted by average A level score (UGINPUT*QUAL)

Labour:STAFF (x3): Number of FTE

academic staffADMIN (x5): Expenditure on administration including staff

Capital:ACSERV (x4): Expenditure on library and computing facilities

Black Box

OutputsTeaching:

PGOUTPUT (y1): Graduates from postgraduate programmes

UGOUTQUAL (y3): Graduates from undergraduate programmes

weighted by degree result (UGOUTPUT*DEGREE)

Research:RESEARCH (y2): Income received in funding council grants plus income

received in research grants and contracts






3. An empirical studyEstimation methods

• a) SFA Time Invariant (TI): Efficiency for each university does not vary over time

• b) SFA Time varying (TV): Efficiency for each university does vary over time

• a) DEA Constant Returns to Scale (CRS)- i) Estimation across the pooled data set- ii) Estimation within each year

• b) DEA Variable Returns to Scale (VRS)- i) Estimation across the pooled data set- ii) Estimation within each year






3. An empirical studyData

• Higher Education Statistical Agency (HESA) data• Unbalanced panel of data from 1996/97 to 2008/09 with n = 1444

(the number of HEIs varies from 108 to 113 in each year) • Diverse groups of HEIs:

Pre-1992 universities n=624Post-1992 universities n=375Former colleges of HE n=445

• All money units in 2008 values






3. An empirical studyOutputs

1996

/97

1998

/99

2000

/01

2002

/03

2004

/05

2006

/07

2008

/090

5001000150020002500300035004000

0100002000030000400005000060000700008000090000

PGOUTPUTUGOUTPUTUGOUTQUALRESEARCH

Year

Num

ber

£ in

200

8 pr

ices






3. An empirical studyInputs

1996

/97

1998

/99

2000

/01

2002

/03

2004

/05

2006

/070

100020003000400050006000700080009000

10000

0

5000

10000

15000

20000

25000

PGINPUTUGINPUTSTAFFUGINQUALACSERVADMIN

Year

Num

ber

£ in

200

8 pr

ices






3. An empirical studyInputs by HEI Type

Post-1992 HEIs Pre-1992 HEIs Former Colleges of HE0

200

400

600

800

1000

1200

1400

1600

0

500

1000

1500

2000

2500

PGINPUT UGINPUT UGINQUAL STAFF ACSERV ADMIN

Num

bers

£ in

200

8 pr

ices






3. An empirical study Outputs by HEI Type

Post-1992 HEIs Pre-1992 HEIs Former Colleges of HE0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

0

20000

40000

60000

80000

100000

120000

PGOUTPUT UGOUTPUT UGOUTQUAL RESEARCH

Num

bers

£ in

200

8 pr

ices






3. An empirical studyEfficiencies

SFA Mean SD Min Max

a) TI 0.803 0.097 0.515 0.987

b) TV 0.801 0.097 0.479 0.990

DEA

a) CRS i) pooled 0.753 0.090 0.341 1.000

a) CRS ii) within year 0.869 0.083 0.422 1.000

b) VRS i) pooled 0.827 0.089 0.429 1.000

b) VRS ii) within year 0.932 0.067 0.547 1.000






3. An empirical studyMean efficiencies over time

1996/97

1997/98

1998/99

1999/00

2000/01

2001/02

2002/03

2003/04

2004/05

2005/06

2006/07

2007/08

2008/09

0.600000000000001

0.650000000000001

0.700000000000001

0.750000000000001

0.800000000000001

0.850000000000001

0.900000000000001

0.950000000000001

1

SFA TVCRS pooledVRS pooledCRS within yearVRS within year






3. An empirical studyMean efficiencies by HE type

Post-92 Pre-92 Former Colleges of HE0.660

0.680

0.700

0.720

0.740

0.760

0.780

0.800

0.820

0.840

0.860

SFA TVCRS pooledVRS pooled






4. Policy issues Are these efficiency measures correlated?

SFA DEA

Model a) TI b) TV a) CRSi) pooled

a) CRSii) within year

b) VRSi) pooled

SFA b) TV 0.987

DEA

a) CRS i) pooled 0.422 0.466

a) CRS ii) within year 0.268 0.296 0.712

b) VRS i) pooled 0.485 0.494 0.767 0.634

b) VRS ii) within year 0.306 0.332 0.583 0.745 0.657






4. Policy issues Are these efficiency measures correlated with other measures?

Model VC Pay

SFA

a) TI 0.046

b) TV -0.002

DEA

a) CRS i) pooled -0.059

a) CRS ii) within year -0.008

b) VRS i) pooled *0.192

b) VRS ii) within year -0.026

Vice Chancellor Pay






4. Policy issues Are our efficiency measures correlated with other measures?

Model VC Pay Times

SFA

a) TI 0.046 -0.029

b) TV -0.002 -0.024

DEA

a) CRS i) pooled -0.059 *-0.248

a) CRS ii) within year -0.008 -0.163

b) VRS i) pooled *0.192 -0.048

b) VRS ii) within year -0.026 -0.100

Times Good University Guide

Correlation between VC Pay and Times is 0.205






4. Policy issuesMerger activity

Pre-merger Post-merger Non-merging

N=142 N=133 N=1169SFA a)TI 0.797 0.890 0.793

b) TV 0.806 0.882 0.791DEA a) CRS i) pooled 0.764 0.780 0.748

a) CRS ii) within year 0.873 0.899 0.865

b) VRS i) pooled 0.833 0.881 0.820

b) VRS ii ) within year 0.943 0.954 0.929






4. Policy issuesOutputs of pre-, post- and non-merging HEIs

PGOUTPUT UGOUTPUT UGOUTQUAL RESEARCH0

500

1000

1500

2000

2500

3000

3500

4000

4500

Pre-merging HEIs

Post-merging HEIs

Non-merging HEIs






4. Policy issuesInputs of pre-, post- and non-merging HEIs

PGINPUT UGINPUT UGINQUAL STAFF ACSERV ADMIN0

5000

10000

15000

20000

25000

30000

35000

Pre-merging HEIsPost-merging HEIsNon-merging HEIs






5. Conclusions• Ratios which claim to measure efficiency or performance of

universities (such as those found in media rankings) should be treated with caution as they do not adequately reflect the university context

• DEA and SFA can provide multi-dimensional measures of efficiency• DEA and SFA provide different estimates of the levels of efficiency in

English higher education• Policy-makers should be wary of using the DEA within-year estimation

approach• Rankings of DEA and SFA efficiency scores are significantly positively

related, but correlations are low• Merging HEIs appear to have higher efficiency than non-merging HEIs• Further analysis is necessary to discover whether this is caused by the

merger or other characteristics





Documents

EFFICIENCY in English higher education