The UK Experience ofQuality Assurance in Research and Doctoral Education

Dr Robin HumphreyDirector of Research Postgraduate TrainingFaculty of Humanities and Social Sciences

Newcastle University, UK

EUROPOS SĄJUNGA

Europos socialinis fondas

MYKOLO ROMERIO

UNIVERSITETAS

Who am I?

Active researcher Have a PhD! PhD Supervisor and Examiner PGR Director in Sociology, then GPS Director of PG Training – Social Sciences ,

and then Arts and Humanities too EUA Doctoral Project, Coordinator Bologna Process, Brussels and Nice

Lessons from International Work UK Most experience with QAA and Training issues But, much to learn from others (Progression and

Examination) However, Beware best practice from other contexts! Embedded in Institutions, and developed in own

cultural and political national and local contexts So, Learn from others, but adapt to own

circumstances

UK Context: What is a Traditional UK PhD?

Three years of Registration One or two supervisors 80,000 – 100,000 word thesis Examined by thesis And a viva Internal and external examiner

National Framework

Research Assessment Exercise (RAE)

QAA (Code of Practice)

Research Councils (Scholarships and Training)

What is the Research Assessment Exercise (RAE)?

Run by Higher Education Funding Council for England, to assess Universities’ research quality on basis of peer review of academic impact

Started in 1986 – occurs every few years – previous RAE in 2001

Details on website www.rae.ac.uk

Why is RAE important?

Direct impact on University funding

Long term impact on funding allocation for research (still funded on basis of 2001 results!)

Dual support system for research Quality of Research (QR) funding from HEFCE Project funding from sponsors

QR funding allocated on basis of RAE performance

Why is RAE important?

Prestige of good RAE result enables University and individual subject areas:

To recruit and retain top quality staff To develop our reputation nationally and

internationally as research institution To enhance student recruitment, esp. PG To encourage funders to invest in Newcastle

research

RAE2008 - How it works

Each area of research – ‘Unit of Assessment’ (UoA) – makes a separate submission to its RAE panel

RAE2008 has 67 individual panels to which submissions are made, grouped in 15 Main Panels

Panels include experts in the field, and international observers

Assessment based on peer review

RAE2008

Each UoA’s submission includes:

Research-active staff details (RA1)(NB not all staff are entered)

Publications & research outputs (RA2) Research students & studentships (RA3) Research income (RA4) Description of UoA, including research

environment and esteem indicators (RA5)

RAE2008

On basis of this information, panels allocate scores for: Outputs Environment Esteem

This leads to an overall quality profile for each UoA NB weighting between 3 elements different in each panel

Quality Profile built up from 3 elements

70%

20%

10%Research Outputs

Research Environment

Esteem Indicators

Research Outputs (70%)

4 outputs (journal articles) per submitted staff member

Must be published between 1 January 2001 and 31 December 2007

Early Career Researchers (appointed after 1 August, 2003) may submit fewer

All cited outputs will be reviewed individually (all abstracts will be read, plus a % of full papers)

Grades 1* - 4*

1* - research judged to be original, significant and rigorous, which has made or is likely to make a contribution to knowledge, theory, practice or policy.

2* - research judged to be of a quality to be at an international standard of quality in terms of originality, significance and rigour; which has made or is likely to make a significant contribution to knowledge, theory, practice or policy.

Grades 1* - 4* (Continued)

3* - research judged to be of a quality that is internationally excellent in terms of originality, significance and rigour; which has made or is likely to make a highly significant contribution to knowledge, theory, practice or policy.

4* - research judged to be of a quality which is world-leading in terms of originality, significance and rigour; regarded as a primary point of reference in its field, which has made or is likely to make an outstanding contribution to knowledge, theory, practice or policy.

Research Environment (20%)

Research Students and Studentships Research Income Sources of Research Income Research Structure Staffing Policy Research Strategy

Esteem Indicators (10%)

Keynote conference papers Impact on government policy Impact on practice Honours and awards Chairmanship of research committees Service on advisory bodies Editorial activities International collaboration

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

eg 70%

Overall quality profile

20

4*

10153025

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

eg 20% eg 10%

The overall quality profilecomprises the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators.

(Minimum 50%)

Quality Level

% of Research Activity

(Minimum 5%)(Minimum 5%)

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

eg 70%

Overall quality profile

20

4*

10153025

u/c1*2*3*

20

4*

10153025

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

eg 20% eg 10%

The overall quality profilecomprises the aggregate of the weighted profiles produced for research outputs, research environment and esteem indicators.

(Minimum 50%)

Quality level

% of research activity

(Minimum 5%)(Minimum 5%)

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

eg 70%

Overall quality profile

20

4*

10153025

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

eg 20% eg 10%

The overall quality profilecomprises the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators.

(Minimum 50%)

Quality Level

% of Research Activity

(Minimum 5%)(Minimum 5%)

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

eg 70%

Overall quality profile

20

4*

10153025

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

eg 20% eg 10%

The overall quality profilecomprises the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators.

(Minimum 50%)

Quality Level

% of Research Activity

(Minimum 5%)(Minimum 5%)

Research outputs Research environment Esteem indicators

25

4*

15201525

u/c1*2*3*

25

4*

15201525

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

30

4*

20201020

u/c1*2*3*

eg 70%

Overall quality profile

20

4*

10153025

u/c1*2*3*

20

4*

10153025

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

15

4*

10153525

u/c1*2*3*

eg 20% eg 10%

The overall quality profilecomprises the aggregate of the weighted profiles produced for research outputs, research environment and esteem indicators.

(Minimum 50%)

Quality level

% of research activity

(Minimum 5%)(Minimum 5%)

Doctoral Students in the RAE

Explicit in Research environment

Can’t be a Research Univeristy with Doctoral Students

If top third of grade, then receive income from HEFCE for each doctoral student.

Quality in Doctoral Education

The Quality Assurance Agency (QAA)

Quality Assurance Agency (QAA)

http://www.qaa.ac.uk/

‘safeguard and help to improve the academic standards and quality of higher education in the UK ‘

QAA’s Code of Practice for Research Degrees

Government focus on improving quality of research degrees;

QAA producing a country-wide Code of Practice for all Universities (see www.qaa.ac.uk for latest details);

Will be included in QAA audit in future;

QAA’s Code of Practice: General PointsGeneral points: Emphasis on the student experience

A list of precepts to apply to research students through entire University ‘life’ - from recruitment to award of degree, ensuring equity for all students;

Producing a document that: is usable by and useful to diverse institutions covers the majority of research students can be applied to a wide range of research qualifications

For use in Institutional Audit

Nature of Institutional Audit (QAA slide) Peer review process Based on self-evaluation Joint agenda – institution / QAA Question re. Research Postgraduates:

How to include PGR activities in internal quality assurance processes that will show auditors institution is maintaining academic standards and delivering a high quality student experience?

QAA’s Code of PracticeKey Features

Requirements for institutional arrangements Quality of the research environment, including critical

mass of students and staff, and adequate resources Selection, admission and induction Quality, frequency and range of supervision and

support Need for recognition and development of research and

other skills Explicit guidance for the conduct of vivas and clear,

fair assessment methods