RAE Briefingby

Keith van Rijsbergenat

CPHCon

19th April, 2007

RAE 2001/2008

• Two tier panel structure• Quality profiles• Outputs, not individual researchers are

rated.• Fairness continues

outputs/individuals

An underpinning principle is that sub-panelsshould assess each submission in the round:they will not make collective judgements aboutthe contributions of individual researchers, butabout a range of indicators relating to theunit, research group or departmentthat is put forward for assessment(Generic; §21)

Size (2001)

No. of submissions cross-refs• Pure Maths 47 3• Applied Maths 58 3• Stats 46 2• CS 80 6

Timeline

• June 2007: Consider survey of intentions...• January 2008: Allocate reading

responsibilities• March 2008: Identify cross-refs & ext. adv• May 2008: Audit cross-ref advice• June/July 2008: First assessment meeting• September 2008: Continue assessment• Oct/Nov 2008: Final assessment meeting• November 2008: Submit profiles to RAE• December 2008: Results Published

CAVEATS

• The published document RAE 01/2006 is the authoritative version.

• The content was arrived at by collective discussion and debate.

• The document contains compromises to achieve some compatibility across subpanels in F and to satisfy the lawyers in HEFCE+

Main Panel F(Nigel Hitchin)

• Pure Mathematics (20)• Applied Mathematics (21)• Stats and OR (22)• CS and Informatics*

• Ken Brown• Tim Pedley• Bernard Silverman• Keith van Rijsbergen

* To be referred to as CS henceforth

International Members

Who is on our sub-panel?

Alan BUNDY University of EdinburghAnthony FINKELSTEIN University College LondonStephen FURBER University of ManchesterW. Alex GRAY Cardiff UniversityWendy HALL University of SouthamptonAndrew HERBERT Microsoft Research LtdPeter JOHNSON University of BathCliff JONES University of Newcastle upon TyneRoss KING University of Wales, Aberystwyth Ursula MARTIN Queen Mary, University of LondonDerek MCAULEY Intel ResearchRon PERROTT Queens University BelfastAndrew PITTS University of CambridgeAlexandra POULOVASSILIS Birkbeck CollegeThomas RODDEN University of NottinghamMorris SLOMAN Imperial College LondonJoseph SVENTEK University of GlasgowChristopher TAYLOR University of ManchesterYorick WILKS University of Sheffield

Quality Levels

4* Quality that is world-leading in terms of originality, significance and rigour.

3* Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence

2* Quality that is recognised internationally in terms of originality, significance and rigour.

1* Quality that is recognised nationally in terms of originality, significance and rigour.

u/c

Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

In assessing excellence, the subpanelswill look for originality, innovation, significance,depth, rigour, influence on the disciplineand wider fields, and relevance tousers.(Main §15)

Excellence

CS: 4*

To be awarded a 4* quality levela research output must exhibithigh levels of originality, innovationand depth, and must have had, or inthe view of the sub-panel be likely tohave, a significant impact on thedevelopment of its field. (CS: §66)

Generic:Section 20

These descriptive accountsshould be read alongside, but do not replace, the standarddefinitions

OutputsAll forms.....including, but not necessarilylimited to: books, chapters in books, articlesin journals, conference contributions; creativemedia and multimedia; standards documents; patents, product or process specifications, itemsof software, software manuals; technical reports,including consultancy reports or independentevaluations. All forms will be given equalconsideration. (CS: §12)

Cross referrals &External advice

Advice from other subpanels will be soughtand given on the basis of the assessmentcriteria for the UOA to which the work wasorginally submitted (Generic: §54)

The advice of external advisers will beused to inform the subpanels’ assessment. (Main: §27)

‘Reading’

As far as possible, the sub-panel willexamine all research outputs, and expects to examine in detail at least25% of the outputs in each submission.The subpanel will use its professionaljudgement to select a subset of outputsto examine in detail (CS: §17)

Research Environment

• Infrastructure, facilities and administrative support for research

• Arrangements for developing and supporting staff in research

• Cumulative impact of research• Industrial collaboration, relationship

with research users,• contribution to public awareness and

understanding

Research Environment

• Academic collaboration, national and international, within discipline and interdisciplinary

• Research degrees awarded• Research income: funding strategy,

amount received and sustainability• Credibility, vitality and sustainability of

research organisation (CS: §27)

Esteem Indicators

• awards, fellowships of learned societies, prizes, honours and named lectures

• personal research awards and fellowships

• keynote and plenary addresses at conferences

• etc etc, see CS: §43

Citations

• No panel will use journal impact factors as a proxy measure for assessing quality (Generic: §32)

• They will not use a formal ranked list of outlets, nor impact factors, nor will they use citation indices in a formulaic way (Main §15)(CS: §19)

• But.....In reaching these judgements the assessor may consider:

citations of the output, relative to others of similar age in the same field (CS: §63)

Quality Profile

By percentage of research activity in the submission judged to reach quality standard

Sub-Panelx

Cat A FTE Staff submitted for assessment

4* 3* 2* 1* Unclassified

University A 50 15 25 40 15 5

University B 20 0 5 40 45 10

Research Outputs Research Environment Esteem Indicators

4* 3* 2* 1* u/c

20 30 15 20 15

4* 3* 2* 1* u/c

30 25 10 20 15

70%

Overall Quality Profile

15

4*

10203025

u/c1*2*3*

4* 3* 2* 1* u/c

10 25 40 15 10

20% 10%

is comprised of the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators.(Annex 1)Quality Level

% of Research Activity

The overall quality profile

Some Do’s and Don’ts

• Do read criteria and working methods carefully

• Do submit early career researchers• Don’t provide info on detailed

organisation and management structure of the department CS: §33

Take home message• Continue to produce high quality

research, papers, etc• Continue to enhance research

environment eg. grants, PhD’s, etc• Continue to attract awards, prizes,

keynotes, etc• All the stuff we normally do.