HEFCE Impact

8/6/2019 HEFCE Impact

http://slidepdf.com/reader/full/hefce-impact 1/8

Research Excellence Framework

Decisionson assessingresearch

impact

This document sets out the UK

higher education funding bodies’

decisions on the broad framework

for the assessment of impact in the

Research Excellence Framework,

and the weightings of the three

elements: outputs, impact andenvironment.

REF 01.2011

March 2011

This document is for information.



REF 01.2011 1

Executivesummary

Purpose

1. ThisdocumentsetsouttheUKhighereducationfundingbodies’decisionsinrelationtothefollowing

aspectsofthe2014ResearchExcellenceFramework

(REF):

• thebroadframeworkforassessingtheimpactof

researchinthe2014REF

• theweightingofthethreeelementsinthe

assessment(outputs,impactandenvironment).

Keypoints

2. Followingtheconclusionoftheimpactpilot

exercise,thefourUKfundingbodieshavedecidedthat:

a. IntheREFtherewillbeanexplicitelement

toassessthe‘impact’arisingfromexcellent

research,alongsidethe‘outputs’and

‘environment’elements.

b. Theassessmentofimpactwillbebasedonexpert

reviewofcasestudiessubmittedbyhigher

educationinstitutions.Casestudiesmayinclude

anysocial,economicorculturalimpactorbenet

beyondacademiathathastakenplaceduring

theassessmentperiod,andwasunderpinned byexcellentresearchproducedbythe

submittinginstitutionwithinagiventimeframe.

Submissionswillalsoincludeinformationabout

howtheunithassupportedandenabledimpact

duringtheassessmentperiod.

c. Aweightingof25percentforimpactwould

giveduerecognitiontotheeconomicandsocial

benetsofexcellentresearch.However,given

thattheimpactassessmentinthe2014REFwill

stillbedevelopmental,theweightingofimpact

intherstexercisewillbereducedto20percent,withtheintentionofincreasingthisin

subsequentexercises.

To

Heads of publicly funded higher education

institutions in the UK

Organisations with an interest in commissioning

or using academic research including businesses,

public sector bodies, charities and other third-

sector organisations

Ofinteresttothoseresponsiblefor

Research

Reference

REF01.2011

Publicationdate

March 2011

Enquiriesto

Rebecca Gordge

tel 0117 931 7477

e-mail [email protected]

Decisions on assessingresearch impact



2 REF 01.2011

d. Theassessmentofresearchoutputswillaccountfor65percent,andenvironmentwillaccountfor

15percent,oftheoverallassessmentoutcomesin

the2014REF.Theseweightingswillapplytoall

unitsofassessment.

Actionrequired

3. Thisdocumentisforinformation.Noactionis

requiredbyhighereducationinstitutions.Further

detailedguidancetoinstitutionsonpreparingtheir

submissionsfortheREFwillbepublishedinJuly

2011.



REF 01.2011 3

IntroductionBackground

4. InNovember2009thefourUKfundingbodies

published‘ResearchExcellenceFramework:Second

consultationontheassessmentandfundingof

research’(HEFCE2009/38).Theconsultationset

outproposalsforallkeyaspectsoftheResearch

ExcellenceFramework(REF)assessment,including

theintroductionofanexplicitelementtoassessthe

impactofresearch.Thisreectspolicyaimsacrossthe

fourUKfundingbodiestomaintainandimprovethe

achievementsofthehighereducationsector,bothinundertakingexcellentresearchandinbuildingonthis

researchtoachievedemonstrablebenetstothewider

economyandsociety.

5. Intermsoftheimpactelement,HEFCE2009/38

setoutanumberofprinciplesandchallengesin

assessingresearchimpact,proposedanapproach

toassessmentbasedoncasestudies,andproposed

thatimpactshouldcarryaweightingof25percent

withintheoverallassessment.Thefundingbodies

alsostatedtheirintentiontorunapilotexerciseto

testanddevelopthecasestudyapproach,andinvitedvolunteerstoparticipate.

6. Theresponsestotheconsultation1indicated

widespreadsupportinprincipleforincludingan

elementfortheexplicitassessmentofimpactwithin

theREF,fromhighereducationinstitutions(HEIs),

missiongroups,thenationalacademies,researchuser

representativesandotherfundersofresearch,and

fromaclearmajorityofacademicsubjectassociations.

Manyqualiedtheirsupportbyemphasisingthe

needtodeveloparobustmethodforassessingimpact,

andstressingtheimportanceofthepilotexercise;theysuggestedtheweightingforthiselementwithinthe

overallqualityassessmentshouldbelowerthanthe

proposed25percent.

Impactpilotexercise

7. Followingtheconsultationexercise,theUK

fundingbodiespublished‘Initialdecisionson

theResearchExcellenceFramework’(AnnexA

of‘ResearchExcellenceFrameworkconsultation

outcomes’,HEFCECircularletter04/2010).These

decisionsappliedtoallkeyaspectsoftheassessment

frameworkexceptfortheassessmentofimpactanditsweightingwithintheframework,pendingthe

outcomesofthepilotexercise.TheREFteamthen

conductedapilotexercisetotestanddevelopthe

impactproposalswith29HEIsmakingsubmissionsto

vepilotpanels,inthefollowingunitsofassessment

(UOAs):

• ClinicalMedicine

• Physics

• EarthSystemsandEnvironmentalSciences

• SocialWorkandSocialPolicy• EnglishLanguageandLiterature.

8. Theimpactpilotexerciseconcludedinmid-

November2010,withthepublicationoftwokey

reports:‘ResearchExcellenceFrameworkimpact

pilotexercise:Findingsoftheexpertpanels’and

‘REFResearchImpactPilotExerciseLessons-

LearnedProject:FeedbackonPilotSubmissions’(by

Technopolis).Thereports,andfurtherdetailsabout

theconductofthepilotexercise,areavailableat

www.ref.ac.ukunderImpactpilotexercise.

9. Thepilotpanelsconcludedthatitispossibletoassesstheimpactofresearchthroughanapproach

basedonexpertreviewofcasestudies.Theyalso

madeaseriesofrecommendationsforreningand

improvingtheprocess,forimplementationinthefull

REF.FeedbackfromthepilotHEIsgenerallyconrms

theseconclusions,andtheywerewidelyaccepted

throughaseriesofeventswiththesectorandother

keystakeholdersheldduringlate2010todisseminate

anddiscussthepilotndings.

DecisionsoftheUKfundingbodies

Keypoints

10. Followingtheconclusionoftheimpactpilot

exerciseandsubsequentdiscussionsabout

thendings,thefourUKfundingbodieshave

decidedthat:

a. IntheREFtherewillbeanexplicitelement

toassessthe‘impact’arisingfromexcellent

research,alongsidethe‘outputs’and

‘environment’elements.

1 A summary of responses is available at www.ref.ac.uk under Background/2009 consultation.



4 REF 01.2011

b. Theassessmentofimpactwillbebasedonexpertreviewofcasestudiessubmittedbyhigher

educationinstitutions.Casestudiesmayinclude

anysocial,economicorculturalimpactorbenet

beyondacademiathathastakenplaceduring

theassessmentperiod,andwasunderpinned

byexcellentresearchproducedbythe

submittinginstitutionwithinagiventimeframe.

Submissionswillalsoincludeinformationabout

howtheunithassupportedandenabledimpact

duringtheassessmentperiod.

c. Aweightingof25percentforimpactwouldgiveduerecognitiontotheeconomicandsocial

benetsofexcellentresearch.However,given

thattheimpactassessmentinthe2014REFwill

stillbedevelopmental,theweightingofimpact

intherstexercisewillbereducedto20per

cent,withtheintentionofincreasingthisin


d. Theassessmentofresearchoutputswillaccount

for65percent,andenvironmentwillaccountfor

15percent,oftheoverallassessmentoutcomes

inthe2014REF.Theseweightingswillapplyto

allunitsofassessment.

Frameworkforassessingimpact

11. TheUKfundingbodieshavedecidedthatthe

overallframeworkforassessingimpactinthe2014

REF,thatwillapplyacrossallUOAs,isasfollows:

Scopeoftheimpactassessment

a. Theimpactelementwillincludeallkindsof

social,economicandculturalbenetsand

impactsbeyondacademia,arisingfromexcellent

research,thathaveoccurredduringtheperiod1January2008to31July2013.

b. Submittedimpactsmaybeatanystageof

developmentormaturity,solongassomechange

orbenetbeyondacademiahastakenplace

duringtheassessmentperiod.Submissionswill

beassessedinrespectofimpactsthathavetaken

placeduringtheassessmentperiod,andnot

futureorpotentialimpacts.

c. Impactsorbenetsarisingfromengagingthe

publicwiththesubmittedunit’sresearchwill

beincluded.Disseminationactivity–withoutevidenceofitsbenets–willnotbeconsidered

asimpact.

Attributionandtimeframed. Tobecreditedforanimpact,thesubmitting

unitmustshowthatitundertookresearchthat

madeadistinctivecontributiontoachieving

theclaimedimpactorbenet,thatmeets

standardsofexcellencethatarecompetitivewith

internationalcomparators.Thesubmittingunit

neednothaveundertakenallofthecontributing

research,orhavebeeninvolvedinexploitingthe

research.

e. Thetimeframefortheunderpinningresearch

willbeupto15yearsbetweenthepublicationofatleastsomeresearchoutput(s)thatmadea

distinctivecontributiontotheimpact,andthe

startoftheassessmentperiod(January2008).

Thistimeframemaybeextendedbyafurtherve

yearsforsomeUOAs,ifthesub-panelmakesan

exceptionalcasefordoingso.

Submissions

f. Submissionswillincludeatotalofonecase

study,plusonefurthercasestudyperupto10

full-timeequivalentstaffreturnedtotheREF.

Submissionswillthereforeincludeaminimumoftwocasestudies.

g. Casestudieswillbesubmittedusingageneric

templatewithwordlimits.Thiswillbedesigned

toenableinstitutionstoclearlyexplainand

demonstratetheimpactsthroughanarrativethat

includesindicatorsandevidenceasappropriate

tothecasebeingmade.Thetemplatewillrequire

informationabout:

• theunderpinningresearchandevidenceof

itsquality

• thewayinwhichthesubmittingunit’s

researchcontributedtotheimpactorbenet

• thenatureandextentoftheimpactorbenet

• referencestoindependentsourcesthatcould

befollowedup(onasamplebasis)toverify

claimsmade.

h. Theonuswillbeoninstitutionstoprovide

evidencewithineachcasestudytodemonstrate

theparticularimpactorbenetclaimed;REF

panelswilldevelopguidanceaboutthekinds

ofevidenceandindicatorstheywouldconsiderappropriatebutthisguidancewillnotbe

exhaustive.



REF 01.2011 5

i. Thecasestudiesareintendedtoshowthestrongestimpactsachievedbythesubmitted

unit’sresearch.Inadditiontothecasestudies,

submissionswillincludeinformationabout

how,duringtheassessmentperiod,theunit

hassupportedandenabledtheachievement

ofimpact.Assessmentofthisinformationwill

makeasignicantcontributiontotheimpact

sub-proles.Thetemplateforthe‘impact

statement’thatwasusedinthepilotexercisewill

bere-designedforthispurpose.

Assessment j. Theoutcomesoftheimpactassessmentwill

bean‘impactsub-prole’foreachsubmission,

showingtheproportionofthesubmissionat

eachstarredlevel(1*to4*and‘Unclassied’).

k. Casestudieswillbeassessedagainstthebroad

genericcriteriaof‘reachandsignicance’of

theimpactorbenet,andtherewillbebroad

genericdenitionsofthestarredlevelsinthe

impactsub-prolethatreectdifferentiallevels

ofreachandsignicance.

l. REFpanelswillbeinvitedtoexplaininmoredetailhowthesecriteriawillbeappliedinways

thatareappropriatetotheirdisciplines.

m. Giventhatthecriteriawillbeappliedinways

thataresensitivetodisciplinarydifferences,

theassessmentofimpactintheREFcannotbe

usedtocompareimpactssubmittedtodifferent

UOAs,ortoprovideamechanismforcomparing

therelativeimpactofdisciplines.

n. Theinformationabouthowtheunithas

supportedandenabledimpactwillbeassessed

andcontributetotheimpactsub-proles.

Furtherguidanceonthecriteriaforthiswillbe

developedinconsultationwiththeREFpanels.

Involvementofresearchusers

o. WeintendthatinallUOAs,expert‘users’of

researchfromacrosstheprivate,publicandthird

sectorswillbefullyinvolvedindevelopingthe

criteriaforimpactandinassessingtheimpact

elementofsubmissions,alongsideacademic

panelmembers.Ourapproachtoinvolvingusers

isintendedtoensuretheymakeastronginput

intotheimpactelementacrossallpanels,whilemakingeffectiveuseoftheirtime:

i. Wehaveappointedresearchusermembers

toallmainpanelsandsub-panels.These

memberswillbefullyinvolvedthroughout

thecriteriasettingandassessment

phases.

ii. During2011wewillrunaseriesof

workshopsforresearchusers,togaina

widerrangeofuserinputintothecriteriafor

assessingimpact.

iii. Duringtheassessmentphase,wewillrecruitanumberofadditionalresearchusersto

beassessorsonsub-panels,tobefully

involvedinassessingtheimpactelementof

submissions.

12. Weareengagedinongoingdiscussionswiththe

ResearchCouncils(throughResearchCouncilsUK)

toensurethattheoverallapproachtorecognising

andencouragingimpactiscoherentacrossthedual

supportsystem.Theaboveapproachtoincorporating

impactwithintheREFcomplementsthattakenby

theResearchCouncils;theREFwillrecogniseandrewardinstitutionsforhavingachievedimpactfrom

theirpastresearch,whiletheResearchCouncils’

‘PathwaystoImpact’focusonexploringpathwaysto

realisingtheimpactofnewresearchprojects.

Weightingofoutputs,impactand

environment

13. Thefundingbodiesarecommittedtoattaching

asignicantweightingtoimpact,ensuringthatitis

takenseriouslybyallkeystakeholders,andtomakethe

benetsofresearchexplicitandpublic.NeverthelesstheimpactassessmentintherstREFislikelytobe

developmental,andthe2014REFwillbetherst

experienceofassessingimpactforthoseUOAsand

HEIsthatwerenotinvolvedinthepilotexercise.

14. TheUKfundingbodiesconsiderthattheoriginal

proposalstoweightimpactat25percentremain

appropriate,givingdueweighttothewiderbenets

ofresearchwithintheoverallassessmentframework.

However,giventhattheimpactassessmentin

therstexercisewillstillbedevelopmental,the

weightingofimpactinthe2014REFwillbereduced

to20percent,withtheintentionofincreasingthisin




6 REF 01.2011

15. TheUKfundingbodieshavealsodecidedthatforthe2014REF,outputswillbeweightedat65per

cent,recognisingthatthispartoftheassessmentis

wellestablished,carrieswidespreadcondence,and

isattheheartofresearchexcellence.Environmentwill

beweightedat15percent,givingdueemphasisto

thisaspectoftheassessment.Theseweightingswill

bethesameinallUOAs.

16. Eachelementoftheassessment(outputs,impact

andenvironment)willresultina‘sub-prole’,tobe

combinedintoan‘overallexcellenceprole’awarded

toeachsubmission,basedontheaboveweightings.The‘overallexcellenceprole’willbetheprimary

assessmentoutcomefromtheexercise,andwewill

alsopublishthesub-proles.Whiletheassessment

outcomeswillbeproducedinthesamewayfor

allsubmissionsacrosstheUK,eachofthefour

UKfundingbodieswillinduecoursetakeitsown

decisionsaboutusingtheassessmentoutcomesfor

fundingpurposes.

Furtherguidance17. Thisdocumentsetsoutthebroadframeworkfor

assessingtheimpactofresearch,includinggenericcriteriaandstandardweightingsacrossallUOAs.

Furtherguidancewillbepublished,asfollows:

a. July 2011:Wewillpublishguidanceon

submissions,settingouttheinformationthatwill

berequiredinHEIs’submissions,andthegeneric

denitionsandcriteriathatwillapply.Thiswill

includeguidanceonthenatureandformatfor

theimpactelementofsubmissions,templates,

moredetailedexplanationanddenitionsthan

appearinthis‘decisions’document,aswellas

detailsofhowthesub-proleswillbecombinedtoformtheoverallassessmentoutcomes.

b. January 2012:Wewillpublishthepanelcriteria

andworkingmethods.Thesewillsetouthow

–withintheoverarchingassessmentframework–

theREFpanelswillapplytheassessmentcriteria

andprovideguidanceonappropriatekindsof

evidenceandindicatorsofimpact.Thepanel

criteriawillbedevelopedbythefourREFmain

panelswithinputfromtheirsub-panels.Wewill

publishdraftcriteriaforconsultationinJuly2011.



Northavon HouseColdharbour LaneBristolBS16 1QD

tel 0117 931 7477

e-mail [email protected]

Documents

HEFCE Impact