Embed Size (px)
DESCRIPTION
Citation preview
Incorporating impact in the REF______________________________________________________________________________________________________________________________
The aim is to identify and reward the contribution that high quality research has made to the economy and society:
– Making these explicit to the government and wider society
– Creating a level playing field
– Encouraging institutions to achieve the full potential contribution of their research in future
Impact: initial consultations______________________________________________________________________________________________________________________________
Widespread acceptance of the principle of incorporating impact in the REF, and agreement that the impact assessment should:
- Be based on expert review
- Review historical impacts, not predict future impact
- Focus on the impact of submitted units’ research, not individual researchers
- Be underpinned by high quality research
- Take a wide view of impact, inclusive of all disciplines
The impact pilot exercise______________________________________________________________________________________________________________________________
Tested and developed a case study approach to assessing the impact of research
Five units of assessment (UOAs) 29 UK higher education institutions each submitting to 2 UOAs Each submission included:
- An ‘impact statement’ for the submitted unit as a whole- Case studies illustrating examples of impacts achieved (a total of one
case study per 10 research staff) Impacts that occurred during 2005-09, underpinned by
research since 1993
The pilot panels______________________________________________________________________________________________________________________________
Membership drawn from academia and research users from the private, public and third sectors
The panels tested the methodology by:- Assessing the case studies in terms of ‘reach and significance’ of
the impacts- Considering the wider ‘impact statements’- Producing impact profiles- Reflecting on the process, identifying issues and making
recommendations on how to improve the process
Key findings______________________________________________________________________________________________________________________________
The process makes explicit the benefits that research in each discipline brings to society
Benefits of research______________________________________________________________________________________________________________________________
Key findings______________________________________________________________________________________________________________________________
The process makes explicit the benefits that research in each discipline brings to society
It is possible to assess the impact of research, through expert review of case studies
A number of refinements are needed for full implementation A generic approach is workable, with scope for REF panels to tailor
the criteria as appropriate to their disciplines The weighting should be significant to be taken seriously by all
stakeholders, and needs careful consideration
Decisions on impact______________________________________________________________________________________________________________________________
Published decisions available online at: www.ref.ac.uk
Weightings of the three elements: outputs, impact and environment
Broad framework for the assessment of impact
The REF Framework______________________________________________________________________________________________________________________________
Broad framework ______________________________________________________________________________________________________________________________
Based on expert review of case studies (1, + 1 in 10) May include any social, economic or cultural impact or benefit
beyond academia, arising from excellent research, that has taken place during the assessment period
Also include information about how the unit has supported and enabled impact during the assessment period
Assessment of reach and significance Involvement of research-users Further guidance: July 2011 and January 2012
BIBLIOMETRICS IN THE REF
Royal Society of MedicineUniversity Health & Medical Librarians’ Group
7 March
Background to bibliometrics Bibliometrics pilot Preliminary feedback from sub-panels Next steps
Contents______________________________________________________________________________________________________________________________
Background to bibliometrics______________________________________________________________________________________________________________________________
Count the number of citations gained by each item under assessment Time dependence Subject dependence Benchmarking of raw counts
22 UK HEIs All UOAs with reasonable coverage in the bibliometric
databases Trialled several approaches
Address based model Author based models
Just RAE submitted people Selected papers
Bibliometrics pilot______________________________________________________________________________________________________________________________
Fully inclusive models difficult Institutional data verification Defining subject areas
Cannot use formulaically - though significant potential to support assessment process
Equalities implications
Conclusions from the pilot______________________________________________________________________________________________________________________________
Interest from main panels A and B Little interest from C and D
Range of options Simple counts and guidance on use Fully normalised / benchmarked
REF Sub-panel feedback______________________________________________________________________________________________________________________________
Not for use formulaically As a means of informing peer review
Cross-check/tie break Reflects academic significance only
Supporting evidence only Concerns about:
Differential coverage Discouraging applied/translational work Timeliness of data
Clarity of guidance on submissions Normalisation of scores Time lag on indicators
REF Sub-panel feedback______________________________________________________________________________________________________________________________
Procurement of citation data Discussions with several suppliers
REF steering group will take a decision informed by Procurement discussions Panel feedback
Decision published in July
Next steps______________________________________________________________________________________________________________________________
Thank you______________________________________________________________________________________________________________________________
Kim HackettPolicy Adviser
HEFCE
0117 931 [email protected]
David MawdsleyAnalystHEFCE
0117 931 [email protected]
