Upload
lseimpactblog-impact
View
224
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Jane Tinkler's presentation on How to: Writing a REF Impact Case Study, from the Future of Impact conference.
Citation preview
Writing a REF Impact Case Study
Impact of Social Sciences Project
LSE Public Policy Group
The current state of play
• 360 days to REF deadline
• Survey of submission intentions due in tomorrow (only 300 words in total)
• Universities have a first draft of prospective cases
• But these still need quite a lot of work
• How your institutions are handling things will vary
• There is still time for action – but much uncertainty, and we have no magic or official answer
Today is a discursive session on key uncertainties:
1. Uncertainties around reach and significance
2. Envisioning the overall narrative
3. Proving 2* quality for underpinning research
4. What is ‘impact’ and what is not?
5. Maximizing intensity of impact evidence
Overview of the session
1. Uncertainties around reach and significance
Definitions of reach and significance
• Reach is understood as extent and diversity of the communities, environments, individuals, organizations or any other beneficiaries that have benefitted or have been affected.
• Significance is understood as the degree to which impact has enriched, influenced, informed or changed policies, opportunities, perspectives, or practices of communities, individuals or organizations.
Mapping HEFCE’s impact criteria
Definitions of reach and significance • What is the relationship between the two? Can
increased reach compensate for reduced significance?
• Emphasis on showing change – this implies need to demonstrate linear effects
• Uncertainty around things like generating debate in a profession, engagement over proposition not taken up
• Is this biased towards public policy impacts?
• Panels reassure us that iterative, dynamic, and narrative accounts will be expected and encouraged
2. Envisioning the overall narrative
Think creatively about the narrative
• It’s all about the narrative – and the ‘so what?’ question
• Start with the user side and work backwards
• Not necessarily your most cited academic work – where have you engaged?
• Research outputs must be ‘credibly linked’ to impact
• Corporate versus individual submissions
• Integrate different streams of work or expertise in the department
The case study narrative
Short summary of the case (100 words)
Underpinning research (500 words)
References to the research (max. 6)
Details of the impact (750 words)
Corroborating sources Max. 10 (5 individuals)
Strong opening headlining the research and impacts
Narrative account of how the research evolved
Majority of 2* references to research outputs
Building to a strong, factoid-intensive narrative on impacts
Good mix of references and testimonies,
Imbalance towards research rather than impacts
Over-emphasis on research outputs and weak quality indicators
Excessive focus on dissemination and advice
Lack of factoid evidence and weak signs of self-research, over-reliance on weak testimonials
IDEALLY… IN REALITY…
Process trace your impact back to your underlying research
Effects
Changes
Benefits
Impact of outputs or activities of
external body/actors
Intermediate impacts – usage or cites, by external actors
Partnership funding /
Direct contacts = auditable impacts
Under- pinning
research
Evidence of (strong)
dissemination
Public engagement contributions
Public engagement contributions
3. Proving 2* quality of underpinning research
Deciding what research to submit
• All work must cross the 2* quality threshold (i.e. it must be ‘internationally recognized’)
• Easier for traditional academic outputs (i.e. already in previous RAE submissions)
• However impact research outputs may be more applied: e.g. grey literature
• Panel will take a look, if not obviously 2*, then it is read.
• Therefore need to reassure the Panel that this work is of the required quality via another means
Critically assessing your own research
• Has any of your impact output been assessed e.g. through an internal or organisational review process?
• For research reports, was an evaluation completed on outputs from a programme?
• Panels C and D accept outputs that can be shown to be highly cited or form a ‘reference point for further research beyond the original institution’
• If none of these apply, then you or your Department will need to organise your own review or evaluation
4. What is impact and what is not?
What is feasible in assessing impact?
Writing about your impacts
– Dissemination activities on their own without any follow-on interaction or take up
– Tricky in the area of public dissemination – a radio broadcast is not enough
– Something that is generally part of academic service
– Just providing expertise and advice to however important a person/organization
Net impacts
Gross impacts
Non-impacts
– Maximizing the level of detail and specifics at this level – always be above this threshold
– Danger of over-claiming or making infeasible estimates
– Making informed guesstimates at net effects
5. Maximizing intensity of impact evidence
Research your impact proactively
• 750 words max to demonstrate intensive impacts
• Narrative should not detract from specifics / data
• Being your own detective and research yourself
• Start with Google and see where your research has been picked up by external users
• Then carefully document official and unofficial dissemination that you undertook
• Outline as clearly as you can the ‘effect, change or benefit’ that your research made.
Research your impact proactively II
• Can you be precise about who exactly used your work in business, government, or third sector?
• Can you offer at least some good evidence of how they used it and the effects it had?
• Did your contacts or involvement continue over a period of time? And did it had knock-on effects?
• Did people pay for your work or advice? Can you track all the flows of money?
For more details see:
Impact of Social Sciences blog:
http://blogs.lse.ac.uk/impactofsocialsciences/
Email: [email protected]
Twitter: @lseimpactblog
Facebook: Impact of Social Sciences
Supplementary Slides
Definition of impact
• ‘an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’
• INCLUDES raising the level of public debate and public scrutiny, public engagement, changing policy and practices, use by professional bodies.
• EXCLUDES (a) impact on research or the advancement of academic knowledge within the HE sector (b) acting as an expert or advisor to a public body
The REF Main Panels
Main Panel A covers: Clinical Medicine Public Health, Health Services and Primary Care Allied Health Professions Psychology, Psychiatry and Neuroscience Biological, Agricultural and Food Sciences
Main Panel B covers: Earth Systems and Environmental Sciences Chemistry and Physics Mathematical Sciences Computer Science and Informatics All areas of Engineering
Main Panel C covers the social sciences including: Anthropology; Business and Management; Education; Economics; Geography, Law; Politics and IR; Social Policy; Sociology
Main Panel D covers the humanities including: Art and Design; Dance; Drama; Communication and Media Studies, English; History; Modern Languages; Music; Philosophy; Theology;
More on the social science REF panel
• One user appointed per sub-panel
• These appointees will coordinate recruiting additional
research users (by Spring 2013)
• Two readers of case studies – 1 user, 1 academic
• Not ‘user-led’ but users will play a key role in
assessment
HEFCE guidance on underpinning research
HEFCE guidance on recording impacts
HEFCE guidance on impact evidence
For public engagement cases
You need to be able to answer these questions:
• Can you show evidence of your research being strongly disseminated?
• Did you undertake outreach activities?
• Can you show evidence of your research being widely received?
• Were there secondary or follow-on effects? i.e. reaching additional audiences, building new interactions, creating additional outputs.