Preparing for Impact

Jane Tinkler

Impact of Social Sciences Project

LSE Public Policy Group

NUI Galway, 7 June 2012

The rise of the ‘impact agenda’ • There is increased discussion of the usefulness of

‘impact’ as a measure to assess academic outcomes.

• In the UK, HEFCE has introduced an impact element to its quality assessment process.

• All UK funders now require an impact statement to be supplied for each funding application submitted to them.

• The ‘academic spring’ has linked impact to the opening up of academic publications allowing research to be read and used without needing to pay subscriptions.

• But for many academics, impact is still a confusing concept and one that is disconnected from their academic lives.

1. What are research impacts and how can you measure them?

What are research impacts?

A research impact is a recorded or otherwise auditable occasion of influence from academic research on another actor or organization.

a. Academic impacts from research are influences upon actors in academia or universities, e.g. as measured by citations in other academic authors’ work.

b. External impacts are influences on actors outside higher education, that is, in business, government or civil society, e.g. as measured by references in the trade press or in government documents, or by coverage in mass media.

Tools for tracking academic impact Tools Pros Cons

Bibliometric databases such as ISI Web of Science and Scopus

Gives accurate citation counts (no duplications)

Biased towards STEM disciplines, US and English language outputs

Only covers articles

Tools for tracking academic impact Tools Pros Cons

‘Tweaked’ versions of Google such as Harzing’s Publish or Perish

Allows computation of citation scores

Covers all academic outputs that are on the web

Easy to correct duplications

Tools for tracking academic impact Tools Pros Cons

Open search via Google Scholar Citations

Covers all academic publications

Can link to both articles and co-authors

Easy to use and will be taken up quickly

2. Then how do you assess your impact scores?

Putting your impact profile in context

A number of factors give context to your impact scores. These can include:

• Your career position: Senior staff generally have higher citation rates and H scores as they have published more and have had longer for these publications to be read.

2.1

2.3

4.6

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Lecturer Senior Lecturer Professor

Ave

rgae

h-s

core

Academic Position

Average H scores by career stage

Putting your impact profile in context

A number of factors give context to your impact scores. These can include:

• Your career position: Senior staff generally have higher citation rates and H scores as they have published more and have had longer for these publications to be read.

• Your discipline: Disciplines vary in the outputs they produce, their citation rates and their H scores.

Types of academic outputs by discipline

0% 20% 40% 60% 80% 100%

Psychology

Political Science

Economics

Business and Management

Media Studies

Philosophy

Law

Sociology

Academic articles

All book outputs

Working papers

Conference papers

Average H-scores by discipline and career position

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

Ave

rage

h-s

core

Discipline

Lecturer

SeniorLecturer

Professor

Putting your impact profile in context

A number of factors give context to your impact scores. These can include:

• Your career position: Senior staff generally have higher citation rates and H scores as they have published more and have had longer for these publications to be read.

• Your discipline: Disciplines vary in the outputs they produce, their citation rates and their H scores.

• Your research focus: Government draws more on some disciplines than others. Civil society groups are more likely to publicly use and quote academic work. Businesses tend to employ graduates to update their knowledge or methods base.

Disciplinary differences in external references

-16-12

-8-4048

12Pol Sci Philo Hist Law Econ Socio

Bus &Man Geog

Comm &Med Anthro Soc Pol Psych

References in traditional academic locations

-4-3-2-101234

Soc Pol Psych Geog Socio Econ Law Pol SciBus &Man Anthro

Comm &Med Hist Philo

References in government bodies only (UK and abroad)

Dev

iati

on

fro

m t

he

mea

n o

f th

e to

tal

of

all r

efer

ence

s

3. Planning for Impact

Academic communication is changing

Academic communication involves: • Journal articles, conference papers, books and reviews. • Journal articles and books are read by some in your field,

but don’t often break into other disciplines and rarely picked up by the media.

• Outputs are often fairly long and in language that is sometimes meaningful only to other academics.

BUT • Academics are observers who need to communicate

their observations to the world (in a timely fashion). • Much of our knowledge and input goes unapplied

because of very long time-lines for outputs, and lack of adaptation or translation.

Seven steps to creating impact

• Step 1: Think about how your research and what types of outputs you produce.

Co-authorship and citations

0

200

400

600

800

1000

1200

0 1 2 3 4 5 6 ormore

Nu

mb

er o

f O

utp

uts

Number of Co-authors

Co-authorship and Number of Outputs

0

1000

2000

3000

4000

5000

6000

7000

0 1 2 3 4 5 6 or

more

Cit

ati

on

s r

eceiv

ed

Number of Co-authors

Co-authorship and Citations

Most outputs in our dataset were single authored, but more cites went to outputs that had at least one other author

• Step 1: Think about how your research and what types of outputs you produce.

• Step 2: Pick as distinctive a version of your author name as possible and stick with it.

• Step 3: Write informative article titles, abstracts and book blurbs.

• Step 4: Build communication and dissemination plans into research projects early on.

Seven steps to creating impact

• Step 5: Make full use of your university’s resources (like online depositories, Expert directories, knowledge transfer schemes) as well as using public resources like creating a profile in Google Scholar Citations.

• Step 6: Ensure you put a version of all publications on the open web and use social media to raise the profile of your research, e.g. write blogs, write online book reviews, tweet.

Seven steps to creating impact

A team from the World Bank examined the influence of economic blogs on download figures for articles

A team from NCRM compared the effect on a paper’s downloads via twitter and other routes

• Step 5: Create a public profile on your university site and on Google Scholar Citations.

• Step 6: Ensure you put a version of all publications on the open web and use social media to raise the profile of your research, e.g. write blogs, write online book reviews, tweet.

• Step 7: Consider working with intermediate organisations where possible to help disseminate your work and create impact e.g. think tanks, community groups.

Seven steps to creating impact

What constrains impacts?

Higher Education Institution

Private / public / third sector organisation

Lack of time Bureaucracy and inflexibility of

HEI administration Difficulties in identifying partners Insufficient rewards and lack of

awareness of the benefits from the interactions

Lack of understanding by academics of the process

Capacity and capability of the KE system still developing / evolving

Lack of resources within external organisations to fund the KE engagement

Insufficient benefits from the interaction

Lack of interest by external organisations and lack of demand for KE

Intellectual property agreements as a barrier to some, albeit minority of, KE engagement

Source: PACEC/CBR Survey of Academics (2008); PACEC/CBR Survey of Enterprise Offices (2010); CBR Survey of Enterprises (2008)

4. What picture of external impact has our research found?

University departments (20%)

Academic publishers

and journals (20%)

All libraries (14%)

Digital aggregators (4%)

Academic assocs.

and societies

(7%)

Independent think tanks

(4%)

Media and press (5%)

Civil society and third sector

(7%)

Govt & policy (5%)

Private sector (3%)

Individs

(4%)

Univ. centres

and instits. (7%)

Digital research

databases

Aca

dem

ic r

esea

rch

an

d

enga

gem

en

t

We used Google to search 264 social science academics and recorded where we found references to them or their research.

Mapping distribution of impacts

66 per cent of references are international

Where we found social science references in UK government

Notes: N = 325. ‘Central Government Departments’ includes the Home Office, the

Department for Work and Pensions, and the Department for International Development, among others. ‘NDPB’ refers to Non-departmental Public Bodies (excluding Research Councils) and includes the National Archives.

0 5 10 15 20 25 30 35

Other

NDPBs

Central Govt Depts

Local Govt

UK Parliamentary

Research Councils

% of total UK government references

Where we found social science references in business

0 5 10 15 20 25 30 35

SMEs

Mediators

Associations

Major corporations

% of total private sector references

Bu

sin

es

s t

yp

e

For more see:

Maximising the Impacts of your Research: A handbook for social

scientists

Using Twitter in University Research, Teaching and Impact Activities: A guide for academics and

researchers

Freely available to download from the Impact of Social Sciences blog:

http://blogs.lse.ac.uk/impactofsocialsciences/ Email: impactofsocialsciences@lse.ac.uk

Twitter: @lseimpactblog Facebook: Impact of Social Sciences