Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities

Can we use

altmetrics at the institutional level?

A case study analysing the coverage

by research areas of four Spanish universities

Can we use altmetrics at the institutional level? A case study analysing the

coverage by research areas of four Spanish

universities

Daniel Torres-Salinas*, Nicolás Robinson-García** and Evaristo Jiménez-Contreras***

•

* Univ. Navarra and Univ. Granada, EC3metrics & Medialab UGR - @torressalinas ** INGENIO (CSIC-UPV), Universitat Politècnica de València - @nrobinsongarcia

*** Universidad de Granada, Grupo EC3 and EC3metrics

To be published in: Proceedings of the 21st International Conference on Science and Technology Indicators València (Spain) | September 14-16, 2016

Full proceeding paper avaible at arxiv: https://arxiv.org/abs/1606.00232

Can we use altmetrics at institucional level?

Introduction


Introduction: altmetric studies

Social media based indicators or altmetrics have been under scrutiny for the last seven years.

1.- The relation between citations and altmetric indicators

2.- Coverage and diversity of social media sources

3.- Their meaning as impact indicators

Three lines of work can be observed


Introduction: a gap in altmetric studies

1. Most studies are focus on the understanding of the nature and relation of altmetric indicators with citation data

No study has been found profiling larger units of research such as institutions, especially universities

2.- Most papers are analysing altmetric indicators at paper (paper level metrics) or author level (author level metrics)

.


Introduction: information sources

On the other hand, many data providers offering social media metrics have been developed

Plum analytics

Impact Story

Altmetric.com


Introduction: altmetric.com

Altmetric.com has positioned itself as the most used source for altmetric studies

1.- Variety of social media platforms with regard to mentions, readership, etc. of scientific papers

2.- Offers an aggregated indicator or ‘altmetric’ score which is a weighted sum of values based of a given article in different social media

3.- Maintaining the ‘history’ of each altmetric indicator to an article

Advantages of altmetric.com

Objetive 1: Analyse the coverage of Altmetric.com at the institutional and field level and verify its validity as evaluation tool

Objetive 2: Analyse differences when obtaining subject bibliometric profiles from Altmetric.co and explore the possibility to make benchmarking studies at university level


Objetives: altmetrics and universities

Our objectives are:


Material & methods


Material & methods: 4 universities

y Size Staff

Type and main field

Foundation Year

Region & City

University of Granada

2399 Multidisciplinary 1531 Granada

University Pompeu Fabra

288 Specialized

Medicine/Biology 1990

Barcelona

Polytechnic University Valencia

1847 Specialized Engineering

1971 Valencia

University Carlos III

539 Specialized

Social Sciences 1989 Madrid

1.- We will analyze 4 different Spanish universities 2.- Present different sizes and research publication specialization 3.- Different Foundation year and located in different regions

Selection of spanish universities


Material & methods: Web of Science

1.- Citable papers for the 2014 year indexed in the Web of Science for the 4 universities were retrieved (5922 records)

2.- Records from the Web of Science provide among other information the DOI number of each publication

3.- The DOI number is very important, as it will allow us to query the Altmetric.com API for each document

4.- A total of 93% of the total share from our data set included DOI

Search for Web of Science papers and check DOI coverage

5.- Publications have been assigned to research areas


Material & methods: Web of Science

1.- We retrieved altmetric data available for our set of papers with DOI.

2.- Then, the share of records with a score assigned by Altmetric.com has been calculated

3.- Altmetric.com covered 2149 records from the Web of Science, representing 36% of our data set

Search for altmetric.com score using DOI

Can we use altmetrics at institutional level?

Results


Results: altmetric.com coverage

Documents retrieved 4 universities

5922 93% with DOI

5922 web of Science Documents

NOT Indexed in altmetric.com 3773 - 74%

Indexed in altmetric.com 2149 -36%

Indexed = at least with a value of 1 in altmetric score



NOT Indexed in altmetric.com 3773 - 74%

There are significance difference in the % of Web of Science documents indexed in altmetric.com


Results: altmetric.com coverage Altmetric.com score are very concentrate just in

Twitter and Mendeley Readers


Results: altmetric.com coverage We can compare the “social media” patron for the four universities



Important differences by university


Results: Research field profile

Gra

nad

a P

om

pe

u F

abra

P. V

ale

nci

a C

arlo

s II

I We have obtained the coverage profile (Web of science papers) of altmetric.com per research fields: • Significant differences in

research field coverage (ej: Pompeu Fabra Vs Carlos III)

• Best coverage in all fields in universities with S&T profile

(ej: P. Valencia & Pompeu Fabra) • Dramatic coverage of

Humanities (ej: Granada) and quite well in Social Science


Results: Research field profile

Tota

l alt

me

tric

.co

m S

core

Nr of Web of Science Papers indexed in Altmetric.com

Map of Web of Science Categories

We can benchmark universities and compare altmetric performance at category level

Multidisciplinary Sciences Pompeu Fabra

Biochemistry Pompeu Fabra Biochemistry

Granada

Multidisciplinary Sciences Granada


Results: Research field profile – top10

Multidisciplinary Science (Science, Nature, Plos one…) have higher coverage and concentration of altmetric.com score. In the case of Pompeu Fabra 60% of altmetric.com score are related to papers published in multidisciplinary journals.

We can see that social sciences and humanites are not well represented in top categories


Results: Research field profile – top10

Top papers according Almetric.com Scores and journals where have been published

We can observed the “Nature” and

“Science” effect that determine altmetrics in these case studies

Final results remarks


Concluding remarks


Concluding remarks


YES √ We can make a local bibliometric database combining bibliometric data

from Web of Science and altmetric data from altmetric.com

√ We can calculate altmetric indicators at university level and offer an

altmetric research field profile in different levels (fields and categories)

√ We can compare and make universities benchmarking studies


Concluding remarks

We observed a low coverage of altmetric indicators with only 36% of all documents have an ‘altmetric’ score

To take into account: Low coverage of web of science papers:

The DOI´s problem: One of the main factors to such low coverage could be the use of DOI in altmetric.com

not all papers indexed in the Web of Science include DOI not all people that mention a paper in social media use the DOI

So using DOI we have a loss of information


Concluding remarks To take into account:

Multidisciplinary Journals concentrate altmetrics

The Nature and Science effect

We observed that “multidisciplinary sciences” accumulate a big piece of the total altmetric score at university level. So universities publishing in nature, science, pnas, etc.… also have high levels of mentions in social media.


Concluding remarks To take into account:

Social science & humanities: low coverage and impact

The national factor and language bias. It is necessary to verify if there is a national or linguistic factor influencing the coverage of altmetric indicators in the areas of Socials Sciences and Humanities & Arts.

Another explanation could be that these areas are represented in social media by small communities, limiting their capacity to

disseminate their research papers in social media.

Can we use

altmetrics at the institutional level?

Full proceeding paper avaible at arxiv:

https://arxiv.org/abs/1606.00232

Education

Can we use altmetrics at the institutional level? A case study analysing the coverage by research areas of four Spanish universities