Normalization of Mendeley reader impact on the reader- and paper-side

Normalization of Mendeleyreader impact on the reader- andpaper-sideRobin Haunschild Lutz Bornmann

09/16/2016

Introduction

AltmetricsAlternative metrics, closely related to article level metricsFacebook: posts, likes, ...Twitter: tweets, retweets, ...Mendeley, CiteULike, Zotero, ...: readers (reader counts,bookmarks, saves, ...)News outlets: stories, mentionsBlogs: stories, mentions...

Two major fields of altmetrics research1. Meaning of altmetrics counts2. Normalization of altmetrics counts

Mendeley reader impact Robin Haunschild et al. 2

Introduction

MendeleyOnline reference managerDesktop and mobile applicationsSocial, academic networking componentAPI for user statistics

Global web traffic rank from Alexa

Mendeley reader impact Robin Haunschild et al. 3

Introduction

Research Questions1. Is normalization important for Mendeley reader counts?2. Which normalization procedures are possible?3. Can analogous versions to citing-side and cited-side

normalizations be done using Mendeley reader counts?4. How do the methods differ?

Mendeley reader impact Robin Haunschild et al. 4

Data set

Data setDOIs from WoS papers from 2012 (Na = 1,133,224 articlesand Nr = 64,960 reviews)Search via Mendeley API for DOI in December 2014Overall 94.8% of the articles and 96.6% of the reviews werefound on Mendeley9,352,424 Mendeley reader counts for the articles and1,335,764 for the reviews.0.05% of the article readers and 0.04% of the reviewreaders did not share their (sub-)disciplinary information

Mendeley reader impact Robin Haunschild et al. 5

Example data from Mendeley

Data available from MendeleyInput DOI, PubMedID, ... (e.g., 10.1063/1.4769790, Insensitivityof the error of the minimally empirical hybrid functionalrevTPSSh to its parameters):

Total reader count (here: 9)Reader count per academic status (here: 2 Researchers, 1Other, 3 Professors, 2 PhD Students, and 1 AssociateProfessor)Reader count per Mendeley discipline (here: 1 inMaterials Science, 1 in Physics, 5 in Chemistry, 1 in SocialSciences, and 1 in Economics)Reader count per country (here: 1 in USA, 1 in the Vatican,and 1 in South Korea)

Mendeley reader impact Robin Haunschild et al. 6

Mendeley readers per document type

Mendeley readers per article in the Mendeley disciplines

Mendeley reader impact Robin Haunschild et al. 7

Mendeley readers per document type (cont’d)

Mendeley readers per review in the Mendeley disciplines

Mendeley reader impact Robin Haunschild et al. 8

Mendeley readers per document type (cont’d)

Mendeley readers per article in the top WoS subjectcategories

Mendeley reader impact Robin Haunschild et al. 9

Mendeley readers per document type (cont’d)

Mendeley readers per review in the top WoS subjectcategories

Mendeley reader impact Robin Haunschild et al. 10

Cited-side and citing-side methods

Cited-side1. Counting all citations a paper has received2. Normalization of citation counts with respect to the

scientific field of the cited paper

Analogous version for Mendeley reader counts: paper-side

Citing-side1. Counting all citations a paper has received separately for

each scientific field of the citing paper2. Normalization of citation counts with respect to the scientific

field of the citing paper

Analogous version for Mendeley reader counts: reader-side

Mendeley reader impact Robin Haunschild et al. 11

Method

Paper-side NormalizationAverage number of readers per paper (ρ) in a scientific field(here: WoS subject category), document type, and publicationyear:

ρc =1

Nc

Nc∑i=1

Ri (1)

Nc : Number of papers in a WoS subject category, document type,and publication yearRi : Number of reader counts of paper i

NRSi =Ri

ρc(2)

NRSi : Normalized Reader Score for paper iMultiplicative (or fractional or full counting) for papers with WoSsubject categories

Mendeley reader impact Robin Haunschild et al. 12

Method

Paper-side Normalization (cont’d)Average over a set of papers of a specific unit:

MNRS =1N

N∑i=1

NRSi (3)

N: Number of papers in a specific research unit

InterpretationAnalogous to MNCS:

(M)NRS ≈ 1: average reader impact of paper(M)NRS < 1: below average reader impact of paper(M)NRS > 1: above average reader impact of paper

Mendeley reader impact Robin Haunschild et al. 13

Method

Reader-side NormalizationAverage number of readers per paper (ρ) in a scientific field(here: Mendeley discipline), document type, and publicationyear:

ρd =1

Nd

Nd∑i=1

Rid (4)

Nd : Number of papers in a Mendeley discipline, document type, andpublication yearRid : Number of reader counts of paper i in Mendeley discipline d

βid =Rid

ρd(5)

βid : Normalized Reader Score for paper i in Mendeley discipline d

Mendeley reader impact Robin Haunschild et al. 14

Method

Reader-side Normalization (cont’d)Sum over all Mendeley disciplines yields a paper-based readerimpact value:

DNRSi =D∑

d=1

βid (6)

D: Number of Mendeley disciplines where paper i has readers.DNRSi : Discipline Normalized Reader Score of paper iAverage over a set of papers of a specific unit:

MDNRS =1N

N∑i=1

DNRSi (7)

N: Number of papers in a specific research unit

Mendeley reader impact Robin Haunschild et al. 15

Spearman correlation coefficients for journals

MDNRS vs. MNRS for different OECD categories

OECD category rs No. of journalsNatural sciences 0.75 3337Engineering and technology 0.81 1556Medical and health sciences 0.82 2855Agricultural sciences 0.89 385Social sciences 0.83 1920Humanities 0.42 563

Mendeley reader impact Robin Haunschild et al. 16

Additional Information

More detailsDOI: 10.1016/j.joi.2016.04.015

Submission history22 February, 2016: Upload of Manuscript (version 1) toFigshare and submission of revised version to JoI.07 March, 2016: Submission of abstract for this contribution14 March/29 March, 2016: Submission deadline for STIcontributions22 April, 2016: Manuscript accepted by JoI and upload offinal manuscript version to Figshare.30 May, 2016: Formal acceptance of this contribution to thisconference.DOI: 10.1016/j.joi.2016.04.015, published in the August2016 issue of JoI.

Mendeley reader impact Robin Haunschild et al. 17

Summary and Conclusions

Answering the Research QuestionsYes, normalization is important for Mendeley reader counts.Raw reader counts should not be used for impactassessment.Paper-side and reader-side normalizations are possible.The paper-side normalization is the analogue of thecited-side normalization, and the reader-side normalizationis the analogue of the citing-side normalization.The reader-side and paper-side normalization methodsprovide slightly different rankings, e.g. for journals.

Mendeley reader impact Robin Haunschild et al. 18

Summary and Conclusions

Normalization of Mendeley reader countsTwo different methods for normalization of Mendeley readercounts were presented.Both methods correlate larger than expected for mostjournals.

OutlookNormalization with respect to Mendeley disciplines hasbeen done.Is it useful to normalize with respect to:

academic status groups orcountry affiliations

of Mendeley readers?

Mendeley reader impact Robin Haunschild et al. 19

53 23 1653 23 16

Mendeley reader impact Robin Haunschild et al. 20