ALTMETRIC

SMarch 2016Ellen Fest, Hugo Besemer

Let’s vote

Altmetrics is meant1. to filter information2. to assess impact

Altmetrics measures the impact of publications that are not covered by Web of Science or Scopus

1. yes2. no

Altmetrics is not about citations1. yes2. no

How it started

2010“Altmetrics manifesto”

www.altmetrics.org

What they wanted to address

1. Peer review is slow (and everything gets published somewhere eventually)

2. Citation counts (and related measures like H-index) are slower)

3. Journal Impact Factor is incorrectly used to assess the impact of individual articles

Six years later (1)

Six years later (2)

Six years later (3)

Six years later (4)

But there are limitations

And more limitations

Some publishers provide their own metrics

PLOS●Article Level Metrics

Elsevier ●researcher dashboard

BMC Frontiers in

Example: PLoS metrics

For all technicalities see Bianca Kramer

Plum analytics: institutional metrics dashboard

http://plumanalytics.com/learn/about-metrics/

Personal profiles: Impactstory (paid)

Personal profiles: Kudos (1)

Personal profiles: Kudos (2)

Some publishers provide authors with metrics (1)

Some publishers provide authors with metrics (2)

Some publishers provide authors with metrics (3)

What meaning can be read into these numbers?

They may seen as a proxy for ●Scientific impact●“Societal impact”●Buzz

Haustein, S. (2014). Social media in scholarly communication, 42. Retrieved from http://www.cirst.uqam.ca/Portals/0/docs/Conference/PPT/StefanieHaustein_PPT.pdf

Intermezzo: citation metrics depends on subject area

Scientist Zacharias Math has a publication from 2003 with 17 citations

Scientist Molecula Biology has a publication from 2009 with 24 citations

Baselines for Mathematics

Baselines for Molecular Biology

0

100

200

300

400

0 2 4 6 8 10 12

Years after publication

Cum

ulat

ive

no. c

itatio

ns

Baselinetop 10%top 1%

How do Altmetrics baselines compare to traditional baselines - traditional

How do Altmetics baselines compare to traditional baselines – “Alt”

Tweets vs traditional metrics

“It is concluded that the scientific citation process acts relatively independently of the social dynamics on Twitter.” [1]

“A moderate negative correlation (ρ=-0.390*) is found between the number of publications and tweets per day, while retweet and citation rates do not correlate.” [2]

“automated Twitter accounts create a considerable amount of tweets to scientific papers and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment” [3]

“The results showed that approximately 76% of the sampled accounts were

maintained by individuals (rather than organizations), 67% of these accounts were maintained by a single man, and 34.4% of the individuals were identified as possessing a Ph.D, suggesting that the population of Twitter users who tweet links to academic articles does not reflect the demographics of the general public” [8]

Mendeley vs citation metrics and reads

“The overall correlation between Mendeley readership counts and citations for the social sciences was higher than for the humanities.” [4]

“…..Mendeley readership can reflect usage similar to traditional citation impact, if the data is restricted to readers who are also authors” [5]

“….it is reasonable to use Mendeley bookmarking counts as an indication of readership because most (55%) users with a Mendeley library had read or intended to read at least half of their bookmarked publications"

Blogs vs citation metrics

“….articles receiving blog citations close to their publication time receive more journal citation later…..” “….7 out of 12 journals (58%) in 2009 and 13 out of 19 journals (68%) in 2010.” [7]

?

References (1)

[1] J. C. F. de Winter, “The relationship between tweets, citations, and article views for PLOS ONE articles,” Scientometrics, vol.

102, no. 2, pp. 1773–1779, 2014. http://dx.doi.org/10.1007/s11192-014-1445-x

[2] S. Haustein, T. D. Bowman, K. Holmberg, I. Peters, V. Lariviere, and V. Larivière, “Astrophysicists on Twitter: An in-depth analysis

of tweeting and scientific publication behavior,” Aslib J. Inf. Manag., vol. 66, no. 3, pp. 279–296, 2014.

http://www.emeraldinsight.com/10.1108/AJIM-09-2013-0081\nhttp://

www.emeraldinsight.com.globalproxy.cvt.dk/journals.htm?issn=2050-3806&volume=66&issue=3&articleid=17112729&show=html

[3] S. Haustein, T. D. Bowman, K. Holmberg, A. Tsou, C. R. Sugimoto, and V. Larivière, “Tweets as impact indicators: Examining the

implications of automated ‘bot’ accounts on Twitter,” J. Assoc. Inf. Sci. Technol., p. n/a–n/a, 2015. http://

doi.wiley.com/10.1002/asi.23456

[4] E. Mohammadi and M. Thelwall, “Mendeley readership altmetrics for the social sciences and humanities: Research evaluation

and knowledge flows,” J. Assoc. Inf. Sci. Technol., vol. 65, no. 8, pp. 1627–1638, 2014. http://dx.di.org/10.1002/asi.23071

References (2)

[5] E. Mohammadi, M. Thelwall, S. Haustein, and V. Larivière, “Who reads research articles? An altmetrics analysis of Mendeley

user categories,” J. Assoc. Inf. Sci. Technol., vol. 66, no. 9, pp. 1832–1846, 2015. http://dx.doi.org/10.1002/asi.23286

[6] E. Mohammadi, M. Thelwall, K. Kousha “Can Mendeley Bookmarks Reflect Readership ? A Survey of User Literature review

Changes in scholarly reading habits in the digital era,” 2014. http://www.scit.wlv.ac.uk/~

cm1993/papers/CanMendeleyBookmarksReflectReadershipSurvey_preprint.pdf

[7] H. Shema, J. Bar-Ilan, and M. Thelwall, “Do blog citations correlate with a higher number of future citations? Research blogs as a

potential source for alternative metrics,” J. Assoc. Inf. Sci. Technol., vol. 65, no. 5, pp. 1018–1027, 2014.

http://dx.doi.org/10.1002/asi.23037

[8] A. Tsou, T. Bowman, A. Ghazinejad, and C. Sugimoto, “Who Tweets about Science ?,” Issi2015, no. 1, pp. 95–100, 2015.

http://www.issi2015.org/files/downloads/all-papers/0095.pdf