THE C

OMPLEXITY

OF

MEASURING BOOKS’

IMPACT Gali HaleviBarnaby

Nicolas

WHAT IS SCHOLARLY IMPACT AND WHY IS IT IMPORTANT?

Scholarly impact = what influence does research have on

scientific discovery and society

Impact is measured by Scholarly Metrics which are used for:

Evaluating the impact of research Promotion and tenure. Mapping the competitiveness of an institution Evaluating journals

Zuccala, A., & Bod, R. (2012). Book reviews as ‘Mega-Citations’: A fresh look at citation theory. Proceedings In 17th international conference on science and technology indicators, STI 2012.

Zuccala, A. A., Verleysen, F. T., Cornacchia, R., & Engels, T. C. E. (2015). Altmetrics for the humanities Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67(3), 320-336. doi:10.1108/Ajim-11-2014-0152

Donovan, C. (2007). The qualitative future of research evaluation. Science and Public Policy, 34(8), 585-597. doi:10.3152/030234207x256538

Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto/

Traditional MetricsUsually based on number of

publications and number of citations

Examples include: Impact Factor: Journal

Citation Reports. H-index SCImagoJournalRank(SJR) Impact Per Publications (IPP Source Normalized Impact

per Paper (SNIP) Book Citation Index (BCKI)

Altmetrics Usually based on measuring

interaction and usage Examples include: Literature readership Downloads Bookmarking Popular mentions and

reviews (Wikipedia, blogging, reviews sites) Social media Multimedia content sharing

applications

TRADITIONAL METRICS & ALTMETRICS

González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of Informetrics, 4(3), 379-391. doi:http://dx.doi.org/10.1016/j.joi.2010.03.002

Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 265-277. doi:http://dx.doi.org/10.1016/j.joi.2010.01.002

Elsevier. (2016). Journal Metrics from Elsevier. Retrieved from http://www.journalmetrics.com/

Gorraiz, J., Purnell, P. J., & Glänzel, W. (2013). Opportunities for and limitations of the Book Citation Index. Journal of the American Society for Information Science and Technology, 64(7), 1388-1398. doi:10.1002/asi.22875

WHY IS IT DIFFICULT TO MEASURE THE IMPACT OF BOOKS?

The citation indexes for monographs (such as Web of Science, Scopus, and Google Scholar) do not have extensive coverage of citations from books and all seem to be biased towards publications from Western nations

Books are not cited as Often as articles.

Diverse format types of monographs include edited volumes, major reference works and graduate level textbooks

In many disciplines books become obsolete much faster than articles.

Abrizah, A., & Thelwall, M. (2014). Can the impact of non-Western academic books be measured? An investigation of Google Books and Google Scholar for Malaysia. Journal of the Association for Information Science and Technology, 65(12), 2498-2508. doi:10.1002/asi.23145

MULTI-FA

CETED

MEASUREMENT O

F E-

BOOKS USING PLUM

ANALYTICS

MetricsCategories

USAGE(clicks, downloads, views,

library holdings, video plays)

CAPTURES(bookmarks, code forks,

favorites, readers, watchers)

MENTIONS(blog posts, comments, reviews, Wikipedia links)

SOCIAL MEDIA(+1s, likes, shares, tweets)

CITATIONS(PubMed Central, Scopus,

patents)

CAPTURES: M

ENDELEY &

GOODREADS

MENDELEY CAPTURES

Phen

omen

olog

y of

per

cept

ion

Prin

ciple

s of fl

uore

scen

ce sp

ec-tro

scop

y

Cont

ent a

nd co

nscio

usne

ss

Netw

orks

, Cro

wds,

and

Mark

ets

Foun

datio

ns o

f Qua

litat

ive

Rese

arch

: In

terp

retiv

e an

d Cr

itica

l App

roac

hes

Prot

ein

Prot

ocol

s Han

dboo

k, T

he

Pyth

on S

crip

ting

for C

ompu

tatio

nal

Scie

nce

An E

cosy

stem

in T

rans

ition

Doin

g Fo

cus G

roup

s

Intro

duct

ory

Tim

e Se

ries w

ith R

2013 2007 2010 2010 2012 2002 2010 2011 2015 2009

0

2000

4000

6000

8000N

umbe

r of

Men

dele

y ca

ptur

es

Philosophy

Medicine Psychology

Business Business Medicine Computer Science

Biology Business Mathematics

GOODREADS – ALL TOP BOOKS ARE CLASSIC ENGLISH LITERATURE

The

grea

t Gat

sby

Prid

e an

d pr

ejud

ice

Rom

eo a

nd Ju

liet

The

Oxfo

rd S

hake

spea

re: R

omeo

and

Ju

liet

The

curio

us in

ciden

t of t

he d

og in

the

nigh

t-tim

e

The

pict

ure

of D

oria

n Gr

ay

The

Coun

t of M

onte

Cris

to

The

adve

ntur

es o

f Tom

Saw

yer

The

scar

let l

ette

r

Ham

let

1998 2006 2004 2000 2012 2006 2008 2010 2007 2003

0500000

100000015000002000000250000030000003500000 3128792

25805021850316 1850042

1141000 927624 882900 699104 696169 694376

Goodreads Captures

VIEWS & DOWNLOADS

HTML CLICKS

A Fr

amew

ork

for K

-12

Scie

nce

Educ

a-tio

n: P

ract

ices,

Cros

scut

ting

Con-

cept

s, an

d Co

re Id

eas

Chee

se a

nd M

icrob

es

One

Heal

th

Polit

ical A

uton

omy

and

Divi

ded

So-cie

ties

Micr

obes

and

Evo

lutio

n

Valu

e-Fr

ee S

cienc

e?: I

deal

s and

Il-lu

sions

Subj

ectiv

e we

ll-be

ing

: mea

surin

g ha

ppin

ess,

suffe

ring,

and

oth

er

dim

ensio

ns o

f exp

erie

nce

Clin

ical L

abor

ator

y Ma

nage

men

t, Se

cond

Edi

tion

For t

he p

ublic

's he

alth

inve

stin

g in

a

heal

thie

r fut

ure

Intro

duct

ory

Econ

omet

rics

2012 2014 2015 2012 2012 2009 2013 2015 2012 2011

0500

10001500200025003000 2810

1052598 352 324 314 231 185 138 130

Num

ber

of B

itly

clic

ks

Education

Biology Medicine Sociology Biology Philosophy

Psychology

Medicine Medicine Economics

DOWNLOADS

Intro

duct

ion

to d

ocum

enta

ry

Socia

l wor

k va

lues

and

eth

ics

Lead

ersh

ip

Deat

h an

d sa

lvat

ion

in a

ncie

nt E

gypt

On G

reek

relig

ion

Tim

e m

anag

emen

t

Mark

etin

g to

mill

enni

als :

reac

h th

e la

rges

t and

mos

t infl

uent

ial g

ener

a-tio

n of

cons

umer

s eve

r

The

Amer

ican

way

of p

over

ty :

how

the

othe

r hal

f stil

l liv

es

The

AMA

Hand

book

of P

roje

ct

Mana

gem

ent

The

proj

ect m

anag

emen

t too

l kit

: 10

0 tip

s and

tech

niqu

es fo

r get

ting

the

job

done

righ

t

2010 2006 2014 2005 2011 - - 2013 2014 2010

0200400600800

100012001400

Dow

nlao

ds

Arts Sociology Business History History Business Business Sociology Business Business

PDF VIEWS

The

A-Z

of m

anag

emen

t con

cept

s and

m

odels

The

AMA

hand

book

of p

rojec

t man-

agem

ent

A Ha

ndbo

ok o

f Med

ia an

d Co

mm

uni-

catio

n Re

sear

ch

Stro

ke re

cove

ry a

nd re

habi

litat

ion

The

powe

r of e

nter

prise

-wid

e pr

oject

m

anag

emen

t

Deba

ting

the

deat

h pe

nalty

: sh

ould

Am

erica

hav

e ca

pita

l pun

ishm

ent?

: th

e ex

perts

on

both

side

s mak

e th

eir b

est

case

Viole

nt V

ideo

Gam

e Eff

ects

on

Child

ren

and

Adole

scen

ts: T

heor

y, R

esea

rch,

and

Pu

blic

Polic

y

Divo

rce

caus

es a

nd c

onse

quen

ces

Harri

son'

s man

ual o

f med

icine

Epile

psy

A to

Z a

con

cise

ency

clope

dia

2005 2006 2013 2009 2007 2004 2007 2006 2009 2009

05000

1000015000200002500030000 28436

19221 16894 15171 12991 12700 12337 11260 10977 10022

Num

ber

of P

DF

View

s

Business Business Business Medicine Business Sociology Psychology

Psychology

Medicine Medicine

MOST FREQUENTLY VIEWED & DOWNLOADED BOOKS

SOCIAL M

EDIA

TWITTER

The

Cons

trual

of S

patia

l Mea

ning

Enab

ling

Thin

gs to

Tal

k

Stem

Cel

ls

Fung

al D

iseas

e in

Brit

ain

and

the

Unite

d St

ates

185

0-20

00

Unde

rsta

ndin

g Ou

r Uns

een

Real

ity

Phys

ics fr

om th

e Ed

ge

Digi

tal H

uman

ities

Ped

agog

y: P

rac-

tices

, Prin

ciple

s and

Pol

itics

Valu

e-Fr

ee S

cienc

e?: I

deal

s and

Il-lu

sions

Wom

en a

nd C

ardi

ovas

cula

r Dise

ase

Berg

en-B

else

n 19

45

2013 2016 2013 2013 2015 2014 2015 2009 2014 2014

020406080

100120

Num

ber

of T

witt

er m

entio

ns

Philosophy

Computer Science

Medicine Medicine Psychology

Physics Education Philosoph

yMedicine History

GOOGLE+ SHARES A

Fram

ewor

k fo

r K-1

2 Sc

ienc

e Ed

uca-

tion:

Pra

ctice

s, Cr

ossc

uttin

g Co

n-ce

pts,

and

Core

Idea

s

Gulf

War

and

Hea

lth: V

olum

es 1

-9

Dete

rrenc

e an

d th

e de

ath

pena

lty

Heal

th IT

and

pat

ient

safe

ty b

uild

ing

safe

r sys

tem

s for

bet

ter c

are

Conf

ront

ing

com

mer

cial s

exua

l ex

ploi

tatio

n an

d se

x tra

ffick

ing

of

min

ors i

n th

e Un

ited

Stat

es

Wat

er re

use

: pot

entia

l for

exp

andi

ng

the

natio

n's w

ater

supp

ly th

roug

h re

use

of m

unici

pal w

aste

wate

r

Subj

ectiv

e we

ll-be

ing

: mea

surin

g ha

ppin

ess,

suffe

ring,

and

oth

er

dim

ensio

ns o

f exp

erie

nce

Crisi

s sta

ndar

ds o

f car

e a

syst

ems

fram

ewor

k fo

r cat

astro

phic

disa

ster

re

spon

se

Appr

oach

es fo

r eco

syst

em se

rvice

s va

luat

ion

for t

he G

ulf o

f Mex

ico a

fter

the

Deep

wate

r Hor

izon

oil s

pill

: in-

terim

repo

rt

The

grow

th o

f inc

arce

ratio

n in

the

Unite

d St

ates

: ex

plor

ing

caus

es a

nd

cons

eque

nces

2012 2010 2012 2012 2013 2012 2013 2012 2012 2014

020406080

100120

96

2911 7 5 4 4 3 3 3

Education

History Sociology SociologyMedicine Engineering

Psychology

Education

Engineering

Sociology

MOST FREQUENTLY MENTIONED IN SOCIAL MEDIA

CITATIO

NS

CITATIONS (VIA SCOPUS)

Intro

duct

ion

to M

eta-

Anal

ysis

The

Neur

opsy

chol

ogy

of A

nxie

ty: A

n en

quiry

into

the

func

tion

of th

e se

pto-

hipp

ocam

pal s

yste

m

Netw

orks

: An

Intro

duct

ion

Expl

orat

ions

in P

erso

nalit

y

Coch

rane

Han

dboo

k fo

r Sys

tem

atic

Revi

ews o

f Int

erve

ntio

ns: C

ochr

ane

Book

Ser

ies

Poly

mor

phism

in M

olec

ular

Cry

stal

s

Mem

bran

e Te

chno

logy

and

App

lica-

tions

Resp

onse

Tim

es: T

heir

Role

in In

fer-

ring

Elem

enta

ry M

enta

l Org

aniza

tion

Emer

ging

Adu

lthoo

d: T

he W

indi

ng

Road

from

the

Late

Tee

ns th

roug

h th

e Tw

entie

s

Mode

lling

Tra

nspo

rt

2009 2008 2010 2010 2008 2010 2012 2008 2014 2011

0500

10001500200025003000 2499 2312 2177

1611 1602 1458 14501013 968 945

Num

ber

of c

itatio

ns

Medicine Engineering

Psychology

Computer Science

Psychology

Medicine Chemistry

Psychology

Education

Engineering

REVIEWS & LINKS

WIKIPEDIA

The

Dino

saur

ia

The

epon

ym d

ictio

nary

of r

eptil

es

Syst

emat

ic Ba

cter

iolo

gy

The

Birth

of t

he P

ales

tinia

n Re

fuge

e Pr

oble

m R

evisi

ted

A m

ilita

ry h

istor

y of

Aus

tralia

Plac

e na

mes

of I

llino

is

Cont

inuu

m e

ncyc

lope

dia

of p

opul

ar

mus

ic of

the

world

.

Hand

book

of t

went

ieth

-cen

tury

lite

ra-tu

res o

f Ind

ia

The

com

plet

e di

nosa

ur

Wor

ld E

ncyc

lope

dia

of C

onte

mpo

rary

Th

eatre

2004 2011 2012 2004 2008 2009 2003 1996 2012 2014

0200400600800

1000 891 845

515 509

199 194 138 115 93 93

Num

ber

of li

nks

on W

ikip

edia

Geography

Biology Biology History History History Arts Literature Geography

Arts

AMAZON REVIEWS

Capi

talis

m a

nd Fr

eedo

m

The

Craf

t of R

esea

rch,

Thi

rd E

ditio

n

Hist

ory

of W

este

rn P

hilo

soph

y

Syria

n Ep

isode

s

A lit

tle h

istor

y of

the

world

Exce

l 201

0 Po

wer P

rogr

amm

ing

with

VB

A

Gend

er h

urts

: A fe

min

ist a

naly

sis o

f th

e po

litics

of t

rans

gend

erism

Build

ing

the

H Bo

mb

The

Fran

co-P

russ

ian

war:

The

Germ

an

conq

uest

of F

ranc

e in

187

0-18

71

Honk

y

2002 2008 2004 2009 2005 2013 2014 2015 2005 2002

0

100

200

300

400 334261 224 221 211

163 14683 75 72

Num

ber

of A

maz

on R

evie

ws

Economics

Education

Philosophy

History History Education

Sociology History History Arts

REVIEWS ON GOODREADS

The

grea

t Gat

sby

Prid

e an

d pr

ejud

ice

The

curio

us in

ciden

t of t

he d

og in

the

nigh

t-tim

e

The

pict

ure

of D

oria

n Gr

ay

The

Coun

t of M

onte

Cris

to

The

Oxfo

rd S

hake

spea

re: R

omeo

and

Ju

liet Ro

meo

and

Julie

t

Grea

t exp

ecta

tions

The

scar

let l

ette

r

Ham

let

North

ange

r Abb

ey

1998 2006 2012 2006 2008 2000 2004 2008 2007 2003 2006

01000020000300004000050000 45625 41970

31191

15869 13844 12177 12176 10722 9961 7294 7115

Num

ber

of R

evie

ws

on G

oodr

eads

MOST FREQUENTLY MENTIONED IN REVIEWS

CONCLUSIONS AND DISCUSSION There is no one single measure that can capture the

impact of books. Books are captured quite often in Altmetrics measures

such as social media, views, downloads and links. Generalization of subject areas and their impact

measures is difficult. It’s important for publishers to utilize social media to

raise awareness to the their books (including text books).

THANK YOU!