Timothy D. Bowman, Ph.D. Candidate | 19th Nordic Workshop on Bibliometrics and Research Policy

CRC.EBSI.UMONTREAL.CA

WHY INVESTIGATE SCHOLARLY ACTIVITY IN SOCIAL MEDIA ?

- New technology allows for reassessment

and reevaluation of academia (Baldwin,

1998)

- Social media use provides insight into

customs and traditions (Greenhow, 2009)

- Social media use unveiling once

invisible backstage activity (Priem, 2014)

CRC.EBSI.UMONTREAL.CA

HAVE WE MOVED “BEYOND BIBLIOMETRICS”?

- We’ve moved beyond simply measuring citations

(Cronin & Sugimoto, 2014).

- New tools and data allow for new kinds of metrics

measuring wide array of indicators (Cronin, 2014)

- Electronic publishing magnifies a scholar’s

awareness of own performance (Wouters, 2014)

- Evaluation of bibliometric indicators by novices

allows for wide use of various ad hoc indicators

(Gingras, 2014)

CRC.EBSI.UMONTREAL.CA

WHY CONSIDER “ALTMETRICS” OR “INFLUMETRICS” OR

SIMPLY “SOCIAL MEDIA METRICS”?

- “Altmetrics” is the measure of scholarly communication

and dissemination within social media contexts (Priem &

Hemminger, 2010; Priem, Taraborelli, Groth & Neylon,

2010)

- Perhaps a better term is Influmetrics (Rousseau & Ye,

2013) or simply “social media metrics”?

- Social media indicators may measure immediate

assessment of academic impact and social impact

(Thelwall, Haustein, Larivière & Sugimoto, 2013)

- “Products,” not “publications” (Piwowar, 2013)

CRC.EBSI.UMONTREAL.CA

DO SCHOLARS USE TWITTER?

- 92% of Semantic Web scholars had Twitter account and

rated it as favorite for spreading scientific information

(Letierce, Passant, Decker, & Breslin, 2010)

- Total of 367 scholars reported increasing acceptance for

blogs and microblogs for consumption and

dissemination of scientific information (Gruzd, Goertzen,

& Mai, 2012)

- Scholars’ tweets tend to share information about (a)

professional discussions, (b) network with others, (c)

offer help / request help, (d) call attention to other social

media involvement, and (e) personal discussions, and (f)

impression management (Veletsianos, 2012)

CRC.EBSI.UMONTREAL.CA

DO SCHOLARS USE TWITTER? (CONT.)

- 43% scholars at 2012 STI Conference using

Twitter; it was used privately and professionally,

to distribute professional information, and to

improve visibility (Haustein et al., 2013)

- 80% DH scholars ranked Twitter as relevant for

consumption and 73% for dissemination of DH

information (Bowman et al., 2013)

- Differences by discipline found regarding the

way scholars used Twitter (Holmberg &

Thelwall, 2014)

CRC.EBSI.UMONTREAL.CA

RESEARCH QUESTIONS

1. What differences exist between the

tweeting behavior of scholars in the

natural and social sciences?

2. What kind of relationships exists

between tweeting and publication

behavior?

3. How does Twitter affordance use differ

across disciplines?

CRC.EBSI.UMONTREAL.CA

WHAT DATA IS IN THIS SAMPLE?

- 16,862 Associate, Assistant, and Full professors from webpages

at 62 AAU-member universities

- The faculty belonged to either Physics, Biology, Chemistry,

Computer Science, Philosophy, English, Sociology, or

Anthropology departments.

- 60 of the 62 universities rank in the top 125 according to 2014

CWTS Leiden Ranking

- Survey sent January and February 2014 with a response rate of

8.5% (1,910 responses)

- Of these responders, 32% (615) reported having at least one

Twitter account

- 289,934 tweets of 585,879 from 445 accounts (391 scholars)

were collected.

CRC.EBSI.UMONTREAL.CA

HOW WAS THE DATA COLLECTED?

- Twitter API, Local WoS Database, Manual

cleaning of authors

- Twitter:

- tweets, # of tweets, followers, friends, retweets,

created date

- affordances: @mention, #hashtag, URLs, media,

symbols, retweets

- WoS

- publications, citation averages

ALL 1,910 SURVEY RESPONDENTS :: HAVE TWITTER ACCOUNTS?

42.65%

36.42%38.89%

40.82%

24.96%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Less than 1Year

1 to 3 Years 4 to 6 Years 6 to 9 Years 10 Years ofMore

I'm not

38.10%

45.09%

38.27%34.31% 35.75%

29.68%26.58%

19.81%16.34%

5.26%2.38%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

26 to 30years

31 to 35years

36 to 40years

41 to 45years

46 to 50years

51 to 55years

56 to 60years

61 to 65years

66 to 70years

71 to 75years

Over 75years

by AGE

by ACADEMIC AGE

33.33%

29.11%

40.38%

25.00%

29.11%

50.00%

28.00%

0%

10%

20%

30%

40%

50%

60%

AmericanIndian /Native

American

Asian Black /African

American

Hispanic /Latino

White /Caucasian

PacificIslander

Other

by ETHNICITY

28.10% 27.52%

37.46% 36.90%

20.71%

50.00%

28.99%

23.64%

0%

10%

20%

30%

40%

50%

60%by DISCIPLINE

ONLY 391 SCHOLARS WITH TWITTER ACCOUNTS :: MEAN OF TWEETS PER DAY

1.06

0.53

1.96

1.41

0.670.52

0.73

1.18

by DEPARTMENT by GENDER1.14

0.80

1.02

Other Female Male

N=232

SD=2.3

N=122

SD=2.1

N=3

0.89

1.11

1.39

0.670.85

I'm Not 10 Yearsor More

7 to 9Years

4 to 6Years

1 to 3Years

Lessthan 1Year

by ACADEMIC AGE

N=2N=207

SD=2.4

N=53

SD=2.2

N=35

SD=2.6

N=39

SD=0.9

N=21

SD=1.1

0.92

0.98

1.03

Professor AssociateProfessor

AssistantProfessor

by PROFESSIONAL TITLE

N=116

SD=2.1

N=116

SD=1.7

N=156

SD=2.9

0

50

100

150

200

250

300

350

400

450

500

0 2 4 6 8

Physics

BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO # OF ARTICLES

0

2

4

6

8

10

12

0 5 10 15

Anthropology

0

5

10

15

20

25

0 5 10 15 20 25

Philosophy

0

10

20

30

40

50

60

0 5 10 15 20

Sociology

Art

icle

s

N=40 N=30

N=66

Tweets per Day

N=19

R² = 0.0133

0

1

2

3

4

5

6

0 5 10 15 20

English

BY DISCIPLINE :: RELATIONSHIP OF MEAN TWEETS PER DAY TO ARTICLES (CONT.)

R² = 0.0291

0

5

10

15

20

25

30

35

40

45

0 2 4 6 8 10

Biology

R² = 0.0118

0

5

10

15

20

25

30

0 1 2 3 4

Chemistry

R² = 0.0026

0

10

20

30

40

50

60

70

0 5 10 15 20

Computer Science

Art

icle

s

N=82N=73

N=20N=40

Tweets per Day

SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY MEAN OF IAC AVERAGE

0

0.5

1

1.5

2

2.5

0 5 10 15

Anthropology

0

5

10

15

20

25

0 5 10 15 20

English

0

1

2

3

4

5

6

7

0 5 10 15 20 25

Philosophy

0

1

2

3

4

5

6

7

8

0 5 10 15 20

Sociology

N=30 N=19

N=82N=66

SCHOLARLY IMPACT? :: MEAN TWEETS PER DAY BY SUM OF IAC AVERAGE

R² = 3E-05

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 2 4 6 8 10

Biology

R² = 0.0126

0

1

2

3

4

5

6

7

8

0 1 2 3 4

Chemistry

R² = 0.0089

0

2

4

6

8

10

12

14

16

18

0 2 4 6 8

Physics

R² = 1E-05

0

2

4

6

8

10

12

14

16

0 5 10 15 20

Computer Science

N=61N=73

N=20N=40

MEAN PERCENTAGE OF TWEETS WITH AFFORDANCE PER PERSON BY DEPARTMENT

AND MEAN OF REWTEETS BY COLLECTED TWEETS

6.41%

7.44%

0%

1%

2%

3%

4%

5%

6%

7%

8%

HASHTAGS

Anthropology

Biology

Chemistry

Computer Science

English

Philosophy

Physics

Sociology

16.28%

20.06%

0%

5%

10%

15%

20%

25%

MENTIONS

Anthropology

Biology

Chemistry

Computer Science

English

Philosophy

Physics

Sociology

0.72%

1.16%

0.25%

1.12%

0.53%

1.69%

0.77%

1.09%

0%

1%

2%

URLs

Anthropology

Biology

Chemistry

Computer Science

English

Philosophy

Physics

Sociology353

3291

0

500

1,000

1,500

2,000

2,500

3,000

3,500

RETWEETSAnthropology

Biology

Chemistry

Computer Science

English

Philosophy

Physics

Sociology

CRC.EBSI.UMONTREAL.CA

SUMMARY

• As expected, the data reflected differences of those who reported

having Twitter accounts based on academic age and actual age.

• Of the 391 scholars (445 Twitter accounts) that were collected, the

data did reflect differences in mean tweets per day based on gender,

discipline, and academic age and title

• Finally, it was found that the data reflected no strong relationships

between mean tweets per day and publication output or

• There was no real relationship between average citations and mean

tweets per day (scholarly impact)

• The data did reflect small differences in affordance use by discipline,

especially the differences in retweets but theses differences in

retweets are not an accurate representation of the retweets by the

scholar

CRC.EBSI.UMONTREAL.CA

ONGOING WORK

• Further analysis of retweets needed attempting to focus solely on

retweets made by the scholars themselves

• Using linguistic tools, the text of the 289,934 tweets will be used to

compare terms used in tweets with article titles at the level of the

scholar and discipline

• A social network analysis will be completed reflecting the mentions

used in tweets at the scholarly and discipline levels

• A closer examination of the actual affordances (unique hashtags,

unique URLs, unique mentions) used

• A categorization of tweets as either personal or professional by

Turkers

• A general discussion on what these social media metrics are actually

measuring including any correlations between social media use and

publication activity

CRC.EBSI.UMONTREAL.CA

THANK YOU

This work was partially funded by a grant

by The Alfred P. Sloan foundation

and a Canada Research Chair grant

DO YOU HAVE ANY QUESTIONS?

REFERENCESBaldwin, R. G. (1998). Technology’s Impact on Faculty Life and Work.

New Directions for Teaching and Learning, (76), 7–21.

doi:10.1002/tl.7601

Bowman, T. D., Demarest, B., Weingart, S. B., Simpson, G. L.,

Lariviere, V., Thelwall, M., & Sugimoto, C. R. (2013). Mapping DH

through heterogeneous communicative practices. In Digital

Humanities 2013. Lincoln, NE.

Cronin, B. (2014). Scholars and scripts, spoors and scores. In B.

Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing

multidimensional indicators of scholarly impact (pp. 3-22). Cambridge,

Mass.: MIT Press.

Cronin, B. & Sugimoto, C.R. (2014). Preface. In B. Cronin & C. R.

Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional

indicators of scholarly impact (pp. vii). Cambridge, Mass.: MIT Press.

Gingras, Y. (2014). Criteria for evaluating indicators. In B. Cronin & C.

R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing

multidimensional indicators of scholarly impact (pp. 109-126).

Cambridge, Mass.: MIT Press.

Greenhow, C. (2009). Social scholarship: applying social networking

technologies to research practices. Knowledge Quest, 37(4), 42–47.

Retrieved from

http://aasl.metapress.com/index/r282223126950757.pdf

Gruzd, A., Goertzen, M., & Mai, P. (2012). Survey results highlights:

Trends in scholarly communication and knowledge dissemination (p.

10). Retrieved from http://socialmedialab.ca/?p=4308

Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., &

Terliesner, J. (2013). Coverage and adoption of altmetrics sources in

the bibliometric community. arXiv, 1–12. Digital Libraries. Retrieved

from http://arxiv.org/abs/1304.7300

Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter

scholarly communication. Scientometrics. doi:10.1007/s11192-014-

1229-3

Understanding how Twitter is used to spread scientific messages. In

Web Science Conference. Raleigh, NC.

Moran, M., Seaman, J., & Tinti-Kane, H. (2011). Teaching, learning,

and sharing: How today’s higher education faculty use social media.

Piwowar, H. (2013). Altmetrics: Value all research products. Nature,

493(159). doi:10.1038/493159a

Priem J., & Hemminger B.M. (2010) Scientometrics 2.0: Toward new

metrics of scholarly impact on the social web. First Monday 15.

Available:

http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2

874/257. Accessed 2011 December 7.

Priem, J., Taraborelli, D., Groth, P., Neylon, C. Alt-metrics: a

manifesto. 2010. Available from http://altmetrics.org/manifesto/

Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.),

Beyond bibliometrics: Harnessing multidimensional indicators of

scholarly impact (pp. 263–288). Cambridge, Mass.: MIT Press.

Rousseau, R., & Ye, F. (2013). A multi-metric approach for research

evaluation. Chinese Science Bulletin, 58(3290), 1–7.

doi:10.1007/s11434-013-5939-3

Thelwall M., Haustein S., Larivière V., Sugimoto, C.R. (2013) Do

Altmetrics Work? Twitter and Ten Other Social Web Services. PLoS

ONE 8(5): e64841. doi:10.1371/journal.pone.0064841

Veletsianos, G. (2012). Higher education scholars’ participation and

practices on Twitter. Journal of Computer Assisted Learning, 28(4),

336–349. doi:10.1111/j.1365-2729.2011.00449.x

Wouters, P. (2014). The citation: From culture to infrastructure. In B.

Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing

multidimensional indicators of scholarly impact (pp. 47–66).

Cambridge, Mass.: MIT Press.

0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0%

Boston University

Brandeis University

Brown University

California Institute of Technology

Carnegie Mellon University

Case Western Reserve University

Columbia University

Cornell

Duke University

Emory University

Georgia Institute of Technology

Harvard

Indiana University

Iowa State

Johns Hopkins

McGill

Michigan State University (1964)

MIT

New York University

Northwestern

Princeton University (1900)

Purdue University (1958)

Rice University (1985)

Rutgers, The State University of New Jersey (1989)

Stanford University (1900)

Stony Brook University-State University of New York (2001)

Texas A&M University

The Ohio State University (1916)

The Pennsylvania State University (1958)

The University of Chicago

The University of Iowa (1909)

The University of Kansas

The University of North Carolina at Chapel Hill

The University of Texas at Austin

The University of Wisconsin-Madison

Tulane University

University at Buffalo, The State University of New York (1989)

University of Arizona

University of California, Berkeley

University of California, Davis

University of California, Irvine

University of California, Los Angeles

University of California, San Diego

University of California, Santa Barbara

University of Colorado Boulder

University of Florida

University of Illinois at Urbana-Champaign (1908)

University of Maryland

University of Michigan

University of Minnesota

University of Missouri-Columbia

University of Oregon

University of Pennsylvania

University of Pittsburgh

University of Rochester

University of Southern California

University of Toronto

University of Virginia

University of Washington

Vanderbilt University

Washington University in St. Louis

Yale University

University of Maryland (3.32%)

University of Wisconsin-Madison (4.85%)

Indiana University (4.08%)

APPENDIX: UNIVERSITY DISTRIBUTION

CRC.EBSI.UMONTREAL.CA

APPENDIX: 62 AAU-MEMBER UNIVERSITIES

Boston University, Brandeis University, Brown University, California Institute of Technology, Carnegie Mellon University, Case Western Reserve University, Columbia University, Cornell, Duke University, Emory University, Georgia Institute of Technology, Harvard, Indiana University, Iowa State, Johns Hopkins, McGill, Michigan State University, MIT, New York University, Northwestern, Princeton University, Purdue University, Rice University, Rutgers, The State University of New Jersey, Stanford University, Stony Brook University-State University of New York, Texas A&M University, The Ohio State University, The Pennsylvania State University, The University of Chicago, Tulane University, University at Buffalo, The State University of New York, University of Arizona, University of California, Berkeley, University of

California, Davis, University of California, Irvine, University of California, Los Angeles, University of California, San Diego, and University of California, Santa Barbara ,The University of Iowa, The University of Kansas, The University of North Carolina at Chapel Hill, The University of Texas at Austin, The University of Wisconsin-Madison, University of Colorado Boulder, University of Florida, University of Illinois at Urbana-Champaign, University of Maryland, University of Michigan, University of Minnesota, University of Missouri-Columbia, University of Oregon, University of Pennsylvania, University of Pittsburgh, University of Rochester, University of Southern California, University of Toronto, University of Virginia, University of Washington, Vanderbilt University, Washington University in St. Louis, Yale University