1

How to Evaluate Research Impact of

Journals

2

Why Journal Impact?

Librarians - can support selection or removal of journals from their collections, and determine how long to keep each journal in the collection before archiving it.

Publishers and Editors - can determine journals’ influence in the marketplace and review editorial functions.

Authors - can identify the most appropriate, influential journals in which to publish, as well as confirm the status of journals in which they have published.

Professors and Students - can discover where to find the current reading list in their respective fields.

3

Journal Evaluation Tools

Ongoing quest for discipline-specific ranking for measuring ‘quality’.

Some journal evaluation tools: Journal Citation Reports (Impact factor) Web of Science (h-index) Scopus (h-index) SCImago Journal & Country Rank (SJR) Other tools can be found at the Library website

http://www.lib.ied.edu.hk/research/journalmetrics/index.html

HKIEd Research Information Core Hub (RICH) https://oraas0.ied.edu.hk/rich/web/search_publication.jsp

4

Journal Citation Reports (JCR) Journal Citation Reports® offers a systematic,

objective means to critically evaluate the world's leading journals, with quantifiable, statistical information based on citation data. By compiling articles' cited references, JCR Web helps to measure research influence and impact at the journal and category levels, and shows the relationship between citing and cited journals. Available in Science and Social Sciences editions.

Available at Library website: http://www.lib.ied.edu.hk/index_e-resources.php

5

JCR is used to …

Discover highest-impact journals Compare a custom selection of journals Analyze self-citations Identify review journals Identify impact factor trend Develop and manage journal collections

6

Access JCR

7

Access JCR (Social Sciences Edition)

8

Access JCR (Social Sciences Edition)

9

Access JCR (Social Sciences Edition)

10

Access JCR (Social Sciences Edition) Search

Criteria

Metrics

Journal Title

11

Access JCR (Social Sciences Edition) Click on the abbreviated

journal title to view the full journal record

12

JCR Year (year under review) Total Cites Impact Factor Immediacy Index Cited Half-Life Citing Half-Life

Definition of the Metrics

13

JCR Year and Total Cites

1. JCR Year: The year you select is the JCR year. All of the data that you see for journals and subject categories come from journal data published in that year. e.g. the current JCR year is 2009

2. Total Cites: The total number of times that a journal has been cited by all journals included in the database in the JCR year.

14

Journal: ACADEMY OF MANAGEMENT JOURNAL 

Total Cites Calculation

Total Cites in 2009 of citations published in a particular year:

41+124+654+612+711 … + 8846 = 15082

15

Impact Factor

3. Impact Factor: It is the average number of times articles from the journal published in the past two years have been cited in the JCR year. (出版當年前 2年該期刊所出版之文獻在今年被引用之平均次數 )

e.g. The journal XYZ has an 2009 impact factor of 2.0 means that, on average, the articles published in 2007 and 2008 was quoted in 2009 an average of 2 times.

16

Impact Factor =Cites in 2009 to 2007 or 2008 papers

Papers published in 2007 or 2008

The impact factor is calculated by dividing the number of citations in the JCR year by the total number of articles published in the two previous years.

Impact Factor CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

17

Implications of Impact Factor Tell us how frequently has the average article

in a journal been cited in a particular year. Tell us something about a journal as a whole

e.g. the extent to which its recently published papers were cited in a given year.

Impact factor > 1 implied a journal is frequently cited

Higher citations rate means your article has higher chances of getting cited or read by researchers.

Tells us NOTHING concrete about any specific paper or specific author.

18

Immediacy Index

3. Immediacy Index: Immediacy Index measures the average number of times that an article, published in a specific year within a specific journal, is cited over the course of the same year.(出版當年該期刊在當年被引用之平均次數 )

e.g. The journal XYZ contained 100 articles in 2009. These articles were in the same year quoted 1000 times, i.e. each article was quoted on average 10 times (Immediacy Index = 10).

19

The immediacy index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Immediacy Index CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

20

Implications of Immediacy Index Measures how quickly the average

article in a journal is cited Tells you how often articles published

in a journal are cited during their year of publication

21

Cited Half-Life

5. Cited Half-Life – It measures the number of years, going back from the current year, that account for half the total citations received by the cited journal in the current year.(該期刊在最新出版年當中 50%被引用文獻都集中在最近回溯年份 )

e.g. In JCR 2009 the journal XYZ has a cited half-life of 7.0. This means that articles published in XYZ between 2003-2009 (inclusive) account for 50% of all citations to articles from that journal in 2009.

* Only journals cited 100 or more times in the JCR year have a cited half-life.

22

Total cites in 2009: 41+124+654+612+ … +8846 = 15082

50% of the total cites in 2009 = 15082/2 = 7541

Total cites from 2009 to 2000 = 6236 (Cumulative % = 41.35%)

Conclusion: 50% of all citations from this journal cited in the year 2009 were referred to articles published from over ten years.

Cited Half-Life CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

Cited Journal data show how many citations a journal received in the JCR year.

23

Implications of Cited Half-Life You use the Journal Cited Half Life to see if articles from a journal

that were published a long time ago are still being cited. This shows you if the journal has a good track record and was producing good articles in the past. A primary research journal might have a longer cited half-life than a journal that provides rapid communication of current information.

The Journal Cited Half Life may be useful to library staff in order to carry out collection management. If a journal has a low Cited Half Life it means that older papers are not being cited as much and it may be a reason for not binding these into volumes and archiving them.

Dramatic changes in cited half-life over time may indicate a change in a journals format. Studying the half-life data of the journals in a comparative study may indicate differences in format and publication history.

24

Citing Half-Life5. Citing Half-Life – identifies the number of years from th

e current year that account for 50% of the cited references from articles published by a journal in the current year. (該期刊在最新出版年當中 50%引用其他期刊文獻都集中在最近回溯年份 )

e.g. In JCR 2009, the journal Food Biotechnology has a citing half-life of 9.0. That means that 50% of all articles cited by articles in Food Biotechnology in 2009 were published between 2001 and 2009 (inclusive).

* Only journals that publish 100 or more cited references have a citing half-life.

25

Total citing in 2009: 40+106+227+247+ … +316 = 5194

50% of the total cites in 2009 = 5194/2 = 2597

Total cites from 2009 to 2000 = 2374 (Cumulative % = 45.71%)

Conclusion: 50% of all citations citing from this journal in the year 2009 were referred to articles published from over ten years.

Citing Half-Life CalculationJournal: ACADEMY OF MANAGEMENT JOURNAL 

Citing journal data show how many citations a journal made to other journals (including itself) in the JCR year.

26

Implications of Citing Half-Life A publisher may use this number to

adjust editorial policies to compete in different market segments.

27

Eigenfactor Score & Article Influence Score

Eigenfactor Score The Eigenfactor™ score of a journal is an estimate of the pe

rcentage of time that library users spend with that journal. The Eigenfactor™ Score calculation is based on the numbe

r of times articles from the journal published in the past five years have been cited in the JCR year, but it also considers which journals have contributed these citations so that highly cited journals will influence the network more than lesser cited journals. References from one article in a journal to another article from the same journal are removed, so that Eigenfactor Scores are not influenced by journal self-citation.

http://www.eigenfactor.org/methods.htm http://www.sciencenet.cn/m/user_content.aspx?id=217750http://www.eigenfactor.org/methods.pdf

28

Eigenfactor Score & Article Influence Score

Article Influence Score The Article Influence determines the average influence of a

journal's articles over the first five years after publication. It is calculated by dividing a journal’s Eigenfactor Score by the number of articles in the journal, normalized as a fraction of all articles in all publications. This measure is roughly analogous to the 5-Year Journal Impact Factor in that it is a ratio of a journal’s citation influence to the size of the journal’s article contribution over a period of five years.

The mean Article Influence Score is 1.00. A score greater than 1.00 indicates that each article in the journal has above-average influence. A score less than 1.00 indicates that each article in the journal has below-average influence.

29

Eigenfactor Score & Article Influence Score

EF:Cites in {2009} to items published in 2004-2008, also take into account:• estimate of the percentage of time that library users spend with that journal.• eliminate journal self-citation.• considers which journals have contributed these citations so that highly cited journals will influence the network more than lesser cited journals.• calculation included both Science and Social Science journals.

AI:• A score greater than 1.00 indicates that each article in the journal has above-average influence. A score less than 1.00 indicates that each article in the journal has below-average influence.

ai is the number of articles published by journal i over the five-year target window, divided by the total number of articles published by all source journals over the same five-year window. Notice that a is thus normalized to sum to 1 and it’s i-th entry specifies the fraction of all published articles that come from journal i.

30

Mark and Export Records

31

Mark and Export Records

Select record and click “Update Marked List”

32

Mark and Export RecordsClick “Marked List”, then select “Save to

File”

33

Journal Selection Process

1. Basic Journal Publishing Standards : timeliness, editorial conventions, English Language Bibliographic Information, peer review

2. Editorial Content : subject, specific category, compared with other similar editorial content.

3. International Diversity : reflect the global context of scholarly research

4. Citation Analysis : expert use of citation data help identify influential useful publications

34

Using Journal Citation Reports Wisely Should not depend solely on citation data in j

ournal evaluations.

Other conditions also can influence citation rates such as language, journal history and format, publication schedule, and subject specialty.

http://admin-apps.isiknowledge.com/JCR/help/h_using.htm

35

Thank You!