Implementation of Citation Count Metrics

H. K. (Rama) RamapriyanNASA/GSFC

Metrics Planning and Reporting (MPAR) WG

9th Earth Science Data Systems Working Group

New Orleans, LA

October, 2010

Citation Count Metrics – Where We Are

At the October, 2009 MPARWG meeting, citation count metrics were discussed, and Deborah Smith presented her project’s approach to obtaining a citation count.

The objective of the citation count metrics is to obtain a better measure of user satisfaction with projects’ products and services and enable a better assessment of their contribution to the NASA science and applications programs and Earth science research in general.

Citations of projects’ data or products in peer-reviewed research publications and other publications provide a direct measure of the value of the projects’ contribution to research and applications.

The MPARWG developed a recommendation for a citations count metric.

After the formal MPARWG review process, on April 6, 2010, the final recommendation was adopted and on April 20, 2010, sent to Martha Maiden, NASA HQ, for review and approval.

NASA HQ approval was received in October, 2010.

Next step: Implementation – the topic of our discussion today.

Definition of the Citation Count Metrics

The count of the number of times a project’s data or products is cited by researchers in peer-reviewed and other publications.

Projects will report (using the MCT) two counts: 1) a count of citations in peer-reviewed publications, and 2) a count of citations in any other publications, conference or

workshop proceedings, posters, online publications, abstracts, etc. Note: a ‘count’ is recorded for each publication that contains one or

more references to a project’s work (e.g. a project’s publication(s) or product(s)).

Reporting of the counts is voluntary, on a ‘best effort’ basis, at an interval to be chosen by the project, with semi-annual or annual reporting recommended.

In addition to the counts, the project will provide the actual citations themselves.

Each project will use, and provide a description of, its own methodology for collecting the citations counts and strive for consistency from one reporting period to the next.

Reports on the citation metric will be provided to NASA Headquarters at least annually.

Citation Metrics – Implementation Plan for FY2011 Objective for FY2011 – Try It! Develop some experience, lessons learned,

and share among the projects at the MPARWG meeting in October 2011.

Projects: To have citations to report, a project’s products must have been

available long enough for researchers to have done their work and published their results (with the appropriate citations).

This probably limits the MEaSUREs projects that could report citation metrics in FY2011 to the seven REASoN continuations.

MPARWG requests that the projects (that can) develop an approach and conduct one count / collection of citations by April, 2011.

Projects then provide a description of their approach, the two citation counts (via the MCT), and the actual citations to Greg Hunolt.

Greg Hunolt will put together a package of the individual project items and provide the package to the MPARWG Co-Chairs, Rama and Clyde. With their approval, Greg will distribute the package to the projects.

At the October, 2011 ESDSWG meeting, the MPARWG will review the results and lessons learned from the FY2011 effort, and determine how to proceed in FY2012 and produce the first annual report to NASA Headquarters.

MCT / E-Books Team: By April 1, 2011, implement and test changes to the MCT to permit

collection of the two citation count metrics. Also implement as needed changes to E-Books to allow entry and

retrieval of the actual citations.