LORRIE JOHNSON U.S. DEPARTMENT OF ENERGY OFFICE OF SCIENTIFIC AND TECHNICAL INFORMATION (OSTI) ICSTI TECHNICAL ACTIVITIES COORDINATING (TACC) MEETING OCTOBER 2013 Traditional Bibliometrics and Alternative Metrics in Research Program Assessment

Scientific and Technical Publication Metrics Scientific and technical publications, such as journal articles, are the primary products of scientific research, and the means by which new knowledge is created and transferred. Publication metrics can provide useful insight into research productivity, and into the impact and influence of research within particular fields of science.

Analysis of Joint Genome Institute Publications

Aggregate JGI Citation Data (from Web of Science) JGI publications were cited 51,590 times (through April 2013) Average citations per article = 56 H Index = 113 (113 articles have been cited more than 113 times) Average citations per year = 4,299 Citations for JGI publications are continuing to grow over time.

Top Journals for JGI Publications Comparison of Citation Rates for JGI Articles versus Other Articles in Journal Journal Titles# Articles % of Total (908) 5 Year Impact Factor Total Number of Citations for JGI Publications Average Number of Citations per JGI Publication Average Number of Citations for All Articles Published in Journal STANDARDS IN GENOMIC SCIENCES16819% JOURNAL OF BACTERIOLOGY728% PLOS ONE425% PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA415% APPLIED AND ENVIRONMENTAL MICROBIOLOGY394% NATURE364% SCIENCE364% GENOME RESEARCH293% BMC GENOMICS283% NUCLEIC ACIDS RESEARCH263% ENVIRONMENTAL MICROBIOLOGY202% ISME JOURNAL162% MOLECULAR BIOLOGY AND EVOLUTION162% NATURE BIOTECHNOLOGY162%

Citation Rates for 2008 JGI Articles in Nature Compared to Average Citation Rate

Citation Rates for 2009 JGI Articles in Science Compared to Average Citation Rate

Paper was published in 2006; citations peak in Indicators of Impact and Influence within Research Field Primary citations = 1,218 Secondary citations = 24,058 Research Areas of Citing Articles = 40 Unique Authors of Citing Articles = 5,149 Distinct Institutions Represented in Citing Articles = 1,112 Analysis of Individual Publication: The genome of black cottonwood, Populus trichocarpa…

Top 10 Journals Citing this Article Top 10 Research Areas of Articles Citing this Article Analysis of Individual Publication: The genome of black cottonwood, Populus trichocarpa…

Article Metrics provided by PLOS One: Other Alternative Metrics Measurements: Total article views = 23,384 Total PDF downloads = 4,403 4 Tweets 4 Facebook posts 15 bookmarks on CiteULike 20 cites on CrossRef Mendeley readers = 212 (89% in the Biological Sciences; 25% of readers from the U.S.) Cited by 17 articles in PubMed Central 46 citations in Google Scholar Altmetric score = 21 Alternative Metrics for JGI Article in PLOS One: “Direct Comparisons of Illumina vs. Roche 454 Sequencing Technologies on the Same Microbial Community DNA Sample”

NISO Alternative Assessment Metrics (Altmetrics) Project Sloan Foundation grant to undertake two-phase initiative to explore, identify, and advance standards and/or best practices for new assessment metrics, including:  Usage-based metrics  Social media references  Network behavioral analysis Will also explore potential assessment criteria for non- traditional research outputs, such as data sets, visualizations, software, and other applications.

NISO Alternative Assessment Metrics (Altmetrics) Project First phase – in person and virtual meetings to solicit input and collaboration from community. ICSTI members are encouraged to participate:  Washington, DC – December 11, 2013  Philadelphia, PA – January 23-24, 2014

NISO Alternative Assessment Metrics (Altmetrics) Project Outcome from Phase 1 – white paper describing outcomes, recommendations, and action items identified during the meetings. Phase 2 and Final Outcome – Published standards or recommended practices regarding alt metrics. For more information, see NISO project website:  Or contact Todd Carpenter: 