1. Measuring Scholarly and Public Impact: Let’s Talk Metrics
Rachel Borchardt Monday, May 16th, 2017
Conference for High-Impact Research

2. Why metrics? Quantifiable measures of research impact
Differentiate articles, journals, authors, institutions
Imperfect calculations of a fuzzy concept

3. How are metrics used? Evaluate researchers: Hiring Tenure/promotion
Merit awards
Grant funding
Benchmark programs and institutions
Image from AU Faculty Manual,

4. What are metrics Traditional Bibliometrics Based on citation counts
Emerging Altmetrics
Based on online tools
Image from Plum Analytics

5. Categories of bibliometrics
Article-level
Journal-level
Author-level
Institutional-level

6. Traditional bibliometrics for journals
Impact Factor and Citation Counts, created to measure
Journals and journal articles
Scholarly (journal) impact
Initially created for librarians, then largely adopted by STEM
Image from Journal Citation Reports (library database)

7. Some Impact Factor limitations
Retrospective measurement
Narrow measurement
Confined by what is indexed as a citation
Can’t be compared between disciplines
Other known issues: language, co-authorship, disproportionate article citation impact
Image from Essential Science Indicators (library database)

8. One way to correct – disciplinary context
Image from Journal Citation Reports (library database)

9. Contextual limitations – “Political Science” category
List from

10. Other major bibliometrics
Metrics from Web of Science: Eigenfactor, Immediacy Index, Cited Half-Life
Metrics from Elsevier/Scopus: SJR, SNIP, IPP, CiteScore
Metrics from Google Scholar: H5-index, H5-median
Image from

11. SJR comparison – “Political Science and International Relations”

12. H5-index comparison – Political Science
List from Google Scholar Metrics

13. Another alternative – citation distributions

14. Why altmetrics? Measures broader scholarly outputs
Books, presentations, data sets, software code, and more
More measures of scholarship
Views, downloads, shares, mentions in blogs, policy papers, and more
More immediate metrics
Measures beyond scholarly impact
Policy, public, and more

15. What are altmetrics?
Definition: The creation and study of new metrics based on the social web for analyzing and informing scholarship.
Altmetrics.org
My definition: The umbrella classification of non-citation based metrics. ;

16. Categories of altmetrics
Article-level [individual work]
Journal-level
Author-level
Institutional-level

17. Where do altmetrics come from?

18. How do we collect altmetrics?
Directly from the individual tools
From publishers (views, download data)
From (some) library databases
From scholarly networks
Through aggregating tools
Slideshare views
PLOS article metrics
Web of Science usage
ResearchGate metrics
Altmetric metrics

19. “Impact” and metrics Inherently fuzzy and subjective Attention
Engagement
Some correlate to bibliometrics; some don’t

20. Deciphering impact - ImpactStory

21. ImpactStory contextual elements

22. Deciphering impact from tools - Altmetric

23. Altmetric contextual elements

24. Impact correlations

25. Discipline-specific impact

26. Limitations of impact: example
objectives/topic/Adolescent-Health
+young+adult+health+in+the+United+States%22+site%3A.gov&oq= %22Trends+in+adolescent+and+young+adult+health+in+the+United +States%22+site%3A.gov&aqs=chrome..69i j0j8&sourceid=chr ome&ie=UTF-8

27. Current uses of altmetrics
External
Applying for funding, promotion/tenure
Showcasing achievements
Internal
Better understand engagement with research
Who, why, where

28. More on grant funders

29. Grant funders: NSF Broader Impacts

30. Grant funders: Wellcome Trust

31. Putting it all together - suggestions
Develop personal impact goals
Discipline-specific, research-specific, university-specific, etc.
Demonstrate achievement of those goals
Tell your story
Bibliometrics, altmetrics, and qualitative measures
Complementary roles
Track early and often!

32. Getting help Rachel Borchardt borchard@american.edu 202-885-3657
(I work year-round!)
Metrics guide,
Borrow a copy of my book

33. Questions?