1 New Scholarly Communications Assessment Methods Andrea Michalek Co-Founder and President,

Answer the questions And tell the stories About your research

3 PlumX can help answer these questions you couldn’t answer before What impact has our research had in the past 12 months? How do we arm our researchers to better compete for funding? Can we measure non-STEM, non-article research?

4 Our Approach to Metrics Citations Usage Captures Mentions Social Media ALLmetrics

5 Journal citations are the current state of Scholarly Measure 5 Journal Impact Factor 3.161

6 2-5 years Idea Blog Post ? years Grant Conference 3-5 years Publication Video Citations Metrics timeline: From Idea to Impact It can take at least years from idea to a published peer-reviewed journal article Before PlumXAfter PlumX Due to the pace of scholarly publishing, it takes another years from the time the work is published to get to critical mass of citation counts From idea to measurable citation counts can take years Metrics available immediately citation counts presentation view share save reference bookmark PDF download click video play dataset download citation counts tweet

7 How can you help researchers get funding? Add more than just a list of publications to their grant applications –Give them real-time metrics about their impact Help determine where they should publish –Give metrics on what gets the most engagement Showcase more than just their journal articles –Get credit for books and other research output

8 Samples of PlumX Metrics Sources Amazon Bit.ly CrossRef Delicious Dryad dSpace EBSCO ePrints Facebook figshare Github Goodreads Google+ Medwave Mendeley PLOS PubMed Central Reddit Research Blogging Scopus SlideShare SourceForge Stack Overflow Twitter USPTO Vimeo Wikipedia Worldcat (OCLC) YouTube

9 All Types of Research Artifacts Articles Blog posts Book chapters Books Cases Clinical Trials Conference Papers Data Sets Figures Grants Interviews Letters Media Patents Posters Presentations Reports Source Code Theses / Dissertations Videos Web Pages

10 Metrics Categories USAGE (clicks, downloads, views, library holdings, video plays) CAPTURES (bookmarks, code forks, favorites, readers, watchers) MENTIONS (blog posts, comments, reviews, Wikipedia links) SOCIAL MEDIA (+1s, likes, shares, tweets) CITATIONS (PubMed Central, Scopus, patents)

11 USAGE Is anyone reading our work? Did anyone watch our videos? Usage is the #1 stat researchers want to know after Citation counts PlumX is the only product that includes Usage clicks, downloads, views, library holdings, video plays

12 Captures indicate that someone wants to come back to the work Early indicator of future citations bookmarks, favorites, readers, watchers CAPTURES

13 This category measures people truly engaging with your research Automatically uncover the conversations about your research Discover feedback, opinions, etc. blog posts, comments, reviews, Wikipedia links MENTIONS

14 Social media measures how well a researcher is promoting their work This is especially important for early career researchers to measure and understand Track the buzz and attention around your research output +1s, likes, shares, tweets SOCIAL MEDIA

15 Citations still a standard of long-term impact Including citations enables side-by-side analysis with other metrics categories PubMed Central, Scopus, patents CITATIONS

5 Categories in Action – Example Article Article level metrics are the building blocks for PlumX

17 Visualizing Impact: Plum Print Includes the 5 categories of metrics Circles dynamically change size based on metrics in each category

18 Each Plum Print looks different depending on the metrics for the artifact At-a-glance insight into relative impact Embed Plum Prints into your Institutional Repository

19 Researchers Need to Showcase their Impact Give them 5 categories of metrics to tell their stories –Not just a singe count with citations Tell the stories of research earlier –Citation counts take 3-5 years to accrue –PlumX gives comprehensive metrics in the first year Help them create better grant applications

20 Plum X Summary CATEGORIZEMEASUREANALYZE By metrics type By type of output By YOUR categories (e.g. department, subject, grant) PlumX platform automatically gathers metrics about each category for each artifact Through Dashboards and Analytics

CASE STUDIES University of Pittsburgh Pacifica Graduate Institute Oregon Health & Science University Autism Speaks

1 University of Pittsburgh CASE STUDY

23 Metrics Dashboards Researchers box: University of Pittsburgh uses PlumX to support their researchers. Narrow By: They also build dashboards for their digital collections, the 26 journals they publish, and for every school, department, and research institute Tabs of Different Artifacts: PlumX gathers metrics from over 20 types of research output.

24 PlumX Customer Data: Citations Lag Research output from 2013 and 2014 will have few citations.

25 Other Metrics Categories Fill the Gap Showcase impact in the past 12 to 24 months.

An article in Pitt’s institutional repository (IR). To incent researchers to contribute their output, Pitt delivers value by giving researchers feedback through article level metrics. 5 metrics categories provide insight into ALL versions of the article IR publisher site open access repositories Discover tweets from potential funders and from around the world

27 Metrics that update in real time provide a Feedback Loop Feedback loops help change behavior.

28 Comments from a Pitt researcher Back to Case Studies »

2 Pacifica Graduate Institute CASE STUDY

30 Graduate degrees in psychology Half the faculty is actively involved in research Books are the main output of faculty members Want to showcase faculty work

Books are their most common research artifact type

32 Books/Book Chapters Matter Citation counts do not do books justice –Never have & never will –We need new book metrics Important to Social Sciences and Humanities Books are often the seminal work from these faculty members

33 PlumX Book Specific Metrics How many libraries hold this book? How many Wikipedia articles reference this book? What are the reviews on Amazon about this book? What is the book’s average rating? How many people added this book to their bookshelf? What reviews have they written? How did they rate the book? How many abstract views, downloads, and clicks did this eBook get?

34 Susan Rowland, Ph.D. Chair of Engaged Humanities & the Creative Life Pacifica Graduate Institute

35 PlumX links directly to Amazon reviews and ratings

36 PlumX links directly to Goodreads See reviews and ratings See individuals who have read or want to read the book

37 Uncover the conversations the author was not even aware of.

38 Discover how many libraries hold a copy of the book Back to Case Studies »

3 Oregon Health & Science University CASE STUDY

40 OHSU – Tagging Medical Subjects Working collaboratively, the library and office of research have defined subjects to use in analyzing their research output They categorize their articles in PlumX by these subjects PlumX dashboards allow comparisons within and across these categories

OHSU – Metabolic Bone Disease Category The sunburst visualization easily identifies the top artifacts in this subject area

42 OHSU – Bringing Altmetrics into Grant Proposals NIH Biosketch –Where researchers highlight accomplishments when submitting grants to the NIH OHSU is helping their researchers add impact metrics to their NIH Biosketch Move beyond just a list of publications –They can make their researchers’ applications stand out Back to Case Studies »

4 Autism Speaks CASE STUDY

44 Funders are using PlumX Researcher from: Autism Speaks analyzes their grants by the following categories: By geography By institution By their major areas of funding By they type of grant Data Table: Looking at one grantee’s output, even for his research published in 2013 and 2014, PlumX metrics are available.

PlumX dashboards are created for every institution where Autism Speaks funds research.

46 Autism Speaks - Captures in 2014 Capture metrics are an indicator of what research happening now will likely be highly cited in the future Back to Case Studies »

There’s a new way to tell the story

48 PlumX can help answer these questions you couldn’t answer before What impact has our research had in the past 12 months? How do we arm our researchers to better compete for funding? Can we measure non-STEM, non-article research?

Thank You Andrea Michalek Find out more information at:

50 PlumX at a Glance Metrics Tracked Metrics from 5 different categories: usage, captures, social media, mentions & citations Usage and citations help provide full research impact Identifier matching Flexible ability to use multiple scholarly identifiers for tracking artifacts Can use URL as ID, e.g. policy docs or blogs Admin & Enterprise Features Designed from the beginning as an enterprise-level product Admin tools for supporting ongoing data maintenance Local customization Artifact types Ability to track virtually any artifact type, including books, book chapters, data sets, etc. Enables full picture of modern research ecosystem Widgets Embed metrics into your repository, journals, or websites at the group, artifact or researcher levels