A New Measure of Knowledge Diffusion Stephen Carley; Alan Porter Georgia Tech

Our Interests Interest in research interdisciplinarity  Possible key to more creative research  Possible means to solve complex scholarly and societal problems Interest in research knowledge diffusion  Trans-disciplinary knowledge enrichment  Translation of research to clinical practice  Fostering science-based innovation

Measures Have devised metrics:  Integration score [how diverse are the research knowledge resources upon which an article (or body of research) draws?]  Specialization score [how diverse are the venues in which a body of research (e.g., the works of one author in one time period) appears?] Now adding -- Diffusion score [how diverse are the fields that cite an article (or body of research)?]

Heuristics of diversity (Stirling, 1998; 2007) (Rafols and Meyer, 2009) Diversity: Attribute of a system whose elements may be apportioned into categories Characteristics: Variety: Number of distinctive categories Balance: Evenness of the distribution Disparity: Degree to which the categories are different. Variety BalanceDisparity Herfindahl (concentration):  i p i 2 [** Shannon & Herfindahl do not include Disparity] Shannon (Entropy):  i p i ln p i Dissimilarity:  i d i Generalised Diversity (Stirling)  ij(i  j) (p i p j )  (d ij ) 

Integration Integration Defined : equivalently: Integration Range: 0 → 1 Integration Benchmarking: ~0.42 for much of modern scientific research (increasing moderately) where i = row, j = column, f = frequency, SC = Cited Web of Science Subject Category; cosine similarity of SC cross-citation (based on one year of Web of Science journal cross-citation)

Changes in Average Integration Scores over Time Porter, A.L., and Rafols, I. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics (2009):

Diffusion Scoring Example Paper 1Paper 2 Cited by : BiologyGeneticsMathMusicForestry In the example above Paper 2 will likely have a higher diffusion score than Paper 1 because there is greater heterogeneity among its citing subject categories [variety & disparity; not clear on balance].

Diffusion Scoring Diffusion Defined: where: p i = proportion of citing references corresponding to the SC i in a given paper s ij = cosine measure of similarity between SCs i & j for cross- citation in WOS Diffusion Range: 0 → 1 Diffusion Benchmarking: ~0.42 for much of modern scientific research [but somewhat higher for papers “out there” longer]

Mean Diffusion scores for all benchmark years combined Modest increase over time since publication

Diffusion Score Behavior Diffusion score behavior is more challenging to study than Integration: Calculating Diffusion score requires additional “citation searching” in Web of Science to retrieve the citing papers A paper’s Diffusion score is not fixed; over time it will accrue more citations Self-citation could affect Diffusion score We examined – for 5 of our 6 target Subject Categories -- minimal effect For Math – removing self-cites increased scores for 1995 publications by , until recent years [Since Math 1995 annual Diffusion scores are ~0.22, this is ~5-10% increase]

Self-Cites (1)

Self-Cites (2)

Diffusion Scores: Total vs. Annual Annual Diffusion may be of interest e.g., to pursue Chen’s hypothesis that broad + rapid diffusion may signal really impactful papers To characterize research knowledge diffusion patterns Calculation issue We use a threshold of at least 3 Citing SCs for a paper to get a Diffusion Score [could be 1 cite by a paper whose journal is associated with 3 SCs (rare)] For Total score calculation this is less a factor than for Annual scores

For most of the 1995 benchmarks, Diffusion scores increase steadily with time. Mathematics is an outlier. Mean Annual Diffusion Scores for 6 Subject Categories

Mean NeuroScience Diffusion scores for all 4 benchmark years

Diffusion Patterns Can we categorize research knowledge diffusion patterns? A first try here: Steady growth Staying power Late-bloomer Robust growth Next 4 slides illustrate for those 1995 publications in the 6 SCs that fit these patterns

Steady Growth Typology Data Source: 1995 benchmarks

Staying Power Typology Data Source: 1995 benchmarks

Late-Bloomer Typology Powerhouse articles (N’s are small) tend to have high Diffusion scores Data Source: 1995 benchmarks

Robust Growth Typology Data Source: 1995 benchmarks

Papers are citing more references over time -- does this exert upward pressure on Diffusion scores? What happens to the speed of diffusion over time? - Will electronic open source publishing boost Diffusion? How differently does research published in various Subject Categories diffuse? Chen: papers which are cited quickly and diffusely are more likely to be blockbusters Tool for tracking individual researchers or research groups’ influence Diffusion Issues

Taking a Look at the Research of Robert Nerem Institute Professor Emeritus and Director, Georgia Tech/Emory Center for Regenerative Medicine 240 Web of Science (WOS) publications, in 76 Journals, over 47 Years, and more than 8,000 Citations

Diffusion Scores v. Publication Age for Dr. Nerem’s Research Increasing Diffusion score from his early papers (high Age) to more recent ones

For more on Integration see:   Porter, Alan, Alex Cohen, David Roessner and Marty Perreault. "Measuring researcher interdisciplinarity." Scientometrics (2007):  Porter, Alan, David Roessner, and Anne Heberger. "How interdisciplinary is a given body of research?" Research Evaluation (2008):  Porter, Alan, and Ismael Rafols. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics (2009):

For more on Diffusion see:  Chen, Chaomei, and Diana Hicks. "Tracing knowledge diffusion." Scientometrics (2004):  Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., & Pellegrino, D. (2009). Towards an explanatory and computational theory of scientific discovery. Journal of Informetrics, 3(3),  Yu, G., Wang, M.Y., Yu, D.R. “Characterizing knowledge diffusion of Nanoscience & Nanotechnology by citation analysis.” Scientometrics, 84.1 (2010):  Carley, S. and Porter, A.L. (2011) "A Forward Diversity Index." Scientometrics, forthcoming.

Thank you!

Mean Biotech Diffusion scores for all 4 benchmark years

Mean EE Diffusion scores for all 4 benchmark years

Mean Math Diffusion scores for all 4 benchmark years

Mean Med-R&E Diffusion scores for all 4 benchmark years

Mean Phy-AMC Diffusion scores for all 4 benchmark years