1. Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences Kevin W. Boyack Mapping Examples Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under Contract DE-AC04-94AL85000.

2. 2 Macromodel: “Best” Map  Each dot is one journal  Journals group by discipline  Labeled by hand Boyack, K.W., Klavans, R., & Börner, K. (2005, in press). Mapping the backbone of science. Scientometrics.

3. 3 Modified from Boyack et al. by Ian Aliman, IU

4. 4 Macromodel: Structural Map  Clusters of journals denote disciplines  Lines denote strongest relationships between disciplines  Enables disciplinary diffusion studies  Enables comparison of institutions by discipline Boyack, K.W., Klavans, R., & Börner, K. (2005, in press). Mapping the backbone of science. Scientometrics.

5. 5 Paper-level Map

6. 6 Disciplinary S&T Model - Details  Uses combined SCIE/ SSCI/ISI Proceedings from 2003 –7445 journals, 1198 proceedings –journals, proceedings treated equivalently –Bib coupling of journals  Initial ordination and clustering of journals gave 852 clusters  Coupling counts reaggregated at the journal cluster level; ordination of journal clusters –(x,y) positions for each journal cluster, journal Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences

7. 7 Research Community Model - Details  Uses combined SCIE/ SSCI/ISI Proceedings from 2003 –997,775 papers from 8643 sources –Bib coupling of papers  Initial ordination and clustering of journals gave 117,433 clusters  Cluster positions calculated using journal positions from the disciplinary map Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences

8. 8

9. 9

10. 10 Accuracy

11. 11 Journal-level: Local Accuracy  For each similarity measure, journal pairs were assigned a 1/0 binary score if they were IN/OUT of the same ISI category  Accuracy vs. coverage curves were generated for each similarity measure  For each similarity measure, distances (in the VxOrd layouts) between journal pairs were calculated  Accuracy vs. coverage curves were generated for each re-estimated (distance) similarity measure  Results after running through VxOrd were more accurate than the raw measures  Inter-citation measures are best Similarity measures After VxOrd Klavans, R., & Boyack, K.W. (in press). Identifying a better measure of relatedness for mapping science. Journal of the American Society for Information Science and Technology.

12. 12 Journal-level: Regional Accuracy  For each similarity measure, the VxOrd layout was subjected to k- means clustering using different numbers of clusters  Resulting cluster/category memberships were compared to actual category memberships using entropy/mutual information method  Increasing Z-score indicates increasing distance from a random solution  Most similarity measures are within several percent of each other Boyack, K.W., Klavans, R., & Börner, K., (submitted). Mapping the backbone of science. Scientometrics.

13. 13 Paper-level: Local Accuracy  Two maps (current and reference), two measures (raw and modified cosine), two aggregation levels  For each similarity measure, paper pairs, ordered by distance on the map, were assigned a 1/0 binary score if they were IN/OUT of the same ISI category  Accuracy vs. coverage curves were generated for each similarity measure  K50 measures have high accuracy at high coverage Klavans, R., & Boyack, K.W. (under review). Quantitative evaluation of large maps of science. Scientometrics. Local Accuracy (Current Papers) K50 (aggregated) RawFreq (aggregated) K50 (paper) RawFreq (paper) 60% Local Accuracy 70% 60% Local Accuracy (%yes) 70% % Coverage Local Accuracy (Reference Papers) K50 (aggregated) RawFreq (aggregated) K50 (paper) RawFreq (paper) % Coverage 60% Local Accuracy (%yes) 70%

14. 14 Disciplinary Bias (Reference Papers) Disciplinary Bias Small[1999] RawFreq (paper) RawFreq (aggregated) K50 (paper) K50 (aggregated) Disciplinary Bias (Current Papers) Disciplinary Bias RawFreq (paper) RawFreq (aggregated) K50 (paper) K50 (aggregated) Paper-level: Disciplinary Bias  Two maps (current and reference), two measures (raw and modified cosine), two aggregation levels  Disciplinary bias measures effect of thresholding with coverage  K50 measures have lowest bias Klavans, R., & Boyack, K.W. (under review). Quantitative evaluation of large maps of science. Scientometrics.

15. 15 Cluster Size (Reference Papers) RawFreq (paper) K50 (paper) RawFreq (aggregated) K50 (aggregated) Log (Rank Size) Log (Size of Cluster) Cluster Size (Current Papers) Log (Size of Cluster) Log (Rank Size) K50 (paper) RawFreq (paper) RawFreq (aggregated) K50 (aggregated) Paper-level: Cluster Distributions  Two maps (current and reference), two measures (raw and modified cosine), two aggregation levels  Cluster size distributions – smaller clusters are usually better – chaining can create communities that are too large  K50 measures have fewer large clusters Klavans, R., & Boyack, K.W. (under review). Quantitative evaluation of large maps of science. Scientometrics.

16. 16 Paper-level: Text Analysis  Current map, K50 similarity  Multidocument summarization –cluster cohesiveness calculated using abstracts for all clusters –compared to values from random clusters  98.3% of the actual clusters of size 10 are more cohesive than random at p<0.0001 each

17. 17 Uses

18. 18 Worldwide S&T Circle size – number of topics Percent conference papers 0-25% 25-50% 50-75% 75-100% Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences

19. 19 Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences Circle size – number of topics Vitality (use of newer ideas) >10% more vital than world 0-10% more vital than world 0-10% less vital than world >10% less vital than world DOE Profile

20. 20 Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences Circle size – number of topics Vitality (use of newer ideas) >10% more vital than world 0-10% more vital than world 0-10% less vital than world >10% less vital than world Sandia Profile

21. 21 Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences Circle size – number of topics Vitality (use of newer ideas) >10% more vital than world 0-10% more vital than world 0-10% less vital than world >10% less vital than world A Specific University Profile

22. 22 Math CS Physics Chemistry Biology Engineering Medical Brain/Neuro Psych Earth Sciences Social Sciences Circle size – number of topics Vitality (use of newer ideas) >10% more vital than world 0-10% more vital than world 0-10% less vital than world >10% less vital than world Potential SNL/Univ Collaborations

23. 23 Identifying Opportunities/Threats Policy Economics Statistics Math CompSci Physics Biology GeoScience Microbiology BioChem Brain Psychiatry Environment Vision Virology Infectious Diseases Cancer MRI Bio- Materials Law Plant Animal Phys-Chem Chemistry Psychology Education Computer Tech GI (36 Research Communities that will impact GI Research… that GI Researchers are least likely to be aware of)

24. 24 Identifying Core Competency Policy Economics Statistics Math CompSci Physics Biology GeoScience Microbiology BioChem Brain Psychiatry Environment Vision Virology Infectious Diseases Cancer MRI Bio- Materials Law Plant Animal Phys-Chem Chemistry Psychology Education Computer Tech GI Funding patterns of the National Science Foundation (NSF)

25. 25 Identifying Core Competency Policy Economics Statistics Math CompSci Physics Biology GeoScience Microbiology BioChem Brain Psychiatry Environment Vision Virology Infectious Diseases Cancer MRI Bio- Materials Law Plant Animal Phys-Chem Chemistry Psychology Education Computer Tech GI Funding patterns of the National Institutes of Health (NIH)

26. 26 Identifying Core Competency Policy Economics Statistics Math CompSci Physics Biology GeoScience Microbiology BioChem Brain Psychiatry Environment Vision Virology Infectious Diseases Cancer MRI Bio- Materials Law Plant Animal Phys-Chem Chemistry Psychology Education Computer Tech GI Funding patterns of the US Department of Energy (DOE)

27. 27 Potential Uses of Science Maps  Overlays –Topic distribution –Opportunity/threat assessment –Core competency identification –Funding (amount) patterns –Impact patterns  Relationships –Interdisciplinary –Community level between disciplines

28. 28 Related Publications –Boyack, K. W., Klavans, R. & Börner, K. (2005, in press). Mapping the backbone of science. Scientometrics. –Klavans, R. & Boyack, K. W. (2005, in press). Identifying a better measure of relatedness for mapping science. Journal of the American Society for Information Science and Technology. –Boyack, K. W., & Rahal, N. (2005, in press). Evaluation of LDRD investment areas at Sandia. Technological Forecasting and Social Change. –Boyack, K. W., Mane, K. & Börner, K. (2004). Mapping Medline papers, genes and proteins related to melanoma research. IEEE Information Visualization 2004, 965-971. –Boyack, K. W. (2004). Mapping knowledge domains: Characterizing PNAS. Proceedings of the National Academy of Sciences 101(S1), 5192-5199. –Boyack, K. W., & Börner, K. (2003). Indicator-assisted evaluation and funding of research: Visualizing the influence of grants on the number and quality of research papers. Journal of the American Society for Information Science and Technology 54(5), 447. –Börner, K., Chen, C., & Boyack, K. W. (2003). Visualizing knowledge domains. Annual Review of Information Science and Technology 37, 179-255. –Werner-Washburne, M., Wylie, B., Boyack, K., Fuge, E., Galbraith, J., Fleharty, M., Weber, J., & Davidson, G.S. (2002). Concurrent analysis of multiple genome-scale datasets. Genome Research 12(10), 1564-1573. –Boyack, K. W., Wylie, B. N., & Davidson, G. S. (2002). Information visualization, human-computer interaction, and cognitive psychology: Domain visualizations. Lecture Notes in Computer Science 2539, 145-160. –Boyack, K. W., Wylie, B. N., & Davidson, G. S. (2002). Domain visualization using VxInsight for science and technology management. Journal of the American Society for Information Science and Technology, 53(9), 764-774. –Davidson, G. S., Wylie, B. N., & Boyack, K. W. (2001). Cluster stability and the use of noise in interpretation of clustering. Proc. IEEE Information Visualization 2001, 23-30. –Boyack, K.W., Wylie, B.N., Davidson, G.S. & Johnson, D.K., Analysis of patent databases using VxInsight. Presented at New Paradigms in Information Visualization and Manipulation 2000, McLean, VA, Nov. 10, 2000.