Scientometrics and the Diplomacy of Science R. D. Shelton and Lance Miller WTEC PICoSEP 2011 Pyongyang

Outline Background: Science Diplomacy ∩ Scientometrics Scientometric assessment as a science of science Quantitative: bibliometrics Qualitative: peer review Input-output models (my research) Enhances scientometrics as a predictive science Finds policy levers Scientometrics in the DPRK

The science of measuring and analysing science. Pioneers: Derek J. de Solla Price (1963) and Eugene Garfield--creator of the Science Citation Index (1955)… Methods: qualitative (peer review), quantitative (bibliometrics), computational (mapping, etc.). Journals: Scientometrics, Journal of the American Society for Information Science and Technology, Journal of Informetrics, … Professional Societies: International Society for Scientometrics and Infometrics (ISSI) founded in Conferences: International Conference of ISSI, International Conference on Science and Technology Indicators, … Sponsors: universities for research and assessments, and from governments to guide investments in science. Snapshot of scientometrics

Snapshot of Science Diplomacy Science diplomacy is the use of scientific collaborations among nations to address the common problems facing 21st century humanity and to build constructive international partnerships. (Wikipedia) The UK Royal Society and the US AAAS (2010) partitioned it as: Science in Diplomacy Diplomacy for Science Science for Diplomacy (most relevant here) Scientific cooperation can improve international relations. By its nature, science is borderless and requires the review, and evaluation to achieve validity, regardless of where peers reside. Scientists and engineers have traditionally worked with international colleagues on issues of common interest. These scientific exchanges can contribute to coalition-building and political dialogues between nations.

These diplomats’ and scientists’ viewpoints need broadening This view that more science is better regardless of where it done is typical of the AAAS, which is an international organization, not a national one. That is true for some applications, but not all. Curing cancer would be a global good regardless of who does it. But economic competitive advantage and national security advantages may accrue mainly to the nation whose scientists make the discoveries and innovations. To be scientifically realistic, we have to admit that there is a international competition aspect to science, not just a cooperative one.

Science Diplomacy ∩ Scientometrics What does scientometrics offer to science diplomacy? Encouraging international collaborations Measurement of international competition motivates better performance—nations often set such goals. Scientometrics itself is an international community of scholars

Methods of scientometric assessment Qualitative: on-site peer review is thought to be best, if it can be afforded. WTEC has done much of this for the US. Quantitative: bibliometric indicators for papers, patents, citations, et al. permit numerical evaluations a low cost

Qualitative WTEC assessments WTEC is a non-profit institute, spun off of Loyola University Maryland, that leads in international R&D assessments:65 to date It operates under a peer-reviewed award from the US National Science Foundation. Studies are usually funded by several US agencies Sponsors are program officers who want international data to guide their own research funding

Purposes of studies  Guide and justify U.S. research investments  Look for good ideas abroad (tech transfer)  Look for opportunities for cooperation and collaboration  Compare U.S. R&D programs and status with those abroad  Researchers in other nations often use these reports to track R&D internationally

WTEC methods  Write grant proposals that can pass peer review  Establish a coalition of sponsors who can make it happen  Recruit a great panel from NSF nominations  Conduct the study effectively  Maintain good host relations, so we can come back  Publish an outstanding report

Current studies ‏  Rapid Vaccine Manufacturing (Europe, Asia)  Nanotechnology 10-year reprise (Worldwide)  Mobility for people with disabilities (Europe)  Human-Robotics Interaction for disabled (Asia)  Stem Cell Engineering (Asia, Europe)  Systems Engineering for Manufacturing (Asia, Europe)  Applications of converging technologies (Worldwide) * The flags are from the 40 countries where we have sent delegations. We would love to add the DPRK one in front.

Quantitative assessments: Output dependent variables (DVs) Papers and Paper Share Science Citation Index, Scopus Patents and Patent Share Triadic, USPTO, PCT PhD graduates in science High-tech exports and market share

Input variables (IVs) from OECD Overall GERD (Gross Expenditures on R&D) GERD source components: Government Industry Abroad (funding from abroad) Other GERD spending components: HERD (higher education sector) BERD (business sector) Non-Profit (other than universities) GOVERD (government labs) Number of researchers

All inputs and outputs depend on the size of the country, making all country-wise correlations high, and obscuring which variables are most important Stepwise multiple linear regression can tease out which input IVs are best for predicting output DV. IVs are added one-by-one in order of which makes the best model for the DV. The size of nations is a confounding factor

Step-wise regression of 2007 SCI paper share (ps07) vs. three IVs Government GERD share HERD share Overall GERD share Fit of regression line

SCI Scopus Capital vs. Labor GERD Researchers Funding Components Industry Government Spending Components HERD BERD Correlations: Papers vs. Inputs Red indicates strongest correlation of pair; it will dominate a 2 IV model

IV1IV2Coeff1Coeff2Const p1 p2 R 2 GERDResearchers % GERD % GovernmentIndustry % Government % HERD % GovernmentHERD % Regressions of SCI paper share in 2007 For example the best single IV model (Shelton-Leydesdorff) is: Papers07 = Governments

Regressions show a trade-off in allocations To maximize papers, a country should maximize its government funding of R&D, instead of industry funding To maximize patents, a country should do the opposite: maximize its industrial funding of R&D Similarly spending in the higher education sector encourages papers, while business sector spending encourages patents Thus these allocations are simply a choice between longer and shorter term benefits of R&D Not surprising, but regressions provide some quantitative confirmation of this logic

Summary of models for paper share Simple extrapolations of trends in output paper share m i provide a reality check for models based on input resource drivers The Shelton Model based on GERD share works well for big countries. It accounts for the decline in US and EU due to the rise of China's share of GERD w i. m i = k i w i The Shelton-Leydesdorff Model based on government share accounts for the EU increase in efficiency in the 1990s, and the long-term US decline. m i = k i ’ w i ’ + c’ Adding a second IV, HERD spending share w i ’’ works even better. This accounts for the EU passing the US in m i = k i ’w i ’ + k i ’’w i ’’ + c’’

Validation of paper share models Like any theory, models need to be tested to see how well they account for new phenomena. Scattergrams can show how well regression models fit a year’s data. They don’t forecast the future so well. Once key IVs are identified by statistics, individual country models can be built and tested by “forecasting the past.”

Scattergram of paper share vs. government funding share

Performance of Shelton Model in forecasting from 2005 to 2010 Based on forecasts of GERD and its share from 2005 data. Accuracy of US and EU is not bad. PRC is growing slower than forecast.

Uses 5-year average of rates of Gov increase. EU and PRC fit well, but US is worse than forecast, because its rate of Gov increase has plummeted to near zero. (Individual models used.) Model performance: forecast 2005 to 2010

Quantitative conclusions Regressions show that investment choices are complementary: some are best for papers and some for patents Models based on these resource inputs have some success in forecasting But a take-away for the professors in the audience: just using HERD share to predict paper share is surprisingly accurate Thus if nations want to excel in papers, they should just give money to professors!

Scientometrics in the DPRK The DPRK has already made great progress in quantitative indicators They show rapidly increasing publication in the world’s leading journals and increasing international collaboration Qualitative peer review and international exchanges, like this one, will speed DPRK progress

Integer counts in the Web of Science version of the SCI Growth of DPRK papers in the world’s best journals

ISOcountryINTFRAC KPNorth Korea CNChina DEGermany AUAustrallia93.6 RORomania63.3 KRSouth Korea72.7 RURussia/USSR62.6 JPJapan52.0 USUSA61.6 UK 41.5 CZCzech Republic31.2 BGBulgaria21.0 HUHungary21.0 CHSwitzerland40.9 DPRK 1970 – 2011: International Collaborations For joint papers: integer counts (INT) and fractional counts (FRAC). From Grant Lewison.

DPRK Papers 1970 – 2011: Subject Areas Other From Grant Lewison.

Conclusions Scientometrics is a tool that can be used to monitor, measure, and assess S&T Science diplomacy can use this tool to speed national progress in S&T Scientometrics scholars form an international community that welcomes everyone And besides, it is fun

More information  WTEC  Shelton  WTEC books and reports. I brought 6 for the PUST library—and CDs with 50 reports on them.  Shelton, RD and P. Foland, The Race for World Leadership of Science and Technology: Status and Forecasts. Science Focus Vol. 5, No. 1, pp. 1-9 (Feb. 2010) in Chinese. Also, Proceedings of the 12th International Conference on Scientometrics and Informetrics, pp , Rio de Janeiro, July,  Shelton, RD and L. Leydesdorff, Publish or Patent: Bibliometric Evidence for Empirical Trade-offs in National Funding Strategies, Journal of the American Society for Information Science and Technology, in press.