The weight of research internationalization indicators Ülle Must Estonian Research Council SCIENTOMETRICS Status and Prospects for development Moscow, October 2013

Visualizing Friendships By Paul Butler, 13 December

Changing publication and collaboration pattern Published documents (in thousands) Research collaboration: % of documents with more than one country SCImago. (2007). SJR — SCImago Journal & Country Rank. Retrieved October 09, 2013, from

King, C. Multiauthor Papers: Onward and Upward. Retrieved July 1, 2013 from:

Data and methods Peer Review survey – General information (field, country of affiliation, citizenship, gender, the profile as researcher); – Experience in the Peer Review process (as applicant or reviewer); – S&T indicators; – Criticism of Peer Review; – The future of the peer review system. Respondents belong to 79 countries by nationality; their places of affiliation were in 66 countries. Among respondents, 1337 were men and 652 were women, 125 did not specify their gender. By academic position, postdoctoral research fellows (643) and full professors (422) constituted the largest share of respondents, followed by lecturers/assistant professors (325), associated professors (323). The largest numbers of respondents belong to natural sciences (1096).

S&T indicators (Un-ty Based Res, 2010) Input indicators External funding Recruitment of PhD students Process indicators Seminar and conference activity Invited keynotes International visiting research appointments Structure indicators Number of PhD students Research collaborations and partnership Reputation and esteem (position as journal editors, membership of editorial boards and scientific committees and membership in learned academies) Output indicators Publications Non-bibliographical outputs Number of PhD graduates and completion rates for graduates Effect indicators Citations H index Number of prestigious national and international awards and prizes External funding Employability of PhD graduates (in private sector) Social indicators Relevance to citizens' concerns Relevance to global societal challenges Usefulness to policy decision makers Relevance to science communication initiatives Contributing to science education Patent development Public outreach (broader dissemination, transfer and exchange of research results)

The rating of the S&T indicators by country groups

The rating of the S&T indicators by fields

The rating of the S&T indicators by career

The rating of the S&T indicators by gender

Availability of data Validity and reliability – Fit to goal – Stability and controllability Economic efficiency – Availability and the cost of processing the raw data The use quality – The uniqueness and clarity of interpretation – The ability to influence the behavior and process

Indicators validity by criteria Indicators/Criteria Validity and reliability Economic efficiency Exploitation quality Research collaborations and partnership +++ Number of prestigious national and international awards and prizes +/--- Relevance to global societal challenges +/--- Invited keynotes +/--- International visiting research appointments +++ External Funding +++

Conclusions The choice of indicators depend on the desired goal; The range of indicators is broad, and the choice of individual indicators should meet certain criteria; Various indicators should be weighted differently in different fields, at different career stages; Indicators have to be fit‐for‐purpose, appropriate and verifiable.