RESEARCH EVALUATION WORKSHOP UNITED KINGDOM OCTOBER 2010.

Slides:



Advertisements
Similar presentations
S Y N E R G I E S The Canadian Research Information Network Canadian Foundation for Innovation External Committee Evaluation –
Advertisements

CYPRUS UNIVERSITY OF TECHNOLOGY Internal Evaluation Procedures at CUT Quality Assurance Seminar Organised by the Ministry of Education and Culture and.
Web of Science Search and Navigation in the Web of Knowledge
ANALYSING RESEARCH – A GLOBAL PERSPECTIVE Krzysztof Szymanski – Country Manager Thomson Reuters October 2009.
Shou Ray Information Service Co., Ltd.
Bibliometrics overview slides. Contents of this slide set Slides 2-5 Various definitions Slide 6 The context, bibliometrics as 1 tools to assess Slides.
Using Web of Science as a Research Tool : Experience at HKUST Library Steve Yip Electronic Information Librarian.
Cracking the Web of Science: International Diversity and Citation Analysis NEICON Conference St. Petersburg, Russia 26 May 2015 Christopher Burghardt Head.
Using Journal Citation Reports The MyRI Project Team.
Distinctive Features of Russian Science and Government Policy Irina Dezhina Head of Economics of Science and Innovations Division, Institute of World Economy.
University ranking improving tools in modern information educational environment M.B. Maksimovich Kherson State University.
THE ROLE OF CITATION ANALYSIS IN RESEARCH EVALUATION Philip Purnell September 2010.
Web of Science Pros Excellent depth of coverage in the full product (from 1900-present for some journals) A large number of the records are enhanced with.
T H O M S O N S C I E N T I F I C Editorial Development James Testa, Director.
Types of Sources Used in Research Nancy McEnery, MLIS.
Rating and Ranking: Pros and Cons Dr. Mohsen Elmahdy Said Professor, Mechanical Design and Production Department Faculty of Engineering – Cairo University.
College of Agriculture and Life Sciences WELCOME Associate Professor P&T Workshop Transitioning from Associate to Full Professor April 23, 2015.
New Web of Science Rachel Mangan Customer Education
Guillaume Rivalle APRIL 2014 MEASURE YOUR RESEARCH PERFORMANCE WITH INCITES.
SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE KIEV, 31 JANUARY.
Orientation to Web of Science Dr.Tariq Ashraf University of Delhi South Campus
Techniques for Indexing a Research Journal in ISI list By Prof. Dr. Matthew Kuofie, MBA, PhD.
The Web of Science database bibliometrics and alternative metrics
The Role of Citations in Warwick’s Strategy and Improving Them Nicola Owen (Academic Registrar) Professor Mark Smith (PVC Research: Science and Medicine)
Making an impact ANU Library What is impact What (the heck) are bibliometrics Publish with impact – an overview Debate on impact How innovative are you.
ISI Web of Knowledge Service for UK Education
IL Step 1: Sources of Information Information Literacy 1.
Experiences with a bibliometric indicator for performance-based funding of research institutions in Norway Gunnar Sivertsen Nordic Institute for Studies.
How Web of Knowledge Can Help Kyrgyzstan Research, It’s Visibility and Influence David Horky & Metin Tunc 1.
Digital Libraries: Redefining the Library Value Paradigm Peter E Sidorko The University of Hong Kong 3 December 2010.
Rajesh Singh Deputy Librarian University of Delhi Measuring Research Output.
Welcome to Web Of Knowledge. Copyright 2006 Thomson Corporation 2 A very brief history of the citation index The concept of the citation index was first.
Discovery tools and research assessment solutions APRIL 2012 Shahrooz Sharifrazy Regional Sales Manager.
Bibliometrics: coming ready or not CAUL, September 2005 Cathrine Harboe-Ree.
The Web of Science, Bibliometrics and Scholarly Communication 11 December 2013
SCOPUS AND SCIVAL EVALUATION AND PROMOTION OF UKRAINIAN RESEARCH RESULTS PIOTR GOŁKIEWICZ PRODUCT SALES MANAGER, CENTRAL AND EASTERN EUROPE LVIV, 11 SEPTEMBER.
THOMSON REUTERS RESEARCH IN VIEW Philip Purnell September 2011 euroCRIS symposium Brussels.
THOMSON SCIENTIFIC Patricia Brennan Thomson Scientific January 10, 2008.
Web of Science® Krzysztof Szymanski October 13, 2010.
April 9, 2003Santiago, Chile The ISI Database: Reflecting the Best of International and Regional Research Keith R. MacGregor Sr. Vice President The Americas,
JOURNAL CITATION REPORTS ® “THE JCR” Jon Stroll, Key Account Manager January 2011.
NIFU STEP Norwegian Institute for Studies in Innovation, Research and Education 7 th euroCRIS strategic seminar, Brussels Recording Research.
Gaining and Maintaining Supported Researcher Status Knowing the Rules of the Game June 2007.
Journal Impact Factors: What Are They & How Can They Be Used? Pamela Sherwill, MLS, AHIP April 27, 2004.
DOAJ Directory of Open Access Journals Berlin March 2006.
Bibliometrics toolkit Website: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Further info: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Scopus Scopus was launched by Elsevier in.
The Web of Science, Bibliometrics and Scholarly Communication
RESEARCH – DOING AND ANALYSING Gavin Coney Thomson Reuters May 2009.
1 Making a Grope for an Understanding of Taiwan’s Scientific Performance through the Use of Quantified Indicators Prof. Dr. Hsien-Chun Meng Science and.
RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010.
Bibliometrics and Publishing Peter Sjögårde, Bibliometric analyst KTH Royal Institute of Technology, ECE School of Education and Communication in Engineering.
LIFE Conference 20 April 2006 The Researchers’ Perspective Michael Jubb Director Research Information Network.
Web of Science: The Use & Abuse of Citation Data Mark Robertson & Adam Taves Scott Library Reference Dept.
EAIR SIG Exploiting Data Repositories Using data repositories and information sources to support KPI’s and Strategic Planning: A Case Study Presented by:
1 RUSSIAN SCIENCE UNDER THE MICROSCOPE Philip Purnell Moscow, October 2013.
RESEARCH EVALUATION WORKSHOP The use of bibliometrics for research evaluation Dr Guillaume Rivalle Product Specialist.
MEASURING RESEARCHERS: FROM RESEARCH PUBLICATIONS AND COMMUNICATION TO RESEARCH EVALUATION Lucie Vavříková 1.
The Thomson Reuters Journal Selection Policy – Building Great Journals - Adding Value to Web of Science Maintaining and Growing Web of Science Regional.
THOMSON REUTERS INCITES Marta Plebani – Country Account Manager – Italy, Slovenia, Croatia 12 May 2011.
Publication Pattern of CA-A Cancer Journal for Clinician Hsin Chen 1 *, Yee-Shuan Lee 2 and Yuh-Shan Ho 1# 1 School of Public Health, Taipei Medical University.
Measuring Research Impact Using Bibliometrics Constance Wiebrands Manager, Library Services.
THOMSON REUTERS INTRODUCTION David Horky Country Manager
INTRODUCTION TO BIBLIOMETRICS 1. History Terminology Uses 2.
Role of librarians in improving the research impact and academic profiling of Indian universities J. K. Vijayakumar Ph. D Manager, Collections & Information.
Academic Ranking of World Universities
Tools for Effective Evaluation of Science InCites David Horky Country Manager – Central and Eastern Europe
The role and contributions of the Information Specialist to the core business/functions of the University of Pretoria Suzy Nyakale: Faculty Library.
Susan Veldsman Director: Scholarly Publishing Unit October 2010
‘Accessing Emerald’ Saudi Digital Library
Bibliometric Analysis of Water Research
Presentation transcript:

RESEARCH EVALUATION WORKSHOP UNITED KINGDOM OCTOBER 2010

WHY EVALUATE RESEARCH PERFORMANCE? Quantitative analysis is the main tool of science Communicating research results complex Personal knowledge no longer sufficient for decision making Need to be selective over support for research projects Peer review was foundation of policy decisions –Library collection decisions –Foundation allocating limited funding –Government office weighing national research needs

WHY EVALUATE RESEARCH PERFORMANCE? Evaluation and strategic planning –Periodic evaluation of research performance –Institution, departmental or researcher level assessments Accreditation, tenure, faculty review –Performance indicators Used in strategic planning Reporting to government bodies, boards of directors/trustees Research Centers –Find new staff –Develop lines of investigation –Compete for funds

HOW IS RESEARCH EVALUATED? Research –Volume, income, reputation Prestigious awards –Nobel Prizes Innovation –Industry income and patents Teaching –Academic Reputation Survey, higher degrees International Mix –National / International staff and students Citation analysis –Normalised for volume and subject area Peer Evaluation –Reputational survey

HOW IS RESEARCH EVALUATED? 5

6

HOW IS RESEARCH EVALUATED? SOUTH AFRICA 7

THE GROWING USE OF BIBLIOMETRICS Nations with significant science enterprises have embraced bibliometrics Today, bibliometrics programs with large teams of analysts are firmly established in many nations –These groups issue bibliometric reports, often called science indicators studies, at regular intervals In almost all cases, the publication and citation data of Thomson Reuters form the basis of their bibliometric analyses

INSTITUTIONS USING WEB OF SCIENCE CITATION DATA FOR EVALUATION (INCL.) United Kingdom: KCL, HEFCE, St. Andrew’s Germany: IFQ, Max Planck Society, DKFZ, MDCUS Netherlands: NWO & KNAW France: Min. de la Recherche, OST - Paris, CNRS European Union: EC’s DGXII(Research Directorate) US: NSF: biennial Science & Engineering Indicators report (since 1974) Canada: NSERC, FRSQ (Quebec), Alberta Research Council Australian Academy of Science, gov’t lab CSIRO Japan: Ministry of Education, Ministry of Economy, Trade & Industry People’s Republic of China: Chinese Academy of Science Multiple rankings agencies 9

THE DATA

A BRIEF HISTORY OF THE CITATION INDEX Concept first developed by Dr Eugene Garfield –Science, 1955 The Science Citation Index (1963) –SCI print (1960’s) –On-line with SciSearch in the 1970’s –CD-ROM in the 1980’s –Web interface (1997) Web of Science Content enhanced: –Social Sciences Citation Index (SSCI) –Arts & Humanities Citation Index (AHCI) The Citation Index –Primarily developed for purposes of information retrieval –Development of electronic media and powerful searching tools have increased its use and popularity for purposes of Research Evaluation

THE VALUE OF A CITATION Why do people cite? –Pay homage / give credit to pioneer –Identifying a methodology –Provide background reading –Quotations –Authenticating data, reproducing work etc –Corrections –Criticizing/Disclaiming someone's work/opinions Citations are an indicator of an article’s impact and usefulness to the research community; they are the mode by which peers acknowledge each other’s research. The value of a citation is only as important as its source. –Clearly a citation from a prestigious peer review journal has more value than a citation from non-scholarly material. –How can you be sure that the citing source is reputable? “When to Cite”, E. Garfield, Library Quarterly, v66, p , 1996

WHY NOT INDEX ALL JOURNALS? 40% of the journals: 80% of the publications 92% of cited papers 4% of the journals: 30% of the publications 51% of cited papers

HOW DO WE DECIDE WHICH JOURNALS TO INDEX? Approx journals evaluated annually –10-12% accepted Thomson Reuters editors –Information professionals –Librarians –Experts in the literature of their subject area Web of Science Journals under evaluation Journal ‘quality’

THOMSON REUTERS JOURNAL SELECTION POLICY Publishing Standards –Peer review, Editorial conventions Editorial content –Addition to knowledge in specific subject field Diversity –International, regional influence of authors, editors, advisors Citation analysis –Editors and authors’ prior work

Region# Journals from Region in Web of Science Europe5,573 49% North America4,251 38% Asia-Pacific965 9% Latin America272 2% Middle East/Africa200 1% Language# Journals in Web of Science English % Other % GLOBAL RESEARCH REPRESENTATION WEB OF SCIENCE COVERAGE

 Full range of scholarly research disciplines  Adheres to a consistent selection policy  Ensures that publications and citations are comparable  Consistent indexing  Cover-to-cover indexing  All author names  All author addresses CONSISTENCY IS THE KEY TO VALIDITY - COMPARE APPLES WITH APPLES

Government agencies/funding organizations Individuals Faculty, staff, students University Departments Institutional research, academic affairs, tech transfer, etc. External Entities University Management Management, including committees, provost, vice provosts PRIMARY USERS OF CITATION DATA IN RESEARCH EVALUATION

Government agencies/funding organizations External Entities EXTERNAL ENTITIES RESEARCH EVALUATION –Higher Education Funding Council for England, UK –National Science Foundation (USA) –European Commission (EU) –L’Observatoire des Sciences et des Techniques (OST ) –National Institute for Science and Technology Policy (NISTEP ), Japan –Human Sciences Research Council, South Africa

University Management Management, including committees, provost, vice provosts UNIVERSITY MANAGEMENT RESEARCH EVALUATION

INSTITUTIONAL LEVEL RESEARCH EVALUATION University Management Management, including committees, provost, vice provosts Source: Thomson Reuters U.S. and Canadian University Science Indicators Number of citations to North American scientific papers

NUMBER OF RESEARCHERS BY DEPARTMENT AND ROLE

PRODUCTIVITY BY UNIVERSITY DEPARTMENT Allows user to analyse output and performance based on their institution’s departments

University Departments Institutional research, academic affairs, tech transfer, etc. SUBJECT CATEGORY LEVEL RESEARCH EVALUATION

DOCUMENT TYPE BY DEPARTMENT

IN WHICH JOURNALS HAS OUR CHEMISTRY DEPT. PUBLISHED?

HOW HAVE THOSE PAPERS PERFORMED? 27

HOW MANY CITATIONS HAS THE CHEMISTRY DEPT. RECEIVED? 28

IN WHICH JOURNALS ARE THOSE CITATIONS PUBLISHED? 29

Individuals Faculty, staff, students INDIVIDUAL LEVEL RESEARCH EVALUATION

EVALUATING INDIVIDUALS Nancy J Rothwell, DBE, FRS President & Chancellor, Univ Manchester Dame Commander Order of the British Empire Fellow of the Royal Society Research Chair, Medical Research Council

EVALUATING INDIVIDUALS 32 Number of articles : 485 Sum of the Times Cited : 18,943 Average Citations / Item : h-index : 70

WHO CITED THIS AUTHOR’S RESEARCH? A very international profile illustrating the global impact of Prof. Rothwell’s research

EVALUATING INDIVIDUALS INDIRECT INFLUENCE

 Full range of scholarly research disciplines  Adheres to a consistent selection policy  Ensures that publications and citations are comparable  Consistent indexing  Cover-to-cover indexing  All author names  All author addresses CONSISTENCY IS THE KEY TO VALIDITY