European Research Council │ 1 Granting and Evaluation of Research in Humanities and Social Sciences at the European Level – and especially at ERC and ESF Alain PEYRAUBE CNRS – EHESS Membre of the ERC Scientific Council

European Research Council │ 2 European Reports on Evaluation AUBR Report « Assessing Europe’s University-Based Research » by Mackiewicz (Summer 2009, 130 pages). Detailed lists on the evaluation system in Australia, Belgium, Finland, France, Germany, Hungary, Ireland, Italy, The Netherlands, Norway, Spain, Sweden, United Kingdom Suède, on Webometrics, on ARWU (Shanghai rankings, on THES, on PRSP (Taiwan), on CWTS (Leiden) SPRU report « Towards a Bibliometric Database for the Social Sciences and Humanities – A European Scoping Project » (ANR, DFG, ESRC, AHRC, NWO, ESF) – January 15, 2010

European Research Council │ 3 European Reports on Evaluation (2) Article « Journal Base – A comparative international study of scientific journals in the social sciences and the humanities », M. Dassa, C. Kosmopoulos, D. Pumain Cybergeo, January 2010 Article « Research Quality Assessment and the Metrication of Social Sciences » (Myriam E. David, European Political Science , 13 pages) Bibliométrie et évaluation en SHS (Ghislaine Filliatreau, Revue d’histoire moderne et contemporaine 55-4, 2008, 4 pages)

European Research Council │ 4 FP7 (EC)  7 th Framework Programme (FP7) Billion Euros, 4 Specific Programs  “Family” of FP7 Specific Programmes  Co-operation 33 B Euros (8 priorities)  Ideas 7.5 B Euros [ERC]  People 5 B Euros (Marie Curie fellowships)  Capacities 4 B Euros (Infrastructures, etc.)  Ideas: complementary to other FP7 support policy vs. science-driven, bottom-up vs. targeted research

European Research Council │ 5 The Scientific Council Members & Role 22 most respected researchers reflecting the full scope of European research and scholarship  proposed by an independent identification committee  appointed by the Commission (for 4 years, renewable once) Role:  Establishes overall scientific strategy establishes annual work programmes (incl. calls for proposals, evaluation criteria); defines peer review methodology; ensures selection and accreditation of experts  Controls quality of operations and management  Ensures communication with the scientific community

European Research Council │ 6 The Executive Agency  Executes annual work programme as established by the Scientific Council  Implements calls for proposals and provides information and support to applicants  Organises peer review evaluation  Establishes and manages grant agreements  Administers financial aspects and follow-up of grant agreements

European Research Council │ 7 Strategic Aims Overview ERC : first pan-European funding agency for frontier research Funding directed to individual teams and projects selected solely on the criteria of excellence Operates according to the principles of scientific excellence, autonomy, efficiency and transparency  Autonomous scientific governance  Simple, user-friendly delivery

European Research Council Strategic Aims (2) Reinforce excellence, dynamism and creativity in European research Improve the attractiveness of Europe for the best researchers from both European and third countries, as well as for industrial research investment ERC grants can complement the efforts of national funding bodies and host institutions to build or reinforce excellence across Europe

European Research Council │ 9 ERC Grants: Open to individual teams anywhere in the world ERC is the only component of the FP7-family to widely open its programs and schemes to individual teams anywhere in the world, under only one condition, i.e. that funded projects will have to be located in EU or its associated countries Thus, PIs and team members can be of any nationality, working in almost any country Team may consist solely of the PI (mathematics,humanities)

European Research Council │ 10 Two types of Grants ERC Starting Grants, supporting excellent early stage independent investigators (2-9 years since completion of PhD) – 2007 and Starting Grant divided into two categories from 2010 on : Starters (PhD : +2, - 7) and Consolidators (PhD : + 7, - 12). Grants : up to 1.5 M euros for a period of 5 years (+ additional Euros 500,000 for eligible ‘start-up’ costs) ERC Advanced Grants, supporting investigators at all subsequent stages – 2008 and Grants up to 2.5 M euros (+ additional Euros 1 M for eligible ‘start-up’ costs)

European Research Council │ 11  For operational reasons the ScC agreed on 3 main research domains: ─Domain 1: Mathematics, Physical Sciences, Information and Communication, Engineering Sciences, Universe and Earth Sciences (PE) ─Domain 2: Life Sciences (LS) ─Domain 3: Social Sciences and Humanities (SH)  Pre-allocation of call budget for ERC Grants per domain currently as follows: 45% PE - 40% LS - 15% SH  2009 : 39% PE – 34% LS – 14% SH + 13% Interdisc.  2011 : 40% PE – 35 % LS – 15% SH + 10% ID ERC Grant schemes Budget Allocation

European Research Council │ 12  Panels have one Panel Chair and Panel Members (now between 12 and 15), all selected by the Scientific Council members  20 Panels for the 1st Call : 8 PE, 7 LS, 5 SH  25 Panels now: 10 PE, 9 LS, 6 SH  Panel Chair oversees evaluation process for the proposals assigned to his/her Panel in collaboration with ERC staff  Panel Chair gives high level stamp of credibility and visibility to the whole evaluation process ERC Grant schemes Panel Structure

European Research Council │ 13 Panels composition Panel Members and Chairs  Make decisions and document the decisions Panel chairs  Ensure quality of the evaluation process  Chair panel meetings (steps 1 & 2)  Participate in panel chair meetings ERC Panel Teams (2 Scientific Officers)  Ensure that rules are respected  Ensure comparable process is followed in all panels  Transition: composed of ERC staff and RTD colleagues

European Research Council │ 14 Evaluation criteria 1. Principal Investigator CV of 2 pages, and Early Achievements track-record of 2 pages for Starting Grant; CV of 2 pages, and 10-year track record of 2 pages for Advanced Grant 2. Research Project Extended Synopsis of 5 pages, Full scientific proposal of 15 pages maximum for both Starting Grant and Advanced Grant

European Research Council Evaluation criteria (2) 3. Research Environment Step 1 of evaluation  criteria 1 and 2 Starting Grant PIs invited for interview for the 2 nd step of evaluation Step 2 of evaluation  all criteria  Referees and panels evaluate and score criteria 1 and 2 numerically, which will result in the ranking of the proposals  Criteria 3 is considered as "pass/fail" and commented upon but not scored. │ 15

European Research Council Evaluation criteria (3) Each proposal marked on a scale of 1 to 4 for each of the 2 evaluation criteria: 4 : Outstanding, 3 : Excellent, 2 : Very Good, 1 : Non-competitive A proposal marked below the quality threshold of 2 on either of the two headings after the 1st stage of evaluation not retained for the second step Proposal marked below the quality threshold of 2 at the end of the 2nd step not funded

European Research Council │ 17 Co-investigators Exceptional For interdisciplinary proposals only PI can include 1 or more co-investigators (co-I) Co-I submitted to same resubmission rule as PI Higher budget (up to 3,5 M€ for Advanced Grant) Scientific added value of including co-investigators to be assessed by panel

European Research Council │ 18 New Panel structure and description Social Sciences and Humanities (18 March 2008) SH1Individuals, institutions and markets: economics, finance and management SH2Institutions, values and beliefs and behaviour: sociology, social anthropology, political science, law, communication, social studies of science and technology SH3Environment and society: environmental studies, demography, social geography, urban and regional studies SH4The Human Mind and its complexity: cognition, psychology, linguistics, philosophy and education SH5Cultures and cultural production: literature, visual and performing arts, music, cultural and comparative studies SH6The study of the human past: archaeology, history and memory

European Research Council │ 19 Domain Number of proposals % of proposals % of budget pre-allocated* Total9,167 ERC-2007-StG Submission (stage 1): Massive response! *Indicative budget established by ScC (Work programme 2007)

European Research Council │ 20 Country of host institution Number of proposals by domain and country of host institution (head quarters) 34 countries

European Research Council │ 21 Domain Number of proposals % of proposals % of budget pre-allocated* Total ERC-2007-StG Granted projects *Indicative budget (Scientific Council, ERC work programme 2007)

European Research Council │ 22 Distribution of the 303 granted 303 proposals granted (3.30%): 137 PE (45%), 108 LS (36%), 58 SH (18%) Distribution of the 303 by host institution: 19% UK, 13% FR, 11% DE, 8.5% NL, 8.2% IT, 8% SP, 8% IL, 1 proposal from CZ The 58 SH: 18 UK, 10 NL, 6 FR, 4 DE, 4 ES, 4 IT, …

European Research Council │ 23 ERC Advanced Grant – 1 (2008) (ERC Advanced Investigator Researcher Grant)  Targeting researchers who have already established their independence as team leaders and are exceptional leaders in terms of significance of their research achievements (in the last 10 years) up to 5 years, up to € 3,5 million per grant ~2000 Starting Grants over 7 years of FP7 ( )

European Research Council │ 24 ERC Advanced Grant 1 Evaluation Criteria: Excellence is the sole award criterion 1.Principal Investigator -Quality of research output/track-record (  10-year track-record) -Intellectual capacity and creativity (  CV + leadership profile) 1.Research Project -Ground-breaking nature of the research -Potential impact -Methodology 1.Research Environment (assessed in Step 2 only) -Contribution of the research environment to the project -Contribution of the project to research environment -Participation of other legal entities (if clear/ substantial added value)

European Research Council │ 25 Domain Number of proposals % of proposals % of budget pre-allocated* Total2167 Distribution / Domain - Applications

European Research Council │ 26 Domain Number of Proposals % of proposals % of budget pre-allocated* Total250 Distribution / Domain - Granted + 29 INTERDISC. (11 PE, 11 LS, 7 SH) = 13% of the budget = 279 (12.9%)

European Research Council │ 27 Advanced Grant 1 – DistrIbution / Host Institution of the granted projects 63 UK, 37 FR, 28 CH, 28 DE, 20 IT, 1 NL, 15 IL, 13 SE, 13 ES, … 2 CZ PE Domain, out of the 117: 20 UK, 15 FR, 12 CH, 12 IT, 11 DE, 10 NL, 9 IL, 9 SE … LS Domain, out of the 84: 18 UK, 11 DE, 11 FR, 9 CH, 5 ES, 5 NL, 4 IT, 4 IL … SH Domain, out of the 49: 12 UK, 9 FR, 5 DE, 5 NL, 5 ES, 4 IT, 3 CH …

European Research Council │ 28 ERC Starting Grant Submitted and mainlist proposals by domain Life Sciences Physical Sciences & Engineering Social Sciences & Humanities 927 (37%) 1112 (44.4%) 464 (18.6%) 69 (31.5%) 93 (42.5%) 41 (18.7%) Submitted proposalsMainlist ∑ = 2503 ∑ = (7.3%) Interdisciplinary Domain

European Research Council │ 29 ERC Starting Grant 2009 ERC Starting Grant 2009 The 238 granted proposals (9.50%) Distribution by host institution: 43 UK, 31 FR, 28 DE, 18 ES, 17 CH, 17 NL, 15 BE, 15 IT, 15 NL, 14 BE, 14 IL, … In the SH domain : 12 UK, 6 DE, 6 NL, 5 BE, 5 ES, 5 IT, 2 DK, 2 FR, …

European Research Council │ 30 Advanced Grant Number of submitted and short-listed proposals Life Sciences Social Sciences & Humanities Submitted proposalsShort List ? Interdisciplinary research Physical Sciences & Engineering ∑ = 1583 ∑ = 554

European Research Council │ 31 Main Lists : 236 (14.9%) PE : 99 proposals SH : 42 proposals LS : 81 proposals ID : 14 proposals (6  PE, 8  LS, 0  SH) Host Institution : 58 UK, 34 FR, 31 DE, 29 CH, 16 NL, 15 IT, 15 NL, 12 SE, 11 IL, 10 ES, …

European Research Council │ 32 Distribution by domain 105 PE : 23 UK, 15 CH, 15 DE, 14 FR, 8 SE, 6 IL, 6 IT, 5 ES, 4 NL, … 89 LS : 21 UK, 15 FR, 13 CH, 12 DE, 6 NL, 5 IT, 4 ES, 3 IL, 3 SE, … 42 SH : 14 UK, 5 FR, 5 NL, 4 DE, 4 IT, 2 IL, 2 NO, 1 AT, 1 CH, 1 ES, 1 IE, 1 HU, 1 SE

European Research Council │ 33 Research Assessment in the Humanities and the ESF-ERIH project

European Research Council │ 34 ESF-SCH workshop in 2001  We can no longer allow ourselves the indulgence of attributing the lack of appropriate tools to a peculiarity of the Humanities and Social Sciences  Specialists in the Humanities should take into consideration its special characteristics and develop the corresponding tools  It is essential for the future of the Humanities to furnish itself with reliable reference tools, following the example of other scientific disciplines.

European Research Council │ 35 ESF-SCH workshop in 2001 (2) AHCI (ISI-Thomson Scientific Web of Knowledge) not appropriate Urgent need for a European Reference Index for the Humanities (ERIH) as an additional tool for research evaluation, and not the exclusive means Request the ESF-SCH to go ahead and try to compile lists of reference journals (Stage 1) Develop methodology for including other formats: monographs, book chapters, edited volumes, etc. (Stage 2)

European Research Council │ 36 ESF-SCH > ERIH Steering Committee  Overall responsibility with the ESF Standing Committee for the Humanities (SCH)  SCH nominates ERIH Steering Committee in Spring 2004: F. Kiefer (HU), A. Mustajoki (FI), A. Peyraube (FR, Chair), Gudrun Gersmann (DE), Marc Waelkens (BE), Michael Worton (UK).  ERIH Steering Committee responsible for:  Identification of the disciplinary structure  Definition of methodology including the definition of categories of journals  Approval of membership of Expert Panels (members suggested by MO’s, SCH, ERIH StComm)  Validation of journal lists proposed by Expert Panels

European Research Council │ ERIH: current disciplinary structure Archaeology Art and Art History Classical Studies Anthropology Gender Studies History Hist. & Phil. of Science Linguistics Literarature Musicology Oriental & African Studies Pedagogical & Educ. Research Philosophy Psychology Religious Studies & Theology Disciplines under considertation Archives, Library & Museum Studies Film, Media & Cultural Studies Area Studies

European Research Council │ 38 Peer review: the basis of methodology Peer review recognised as the only practicable method of evaluation in basic research (standard method used in evaluation of scientific production)  nomination of panel members ESF Mos approached – with guidelines – to provide lists of reference journals discipline by discipline

European Research Council │ 39 ERIH – Journal categories: A category (expected: 5-20% of all titles):  High-ranking, international level publications;  Very strong reputation among researchers in the field;  Regularly cited all over the world B category:  Standard, international level publications;  Good reputation among researchers in the field in different countries, occasionally cited all over the world

European Research Council │ 40 ERIH – Journal categories (2): C category: Important local / regional level publication; Mainly local / regional readership, but occasionally cited outside the publishing country Only European publications are considered in this group

European Research Council │ 41 Process of accreditation Input: National panels / scientific communities provide lists of journals Selection: Expert Panels define scope, analyse and assess input, produce lists Consultation: MOs, subject associations (European level and some national), specialist research centres Calibration/harmonisation: ERIH Steering Committee; After approval by ESF-SCH, publication of ‘initial lists’ took place in 2007 Open feedback process via on-line questionnaire for editors and publishers ‘Revised lists’ to be published in 2010

European Research Council │ 42 Some examples Anthropology: 752 journals proposed by Mos  242 journals selected: 40 A (16.5%), 112 B (46.3%), 90 C (37.2%) Archaeology: 1312 (Mos)  425: 91 A (21.4%), 170 B (40%), 164 C (38.6%) History: 1419 (Mos)  934: 135 A (14.45%), 380 B (40.7%), 419 C (44.9%) Linguistics: 1093 (Mos)  585: 96 A (16.4%), 92 B (36.5%), 275 C (47%)

European Research Council │ 43 Some examples (2) Literature: 1453 (Mos)  976: 255 A (26.1%), 481 B (49.2%), 240 C (24.6%) Music and musicology: 300 (Mos)  166: 21 A (12.6%), 86 B (51.8%), 59 C (35.5% Philosophy: 658 (Mos)  305: 44A (14.4%), 130 B (42.6%), 131 C (42.9%) Relig. Stud. &Theology: 745 (Mos)  371: 76 A (20.5%), 203 B (54.7%), 92 C (24.8%)

European Research Council │ ERIH: challenges and criticism Misunderstandings about the character of the currently used A/B/C categories: ranking or assessment of audience, distribution and reach? Misunderstandings around category C seen as ‘low quality’ when the idea is to identify quality European journals with, mostly linguistically, limited circulation; identification of quality national journals is the main innovation of ERIH Following discussions in the research community, process of renaming ERIH categories is underway

European Research Council │ 45 More misunderstandings Some research councils and research organisations have used ERIH as a tool for assessment of individual research production / productivity ERIH “initial lists” were used when they were still under revision

European Research Council │ 46 New categorization of Journal categories National Journals (former category C): European publications with a recognised scholarly significance among researchers in their respective research domains in a particular (mostly linguistically circumscribed) readership group in Europe; occasionally cited outside the publishing country, though their main target group is the domestic academic community International Journals (former categories A and B): both European and non-European publications with an internationally recognised scholarly significance among researchers in their respective research domains, and which are regularly cited worldwide

European Research Council │ 47 New Journal categories (2) Differentiation between current categories A and B is based on a combination of two criteria: influence and scope: Category A international publications with high visibility and a very strong reputation and influence among researchers in the various research domains in different countries Category B international publications with significant visibility and a good reputation and influence in the various research domains in different countries.

European Research Council │ 48 Next steps Reorganisation of the expert panels by using a panel rotation mechanism and inclusion of new experts Expert Panel meetings revising ‘initial lists’ based on received feedback (adding in new journals, eliminate some journals, changing and revising categories) Consolidation of the lists in 2009 (First update) > New lists (Revised lists) to be published in 2010 Workplan for inclusion of other publications (monographs, edited books, proceedings, etc.)

European Research Council │ 49 ERIH - contact

European Research Council │ 50 Merci Thank you 谢谢