1. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Timeliness and quality Information products based on process-produced data – requirements on the quality assurance process Presentation at the 7 th euroCRIS Strategic Seminar Brussels, September, 14-15 th 2009

2. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Agenda 1. DFG at a glance 2. DFG‘s main source of information 3. Timeliness and Quality 4. Scope of DFG information products

3. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Third-party funding income of German higher education institutions (HEI) by source (2006)

4. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 DFG Awards by scientific discipline (2007) Mechanical and Industrial Engineering 6% Geosciences 6% Mathematics 3% Physics 10% Chemistry 7%Agriculture and Veterinary Medicine 2% Medicine 21% Biology 13% Social and Behavioural Sciences 6% Humanities 10% Construction Engineering and Architecture 1% Computer Science, Electrical and System Engineering 8% Material Science 3% Thermal and Process Engineering 4% Life Sciences Engineering Sciences Soc.Sc. & Humanities Natural Science Research fields: ► Soc.Sci.+Humanities 16% ► Engineering Sciences 22% ► Natural Sciences 26% ► Life Sciences 36%

5. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 DFG programme portfolio and budget (2007) 29.3%: Individual Grants Programme 54.6%: Coordinated Programmes 06.3%: Promoting Young Researchers 01.2%: Scientific Prizes 07.2%: Scientific Instrumentation and Infrastructure 01.4%: Encouraging Scientific Contacts (and other)  Programme portfolio and budget: ~2 Billion €, more than 1.000 programmes and 20.000 projects funded

6. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Department I Central Administration Strategic Planning Committees Executive Office Quality Assurance and Programme Development International Affairs Press and Public Relations Internal Auditing Berlin Office Internal Advisory Committee Extended Executive Board Heads of Divisions and Executive-Level Offices Executive Board President Secretary General Deputy Members: Heads of Departments I - III Forum Heads of Divisions  Budget and Accounting  Human Resources and Legal Affairs  Information Technology and Infrastructure  Information Management Department II Scientific Affairs Department III Coordinated Programmes and Infrastructure  Research Centres  Research Careers  Scientific Library Services and Information Systems  Scientific Instrumentation and Information Technology  Humanities and Social Sciences  Life Sciences 1  Life Sciences 2  Physics, Mathematics, Geosciences  Chemistry and Process Engineering  Engineering Sciences Dept. directly involved in the funding process The DFG´s Head Office: An overview

7. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 The funding-process-chain Whole chain is represented in the DFG-internal databases and document management systems Advice Submission Formal examination Reviewer Statutatory bodies Report Proposal Peer Review / Assessment Performance Evaluation Funding Decision

8. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Agenda 1. DFG at a glance 2. DFG‘s main source of information 3. Timeliness and Quality 4. Scope of DFG information products

9. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Facts and figures ► DFG internal electronic proposal processing system ► in use since 2005 (data coverage from 1982) ► used for all funding programmes ► used by about 400 out of 750 staff members directly involved in funding processes ► data source for all information products published by the DFG ElektrA – the data source

10. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009

11. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Agenda 1. DFG at a glance 2. DFG‘s main source of information 3. Timeliness and Quality 4. Scope of DFG information products

12. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Timeliness and quality ► Data in ElektrA are very actual („day before“) ► Because of the use of data for funding-process related products (e.g. letters to applicants and reviewers) most information are of high quality ► But: less use in funding related products results in lower quality

13. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Organisational model for quality assurance (QA) ► Quality responsibility is decentralised ●staff members are trained to produce good quality data ●software helps to avoid mistakes ► Nevertheless there is a need for a central quality assurance unit: ●development of complex quality assurance routines ●occasionally very high demand for timeliness (e.g. ad hoc statistics or „the day after“ information products like reports on decisions of DFG boards) ●most important: Training of data producing staff on how to avoid mistakes and – very helpful! – elucidation on the importance and use of infomation products based on these data

14. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Arguments for investing high effort in quality Only one: ► Quality of data ist quality of information products is acceptance and practical use for planning purposes of these products

15. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Agenda 1. DFG at a glance 2. DFG‘s main source of information 3. Timeliness and Quality 4. Scope of DFG information products

16. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Statistical reporting based on the DFG internal database Example: Quarterly statistics for the statutory bodies ► Statistical reporting for the statutory bodies of the DFG: ●e.g. success rates by programme ●e.g. number of programmes funded/rejected ●e.g. development of success rate over time

17. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 DFG internal data as a source for evaluative studies Evaluative Study: „Women in the DFG“ (2008) ► Topic: Gender equality has been one of the aims of the foundation since 2002 (Article 1 of its statutes) ► Objective: Transparency, planning criteria (e.g. development for the modification of funding instruments) ► The project team: Prof. Thomas Hinz, Ina Findeisen, Kathrin Auspurg, University of Konstanz ► Target group: Statutory bodies, interested members of the public ► Data: Mainly based on process-produced data; many other sources of data included ► Study (german) and results in brief (engl.): www.dfg.de/dfg_im_profil/zahlen_und_fakten/statistisches_berichtswesen/gleichstellung/ www.dfg.de/dfg_im_profil/zahlen_und_fakten/statistisches_berichtswesen/gleichstellung/

18. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Characteristics ► information system on DFG-funded projects ► different search functions ► shows all funded projects at a university, department, or institute since 1999 (~ 60.000 projects) Contribution to QA-process ► three quality checks – ●first during proposal processing ●second by applicant before publishing ●constant email-feedback by users and applicants GEPRIS – German Project Information System

19. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Research Explorer (REx): www.research.explorer.dfg.de Characteristics ► bilingual database (german / english) on research institutes at German HEIs and other institutions ► detailed information on nearly 20,000 institutes at these institutions ► links to GEPRIS Contribution ► daily by proposal processing within DFG ► periodically by QA-team for each university ► occasionally by user-feedback

20. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 The DFG Funding Ranking Planning tool for universities and other actors engaged in research planning and strategy. Helps to identify research profiles by means of: ► Institutional Profiles: Information about research profiles and main research fields covered by German HEIs ► Regional Profiles: Unveils regional profiles and clusters of excellence ► Structural Profiles: Informs about networks in science ► Disciplinary Profiles: Detailed analysis on research activities in different disciplines and fields of research (e.g. biotechnology)

21. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 iele, Objectives, approach and databases of the DFG Funding Ranking Deutsche Forschungsgemeinschaft German government European Union European Research Council German Federation of Industrial Reserach Organizations Alexander von Humboldt Foundation German Academic Exchange Service iele, Analyses by source Analyses by recipients dimensions of differentiation: countries, states, regions, institutions. Indicators cover approximately 90 percent of the third-party funding income of scientific and higher education institutions in Germany. Huge set of additional non-monetary indicators (intern. attractivity, scientific expertise...) No costly and laborious data collection from the research institutions but direct processing of data concerning funding activities of central sources Subject-related Analyses Classification according to four scientific disciplines Specifics: DFG: 48 research fields German government: 12 funding fields EU: 8 funding fields AvH / DAAD a.o: 14 research areas

22. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 DFG Funding Ranking Influence of programmes (e.g. ExIn) Gender equality International appeal Networks and cooperation structures Funding rankings and profiles Participants in peer review processes Aspects of statistical reporting within the DFG Funding Ranking

23. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Disciplinary profiles of universities ► Example: DFG-funding in the humanities ► range of information: ●Volume of funding ●spectrum of disciplines of each HEI ●participation of HEI in „Excellence Initative“ ► FU Berlin – LMU Munich: Similar vol of funding und similar disciplinary profile – different participation in ExIn

24. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Networks and cooperation structures within DFG funded programmes: humanities ► Very dense coll. cluster in Berlin ► = Berlin capitalizes its ressources best

25. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Recipients in Germany Ranking of countries LandMio. €kum. % Deutschland Großbritannien Deutschland Italien Niederlande Spanien Belgien Frankreich Insgesamt 3.024,0 2.369,6 2.172,3 1.457,9 1.107,4 943,8 707,9 16.665,3 1,8 3,2 4,5 5,4 6,1 6,6 7,1 100,0 Art der Einrichtung Hochschulen Außeruniversitäre Einrichtungen Max-Planck-Gesellschaft (MPG) Fraunhofer-Gesellschaft (FhG) Helmholtz-Gemeinschaft (HGF) Leibniz-Gemeinschaft (WGL) Bundeseinrichtungen Weitere Einrichtungen Industrie und Wirtschaft Nicht-institutionelle Mittelempfänger Ingesamt Mio. € 947,9 1.173,5 154,3 216,4 316,9 94,5 128,1 263,2 900,9 1,7 3.024,0 % 31,3 38,8 5,1 7,2 10,5 3,1 4,2 8,7 29,8 0,1 100,0 6th Framework Programme of the European Union

26. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 ElektrA management reports individual Letters funding ranking GEPRIS Research Explorer applicants, reviewers report users Ranking recipients GEPRIS users REx users Quality assurance team Feedback quality assurance different productsdifferent usersone data source one qa-team The quality assurance chain

27. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Conclusions ► Good quality of data is costly and need personnel ► Decentralization and direct use of data for business related products is the best guarantor for a good data base ► Besides this it is necessary to centralize QA processes ► The more products with certain specifications and certain target groups are based on the same data, the more feedback is possible to improve quality ► every data problem causes trouble to a broad set of information products – so high data quality is a good reason for investing in well defined QA processes

28. 7th euroCRIS Strategic Seminar, Juergen Guedler Brussels, September, 14-15th, 2009 Thank you for your attention ! more information: ► DFG: www.dfg.de ► DFG-Funding-Ranking: www.dfg.de/ranking ► DFG- funded projects: www.dfg.de/gepris ► research institutes at German HEI: www.dfg.de/research_explorer