1. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 International Workshop on the Evaluation of Publicly Funded Research Berlin, 26/27 September 2005 Summative evaluations and their effects on institutional strategy and performance. Experiences from the German Wissenschaftsrat‘s evaluations of Blue List institutions Professor Dr. Reinhard Hüttl Chairman of the Scientific Commission of the German Wissenschaftsrat WISSENSCHAFTS RAT

2. Workshop Evaluation, Berlin, 26/27 September 2005 Scientific members Federal and state governments The German Wissenschaftsrat Committees and working groups (complemented by external experts) prepare recommendations send members

3. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Level 2 Comparative assessment Level 1 Scientific assessment Evaluation procedure Working group: evaluation report Evaluation committee/ Wissenschaftsrat: Science policy evaluation statement

4. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Experience in institutional evaluation: Ex-ante evaluations concerning large scale investment in scientific instruments Cross sectional evaluations of research fields System evaluations of research organisations Structural evaluations of specific disciplines Evaluations of single research institutes Research rating Comparative evaluations of institutional strategies

5. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 „Blue list“ institutes: Approximately 80 non-university institutes Jointly funded by Federal Government and state governments Missions include research, service and policy advice Evaluation of all institutes completed in 2001

6. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Criteria for institutional evaluation: Core criterium: quality of scientific work Peer review-based methodology Flexible adjustment of general criteria

7. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005

8. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Effects of Blue List evaluation gains in reputation shared funding stopped in five cases, nine new affiliations improved organisation and research strategy

9. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005

10. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005

11. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Implementation of results evaluation must have consequences Wissenschaftsrat keeps track of implementation shared interest of science and science policy not to damage the instrument of evaluation

12. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005

13. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Evaluation of institutes of scientific policy advice: Policy advice needs firm scientific basis Evaluation of governmental research institutes currently under way

14. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005

15. WISSENSCHAFTS RAT Workshop Evaluation, Berlin, 26/27 September 2005 Further developments Research rating: pilot study on chemistry and sociology Excellence initiative: institutional strategies to develop top-level university research