1. RD4S Exercise Pillar C: How can the contribution of research to SD be measured ? By L. Esterle, Cermes and Ifris, France

2. RD4SD exercise-Pillar C L. Esterle May 26, 2009 2 Measurement of R&D contribution to SD Case figure: Framework program Indicators of R&D Proposals for a new method

3. RD4SD exercise-Pillar C L. Esterle May 26, 2009 3 Which indicators? Three types of indicators: Input Output Outcome

4. RD4SD exercise-Pillar C L. Esterle May 26, 2009 4 Input and output indicators are useful for policymakers Input indicators: R&D expenditures Number of researchers =>specific surveys Output indicators Scientific production Patenting activities

5. RD4SD exercise-Pillar C L. Esterle May 26, 2009 5 Measurement of the impact: outcome indicators Outcome indicators: measuring the indirect effect of R&D on society Depending the objectives of the R&D program Difficult to construct Nature of innovation process More efforts on economic impacts than on society or environment impacts An interesting issue: measurement of behavioural additionality

6. RD4SD exercise-Pillar C L. Esterle May 26, 2009 6 First recommendation (C1) There is a need to collect information about existing surveys, about analyses to build input and output indicators and about experiments conducted at any level to measure the contribution of R & D to SD.

7. RD4SD exercise-Pillar C L. Esterle May 26, 2009 7 An alternative method for measuring the contribution of R&D to SD A combined method: Production of indicators on the performance and contribution of the researchers to SD Evaluation b y a Jury in the framework of a participative debate

8. RD4SD exercise-Pillar C L. Esterle May 26, 2009 8 Measuring the contribution of researchers to SD by measuring their activities Many benefits from R&D are expected by policy makers The benefits can be matched with activities of the research groups Traditional R&D activities: e.g. knowledge production Societal activities : e.g. expertises

9. RD4SD exercise-Pillar C L. Esterle May 26, 2009 9 Examples of activity indicators Impact on peoples behavior: Number of communications through various media Number of participations to public conferences, to public debates Participation of stakeholders in the research projects …

10. RD4SD exercise-Pillar C L. Esterle May 26, 2009 10 Second recommendation (C2) SD research is best measured via proxy indicators linked to the direct results of research activities contributing to SD.

11. RD4SD exercise-Pillar C L. Esterle May 26, 2009 11 A Jury approach to complete the measurement SD : a political issue Indicators will provide a pretext for debates

12. RD4SD exercise-Pillar C L. Esterle May 26, 2009 12 The Jury approach Constitution of the Jury Methods Involvement of experts in science policy evaluation

13. RD4SD exercise-Pillar C L. Esterle May 26, 2009 13 Third recommendation (C 3) The production of indicators should be completed by the analysis of their results by a Jury in the framework of participative debate. The composition of the Jury should reflect the interests represented by the stakeholders and the beneficiaries of the research. Its role will be to identify the relevant contributions of the research to SD and to assess the achievements of the research program. The combined method needs to be tested and developed.

14. RD4SD exercise-Pillar C L. Esterle May 26, 2009 14 Conclusions The combined method will allow to associate objective measurement with subjective evaluation in the political and citizen arena It will allow for a learning process : for researchers, policy makers and all stakeholders.

15. RD4SD exercise-Pillar C L. Esterle May 26, 2009 15 Fourth recommendation The development of other methods, such as the measurement of the behavioural additionality, needs to be encouraged.