1 Introduction of Research and Development Project Evaluation System at NEDO Momoko OKADA New Energy and Industrial Technology Development Organization(NEDO) 28 October 2005 CES/AEA Conference in the Sheraton Centre, Toronto

2 -CONTENTS- ABOUT NEDO R&D EVALUATION SYSTEM AT NEDO -INTERMEDIATE & EX-POST EVALUATIONS -FOLLOW-UP MONITORING& EVALUATIONS CLOSING

3 -ABOUT NEDO- Establishment -Established in October 1980 as a semi-governmental organization under the supervision of METI. - Original activities : ・ national R&D projects on new energy -Current activities : ・ Support of industrial technology R&D (in 1988), ・ Support of environmental technology R&D (in 1990), and ・ Introduction and promotion of new energy and energy conservation technology( in1993). -NEDO became an Incorporated Administrative Agency as of October 2003 in the process of recent government reform.

4 Mission -NEDO’s current missions are : ・ Promote new energy and energy conservation technologies through R&D and subsidy programs, ・ Conduct and support industrial technology R&D, and ・ Enhance international cooperation concerning energy and other technologies. -ABOUT NEDO-

5 Organization Staff : approximately 1,000 Budget : US$2.1 billion (FY2005) -ABOUT NEDO- Machinery System Technology Development Dept. Environment Technology Development Dept. Fuel Cells & Hydrogen Technology Development Dept. New Energy Technology Development Dept. Energy Conservation Technology Development Dept. Coal Projects Dept., Coal Mine Subsidence Dept. Biotechnology & Medical Technology Development Dept. Nanotechnology & Materials Technology Development Dept. Electronic & Information Technology Development Dept. International Projects Dept. Energy and Environment Policy Dept. R&D Promotion Dept. R&D Project Evaluation Dept. Policy Planning and Coordination Dept General Management Depts. General Affairs; Accounting; Assets Management; Inspection and Operational Management; and Information and Systems. Chairman, President & Executive DirectorsAuditor Headquarters

6 R&D support -R&D support is NEDO’s core activity. ・ 58% of the total budget (US$1.2 billion) is allocated for R&D activities in FY ・ Number of NEDO’s R&D projects in FY2005 is ABOUT NEDO- - Electronics and information- Environment - Nanotechnology and materials- Fuel cells and hydrogen - Biotechnology and medical technology- New energy - Machinery systems- Energy conservation NEDO’s R&D Technology Areas

7 Challenge for NEDO -Need more effective and efficient operation through: ・ Improved services, and ・ Flexible resource management -ABOUT NEDO-

8 R&D Project Management Cycle PLAN DO SEE R&D Project Evaluation Dept. Board of Executive Directors Board of Executive Directors Technology Development Depts. Policy Planning and Coordination Dept. -R&D EVALUATION AT NEDO-

9 Purposes of R&D Project Evaluations as an important part of the R&D project management cycle. -To reflect economic and social needs on R&D activities, -To implement R&D projects more effectively and efficiently, and -To improve project management system -R&D EVALUATION AT NEDO-

10 Transparent -Public disclosure of evaluation results, methods, and reflections Explicit -Encourage discussions Effective -Adopt an evaluation method for self- improvement of management Neutral -External evaluators Efficient -Simple evaluation procedures 5 Principles of Evaluation at NEDO -R&D EVALUATION AT NEDO-

11 R&D Project Evaluations Post-project activityProject implementation Ex-ante evaluation (2004-) At R&D theme selection Ex-ante evaluation (2004-) At R&D theme selection Intermediate evaluation (2001-) In the middle of project term Intermediate evaluation (2001-) In the middle of project term Ex-post evaluation (2001-) 1 year following project completion Ex-post evaluation (2001-) 1 year following project completion ▼▼▼ ▼ Follow-up monitoring & evaluation (2004-) For 5 years after project completion Follow-up monitoring & evaluation (2004-) For 5 years after project completion -R&D EVALUATION AT NEDO-

12 Intermediate & Ex-post Evaluations -In all procedures, 5 principles are followed -Each project is assessed and evaluated from the following areas : ・ purpose and strategy, ・ project management, ・ R&D achievements, and ・ prospect for practical applications and other impacts -R&D EVALUATION AT NEDO-

13 Intermediate & Ex-post Evaluations - R&D Evaluation Methodology (Rating Method) 1. Each evaluator grades (A, B, C or D) for each evaluation item. A: Excellent, B: Good, C: Acceptable, D: Not acceptable 2. An average of rating is used as an indicator. A=3, B=2, C=1, D=0 Items for evaluation average ratings 1.Purpose and strategy 2.3 A A B B B B 2.Project management 1.8 A B C B C B 3.R&D achievement 1.5 B C C C B B 4. Prospect for practical applications and other impacts 1.2 B B D C C C -R&D EVALUATION AT NEDO- 【 example 】

14 -R&D EVALUATION AT NEDO ,10, , , , R&D achievement (X) Prospect for practical applications （Ｙ） X+Y=4 X+Y=3 Good Excellent Score Results for Ex-post Evaluation in FY2004

15 Intermediate & Ex-post Evaluations Public (citizens) NEDO Chairman Releasing of evaluation results to the public Deliberation and approval of the evaluation report (draft) Reflection of evaluation results in projects, etc. Promoting Department Evaluation report Preparation of evaluation report (draft) Promotion Department Project Participants Subcommittee A Subcommittee C Subcommittee B Subcommittee D Evaluation Secretariat Explanation of the project (4-9 persons ） (13 persons ） R&D Project Evaluation Committee -R&D EVALUATION AT NEDO-

16 -R&D EVALUATION AT NEDO- Intermediate & Ex-post Evaluations -In response to evaluation results: Lessons learned from ex-post evaluations may be reflected upon other on-going or new projects. Intermediate evaluations may result in modification or termination of projects. Responses to Intermediate Evaluation Results Accelerated Continued as planned 4212 Modified Terminated 2522 total * *some items are overlapping: all items added up do not match the total (start)

17 -R&D EVALUATION AT NEDO- Follow-up Monitoring & Evaluations (since 2004) -Objectives of this new initiative are: ・ To understand in what ways specifically the outcome of R&D provides benefits to the public ・ To reflect the outcome of R&D in improvement of the management and planning function at NEDO

18 Follow-up Monitoring & Evaluations -R&D EVALUATION AT NEDO- Tracked with the Follow-up Monitoring Post-Project Activities NEDO’s R&D Project (3-5 years) R&D Practical Applications Commercialization Post-Project Activities of Private Companies

19 Follow-up Monitoring & Evaluations -NEDO initiatives as of October 2005: Subjects: 56 R&D projects, 642 entities ・ These projects subject to follow-up monitoring & evaluations have received ex-post evaluation in FY ・ These entities include private companies, universities, research institutes, and public institutions. -R&D EVALUATION AT NEDO-

20 Follow-up Monitoring & Evaluations Findings: -64% of the above entities are currently conducting post- project activities. -22% of the entities have reached a stage of practical applications or commercialization of R&D achievements. *Detailed information is being collected from 51 entities. Utilization: -Make suggestions for improvement -Document key points to R&D management as a guideline -R&D EVALUATION AT NEDO-

21 ・ Evaluations will play an increasingly important role in: -Improving performance of NEDO as an Incorporated Administrative Agency, and -Facilitating effective and efficient implementation of R&D project managements. CLOSING

22 CLOSING NEDO intends to ・ Review the current evaluation system ・ Make appropriate modifications ・ Propose new initiatives

23 Thank you for your kind attention. To be continued…