1. TNO Quality Management EARTO Working Group on Quality and Excellence

2. April 8 2011 TNO Quality Management 1 Content Introduction of TNO TNO audits (general) TNO Cofinancing Programme (specific example) Customer satisfaction / impact assessment Conclusions

3. April 8 2011 TNO Quality Management 2 TNO: founded by law (1932) Article 4 TNO Act The Organisation aims to ensure that applied engineering and scientific research and other associated social scientific and applied research is made suitably serviceable for the general good and the distinctive interests contained therein. TNO caters to industry and government needs for specific R&D TNO is independent of public and private interests 7-12-2015 2

4. April 8 2011 TNO Quality Management 3 Core data 2010 (amounts x EUR 1 m.) incl. group companies 2009 2010 564 195 369 568 203 365 2.8 2.4 441 -16.7 -14.2 434 190 43% 4,189 187 43% 4,424 2008 593 195 398 -10.6 -5.7 455 201 44% 4,572 Operating result Result Balance Sheet total Equity capital Consolidated income share of government funding share of market income Solvency Number of employees (effective average) 7-12-2015 3

5. April 8 2011 TNO Quality Management 4 TNO in the Netherlands 7-12-2015 4 4100 employees 564 M€ turnover

6. April 8 2011 TNO Quality Management 5 Themes and innovation areas 7-12-2015 5

7. April 8 2011 TNO Quality Management 6 Areas of expertise Technical Sciences Behavioural and Societal Sciences Earth, Environmental and Life Sciences 7-12-2015 6

8. April 8 2011 TNO Quality Management 7 7-12-2015 7 The TNO organisation Supervisory Board Board of Management Council for Defence Research Corporate Staff Knowledge Centres TNO Companies 3 Expertise lines 7 Theme lines

9. April 8 2011 TNO Quality Management 8 Quality assurance Projects are evaluated Regular customer satisfaction audits International technology position audit Constructing roadmaps with business relations Employees satisfaction audit All relevant quality systems (ISO, NEN, etc) NB: Theme line vs Expertise line 7-12-2015 8

10. April 8 2011 TNO Quality Management 9 TNO Quality System Complies to the demands of ISO 9001:2000 Applicable to the whole TNO organisation For some working areas: laboratory accreditation, Good Laboratory Practice, Good Clinical Practice

11. April 8 2011 TNO Quality Management 10 Periodical Auditing Employee Satisfaction (3-yearly) Internal (intranet) enquiry with additional interviews (external advisors) Customer Satisfaction Enquiry form after each project (operational unit) interviews (external advisors) (samples) (corporate) Q-system (ISO) Yearly audit (DNV) Technology Position of Research Groups (4-yearly)

12. April 8 2011 TNO Quality Management 11 TNO Cofinancing programme Aim: TNO Research together with industry (SME and large) Research budget Ministry of Economic Affairs with cofinancing from industry (total budget 40 million EUR p.a. i.e. 7% of TNO turnover) cofinancing: 10% - 25% - 50% 250 projects running, size 100-2000 kEuro inherently complicated projects with multiple stakeholders Conditions cf Community framework for state aid for research and development and innovation IPR for TNO, right of use for cofinancer

13. April 8 2011 TNO Quality Management 12 TNO Cofinancing programme auditing After finalizing project (about 50 projects, 60 cofinanciers p.a.) customer (i.e. cofinancer) audit (external advisor, by telephone) satisfaction (results, services) (aimed) application of results 1-3 years after finalizing projects internal impact (spin-off projects, licences) economic impact cofinancer (external advisor, by telephone) (attained) application of results, extra turnover, benefits

14. April 8 2011 TNO Quality Management 13 Scaling Issue for discussion General satisfaction: scale 1-10 Specific aspects: scale 1-5 very satisfied satisfied satisfied nor dissatisfied dissatisfied very dissatisfied Choice determines acceptable targets

15. April 8 2011 TNO Quality Management 14 Impact assessment Issue for discussion Impact of TNO projects to individual customers (economic or other) Impact of TNO programmes/projects to NL en EU society

16. April 8 2011 TNO Quality Management 15 Reporting Report by the advisor contains both ratings as well as citations Report presented internally: TNO Board of Management Individual interview forms to management of operational units Follow up by operational units Management Summary presented to: Ministry of Economic Affairs Aim of the audit: improvement of quality of projects relation with (cofinancing) industry relation with and account to the Government

17. April 8 2011 TNO Quality Management 16 Conclusions Customer satisfaction measurements are standard at TNO Corporate coordination required? Interviews by telephone enhance the response Mutual commitment in cofinancing projects gives opportunity for ‘sensitive’ questions to assess impact Used for internal feedback as well as external accountability Impact assessment is a real challenge