TNO Quality Management EARTO Working Group on Quality and Excellence

April TNO Quality Management 1 Content Introduction of TNO TNO audits (general) TNO Cofinancing Programme (specific example) Customer satisfaction / impact assessment Conclusions

April TNO Quality Management 2 TNO: founded by law (1932) Article 4 TNO Act The Organisation aims to ensure that applied engineering and scientific research and other associated social scientific and applied research is made suitably serviceable for the general good and the distinctive interests contained therein. TNO caters to industry and government needs for specific R&D TNO is independent of public and private interests

April TNO Quality Management 3 Core data 2010 (amounts x EUR 1 m.) incl. group companies % 4, % 4, % 4,572 Operating result Result Balance Sheet total Equity capital Consolidated income share of government funding share of market income Solvency Number of employees (effective average)

April TNO Quality Management 4 TNO in the Netherlands employees 564 M€ turnover

April TNO Quality Management 5 Themes and innovation areas

April TNO Quality Management 6 Areas of expertise Technical Sciences Behavioural and Societal Sciences Earth, Environmental and Life Sciences

April TNO Quality Management The TNO organisation Supervisory Board Board of Management Council for Defence Research Corporate Staff Knowledge Centres TNO Companies 3 Expertise lines 7 Theme lines

April TNO Quality Management 8 Quality assurance Projects are evaluated Regular customer satisfaction audits International technology position audit Constructing roadmaps with business relations Employees satisfaction audit All relevant quality systems (ISO, NEN, etc) NB: Theme line vs Expertise line

April TNO Quality Management 9 TNO Quality System Complies to the demands of ISO 9001:2000 Applicable to the whole TNO organisation For some working areas: laboratory accreditation, Good Laboratory Practice, Good Clinical Practice

April TNO Quality Management 10 Periodical Auditing Employee Satisfaction (3-yearly) Internal (intranet) enquiry with additional interviews (external advisors) Customer Satisfaction Enquiry form after each project (operational unit) interviews (external advisors) (samples) (corporate) Q-system (ISO) Yearly audit (DNV) Technology Position of Research Groups (4-yearly)

April TNO Quality Management 11 TNO Cofinancing programme Aim: TNO Research together with industry (SME and large) Research budget Ministry of Economic Affairs with cofinancing from industry (total budget 40 million EUR p.a. i.e. 7% of TNO turnover) cofinancing: 10% - 25% - 50% 250 projects running, size kEuro inherently complicated projects with multiple stakeholders Conditions cf Community framework for state aid for research and development and innovation IPR for TNO, right of use for cofinancer

April TNO Quality Management 12 TNO Cofinancing programme auditing After finalizing project (about 50 projects, 60 cofinanciers p.a.) customer (i.e. cofinancer) audit (external advisor, by telephone) satisfaction (results, services) (aimed) application of results 1-3 years after finalizing projects internal impact (spin-off projects, licences) economic impact cofinancer (external advisor, by telephone) (attained) application of results, extra turnover, benefits

April TNO Quality Management 13 Scaling Issue for discussion General satisfaction: scale 1-10 Specific aspects: scale 1-5 very satisfied satisfied satisfied nor dissatisfied dissatisfied very dissatisfied Choice determines acceptable targets

April TNO Quality Management 14 Impact assessment Issue for discussion Impact of TNO projects to individual customers (economic or other) Impact of TNO programmes/projects to NL en EU society

April TNO Quality Management 15 Reporting Report by the advisor contains both ratings as well as citations Report presented internally: TNO Board of Management Individual interview forms to management of operational units Follow up by operational units Management Summary presented to: Ministry of Economic Affairs Aim of the audit: improvement of quality of projects relation with (cofinancing) industry relation with and account to the Government

April TNO Quality Management 16 Conclusions Customer satisfaction measurements are standard at TNO Corporate coordination required? Interviews by telephone enhance the response Mutual commitment in cofinancing projects gives opportunity for ‘sensitive’ questions to assess impact Used for internal feedback as well as external accountability Impact assessment is a real challenge