1. www.uis.unesco.org Innovation data collection: Advice from the Oslo Manual South East Asian Regional Workshop on Science, Technology and Innovation Statistics Hanoi, Viet Nam 5-8 December 2011 Luciana Marins

2. www.uis.unesco.org Ch 8 OM - Innovation Survey Procedures Guidelines - collection and analysis of innovation data; Comparable results over time and across countries; Particular circumstances may require other methodology comparability.

3. www.uis.unesco.org The survey approach The subject approach: Innovative behaviour and activities of the firm as a whole; The object approach: Specific innovations (significant innovation of some kind, firms main innovation).

4. www.uis.unesco.org Populations (1) The target population: The target population: Innovation activities in the business enterprise sector (goods-producing and services industries); Minimum: all statistical units with at least ten employees: Small: 10-49; Medium: 50-249; Large: 250 and above.

5. www.uis.unesco.org Populations (2) The frame population: The frame population: Units from which a survey sample or census is drawn; Based on the last year of the observation period for surveys; Ideal frame = up-to-date official business register NSOs; If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.

6. www.uis.unesco.org Survey methods (1) Mandatory Mandatory surveys increase response rates; Census or sample surveys? Census or sample surveys? Sample surveys - representative of target population (industry, size, region) stratified sample; Census - costly but unavoidable in some cases.

7. www.uis.unesco.org Survey methods (2) Domains (sub-populations): Domains (sub-populations): Subsets of the sampling strata; Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active; Guidelines: » Same statistical units and classifications; » Consistence of the methods for results calculation; » Documentation of deviations in data treatment or differences in the quality of the results from the domains.

8. www.uis.unesco.org Survey methods (3) Sampling techniques: Sampling techniques: Stratified sample surveys: size and principal activity; Sampling fractions should not be the same for all strata; Cross-sections: Cross-sections: standard approach - new random sample for each innovation survey; Panel data: Panel data: alternative/supplementary approach.

9. www.uis.unesco.org Survey methods (4) Suitable respondents: Suitable respondents: Methods - ex: postal surveys, personal interviews; Units most suitable respondent - very specialised questions that can be answered by only a few people; Try to identify respondents by name before data collection starts. Try to identify respondents by name before data collection starts.

10. www.uis.unesco.org Survey methods (5) The questionnaire: The questionnaire: Pre-test; Simple and short; Order of the questions; Questions on qualitative indicators - binary or ordinal scale; International innovation surveys - attention to translation and design; Short-form questionnaires - units with little/no innovation activity previously reported.

11. www.uis.unesco.org Survey methods (6) Combination of Innovation and R&D surveys: Combination of Innovation and R&D surveys: Reduction in the overall response burden; Scope for analysing the relations between R&D and innovation activities; Increase in the frequency of innovation surveys; Country experiences - it is possible to obtain reliable results for R&D expenditures; Longer questionnaire; Units not familiar with the concepts of R&D and innovation may confuse them; Different frames for the two surveys.

12. www.uis.unesco.org Survey methods (7) Guidelines for conducting combined surveys: Guidelines for conducting combined surveys: Questionnaire - two distinct sections; Smaller individual sections; Comparison of results from combined and stand-alone surveys should be done with care - surveying methods should be reported; Samples extraction from a common business register.

13. www.uis.unesco.org Estimation of results (1) Weighting methods: Weighting methods: Weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non- response; If a stratified sampling technique with different sampling fractions is used, weights should be calculated individually for each; Based on the number of enterprises in a stratum; International and other comparisons: same weighting method.

14. www.uis.unesco.org Estimation of results (2) Non-response: Non-response: Unit non-response: reporting unit does not reply at all; Item non-response: response rate to a specific question - percentage of blank or missing answers; biased results » Disregarding missing values + applying simple weighting procedures based on the responses received assumes that respondents and non-respondents are distributed in the same way biased results; imputation methods » Possibility: imputation methods.

15. www.uis.unesco.org Presentation of results Variance for the results: (average) values for innovation indicators and their coefficients of variation and/or confidence intervals; Results presentation: metadata (including data collection procedure), sampling methods, procedures for dealing with non-response and quality indicators.

16. www.uis.unesco.org Frequency of data collection Every two years; If not economically feasible frequency of three or four years; Specify an observation period; The length of the observation period for innovation surveys should not exceed three years nor be less than one year.

17. www.uis.unesco.org Annex A - 5. Methodological issues for developing country contexts (1) Information system specificities: Relative weakness of statistical systems: » Absence of linkages between surveys and data sets; » Lack of official business registers; Involvement of NSOs; When lacking, basic variables about firms performance can be included in the innovation survey.

18. www.uis.unesco.org Annex A - 5. Methodological issues for developing country contexts (2) General methodological considerations: Survey application: » In-person; » Trained personnel; Questionnaire design: » Sections can be separated to allow different persons in the firm to reply them; » Guidance/definitions; » Language and translation of technical terms.

19. www.uis.unesco.org Annex A - 5. Methodological issues for developing country contexts (3) General methodological considerations: Frequency: CIS CIS » Every three to four years (e.g., timed to CIS rounds);CIS » Update a minimum set of variables every year; Purpose of surveys; Clear questions; Adequate legislative base; The results should be published and distributed widely. The results should be published and distributed widely.

20. www.uis.unesco.org Example - product innovation/degree of novelty

21. www.uis.unesco.org Example - innovation activities and expenditures for product and process innovations

22. www.uis.unesco.org Example - organisational innovation

23. www.uis.unesco.org Example - co-operation

24. www.uis.unesco.org Example - hampering factors

25. www.uis.unesco.org Thank you! http://www.uis.unesco.org l.marins@unesco.org