Surveying eValues: Experiences and Challenges Measuring Information Society in the Community Survey on ICT Usage and e-Commerce in Enterprises Fernando Reis EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics 19th Bled eConference "eValues" June 5 - 7, 2006 Bled, Slovenia IndicatorsIndicatorsIndicatorsIndicators

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Information Society statistics at Eurostat The role of Eurostat Provide the European Commission with statistics The role of official statistics Large scale representative and harmonised statistics Origins of IS statistics in various ICT surveys in the Member States Big boost: Lisbon process and eEurope Action Plans, which included a benchmarking exercise for coherent and comparable statistics at European level Two surveys on ICT usage: - Enterprise Survey (since 2001) - Household / Individuals Survey (since 2002) Legal basis: Framework Regulation April 2004, implementing regulation (for the 2006 survey) published on 13/07/2005

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics The Community Survey on ICT Usage and e-Commerce in Enterprises Network of national official surveys on ICT uptake 30 countries (EU MS + IS + NO + RO + BG + TR) Net sample of around enterprises (5 000 per country) 66 variables collected: – Basic ICT uptake (e.g. computers, LAN) – Internet use – e-Commerce – e-Business (e.g. integration of business processes) 2 background variables used to breakdown results: – Size (3 categories) – Economic activity (14 categories)

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Contents of the survey Type of questions: – Simple format: most questions/items only require a “Yes” or “No” answer – e-Commerce questions asked in terms of percentages – But sometimes difficult topics … (e.g. advanced uses of e-business solutions sometimes difficult to understand by some respondents) Scope / Target population – Sections D to K of NACE + Divisions 92 and 93 – Financial sector covered by a specific model questionnaire – Enterprises with 10 or more persons employed (optionally enterprises with less than 10)

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Results of collections from 2003 to 2005 Computer systems and their integration - (E.U.)

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Coordination of ICT statistics in the EU Every year: model questionnaire developed by Eurostat in coordination with the national statistical institutes (via Task Forces and Working Groups) and other services of the European Commission (DG INFSO and DG ENTR) Data collection by NSI’s and transmitted to Eurostat Publications by the NSI’s as well as by Eurostat

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Survey and reference period Reference period – previous calendar year for the monetary variables (turnover and purchases) and e-government; – January of survey year for other variables. Survey period – recommendation is first quarter of the survey year to keep it as close as possible to the reference period (reduce recall bias) and to allow the release of the results as soon as possible – But for practical reasons sometimes different: Internal planning of the NSI Data collection linked to other surveys

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Survey period (2005 survey)

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Sampling: frame = the frame from which the sample will be drawn  Should be as close as possible to the target population as both overcoverage and undercoverage can induce bias and affect the reliability and comparability of the survey results. – E.g. overcoverage due to inclusion of enterprises with less than 10 persons employed in the sampling frame: as ICT usage in these enterprises is lower than the average, the population total will show a downwards bias.  Due to:  Low quality of the register;  Instability of the population (high percentage of enterprises changing size class and/or economic activity).  Correction: 1. Collection of the background variables in the survey; 2. Reclassification of the respondents in the new/correct categories before grossing-up.  Recommendation:  Use the same frame as the structural business statistics.

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Sampling: design Highly harmonised between countries:  Stratified random sampling in most countries (smaller countries – e.g. LU, CY, MT - may use census)  Stratification based on the background variables (persons employed and economic activity) and for the majority of the countries also the region.  Sometimes the stratification is more detailed  But different approaches are allowed  … assuming that different ways to draw a representative sample should nevertheless lead to comparable and harmonised data  … NSI’s are best placed to decide on sampling design (competence, ‘local’ knowledge, …)

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Sampling: sample size In practice:  Output (indicators): proportions or percentages  Requires smaller sample sizes then for estimating totals;  Very dependent on population size: small countries with small populations require higher sampling rates;  EU25 countries: average net sample size around 5000 enterprises;  Gives very good results at overall level;  Sometimes problematic for more detailed information;  E.g.: the distribution of the enterprises by class of the percentage of turnover from e-commerce, cannot be accurately estimated for several countries because not enough respondents;  Decision also depends on national data needs (e.g. specific need for data at regional level)  E.g.: DE, ES, IT which produce regional breakdown have average net sample size of enterprises.

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Survey type (I) Data collection method:  The vast majority of EU25 Member States use postal surveys (self-administered)  Many countries when launching the survey in the firsts years used face-to-face interviews  Other collection methods are combined to increase response rate (contact by telephone, electronic questionnaire, etc.)  Face-to-face or telephone interviews are usually preferred to postal surveys (interaction between the respondent and the interviewer can be important for the understanding of the questions and the answers)  Postal surveys: lower survey costs (and can give the respondent the time to reflect on his/her responses) and e-commerce information require enterprises to consult their accounts or other registers  But … in general budgetary constraints will be the determining factor …

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Survey type (II) Survey vehicle:  Independent, stand-alone survey  Embedded survey  Embedded survey was sometimes used in the first years (e.g. structural business statistics survey - SBS);  Currently, all EU25 Member-States use stand-alone surveys to collect ICT usage and e-commerce variables + SBS surveys are usually slow to release results and timeliness is very important for ICT statistics - requires the coordination of the economic variables (employment, turnover, purchases) with the SBS with which ICT statistics need to be coherent - Crossing of ICT use data with economic performance data (from SBS) is more difficult because samples don’t overlap

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Response and non-response  Countries’ experience show that the final response rate is on average 74%  When aiming at a net sample (= number of elements in the final database) of e.g individuals, it may be necessary to select 4000 individuals.  Risk of biased results: non-users of ICT may be more inclined not to return the questionnaire filled in  Importance of imputation (e.g. post-stratification methods to improve the representativeness of the estimates)  Item non-response is low (excepting for a few complex questions)  Some methods: see the Methodological Manual

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Questionnaire design How to measure such a complex subject as e-business – Needs to be general – applicable to all economic activities and all enterprise sizes; – Needs to be specific – e.g. references to examples; – Needs to be explicit – no references to abstract concepts, but to events or situations easily identifiable by the respondents.

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Three main problems to be tackled in the next five years 1. High sensitivity of the results to questionnaire wording 2. Measurement of the impact of ICT in enterprises 3. Development of a conceptual framework for statistical measurement of ICT usage and the Information Society

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics Contact and Community Survey in the Internet

EUROPEAN COMMISSION EUROSTAT Directorate F: Social Statistics and Information Society Unit F-6: Information Society and Tourism Statistics Information Society Statistics in the Internet: EUROSTAT website -> Themes -> Science and technology -> Data -> Information society statistics ICT usage data for the EU  Statistics are available free of charge on Eurostat’s website:   Click « Data »  Click « Science and technology »  Via the NewCronos database, tables are defined by the user on extraction  Metadata including a first release of the Methodological Manual: