RECENT DEVELOPMENT OF SORS METADATA REPOSITORIES FOR FASTER AND MORE TRANSPARENT PRODUCTION PROCESS Work Session on Statistical Metadata 9-11 February 2004 Andreja Arnič and Julija Kutin Statistical Office of the Republic of Slovenia

Overview  Introduction  Recent development of SORS metadata repositories  Classification server  METIS repository  Findings

INTRODUCTION  metadata within the statistical production process  centralized repositories  SWOT analysis  E-CoRE  further development of the classification server (KLASJE)  notifications  e-government: searching within classifications  E-KLASJE: feasibility study

RECENT DEVELOPMENT OF METADATA REPOSITORIES  The main goal: develop an efficient and effective, standardized and integrated system for collecting and editing metadata.  From that system metadata could be quickly and easy exported and used in other applications and programs.

RECENT DEVELOPMENT OF METADATA REPOSITORIES Whole life cycle of a statistical survey to be covered by metadata or a statistical information system:  including design  implementation  operation  monitoring  maintenance  evaluation

CLASSIFICATION SERVER  KLASJE enables various contents and time comparisons between classification versions.  KLASJE represents the basic metadata infrastructure  enabled the standardization of processes  directly influenced the quality of statistical data  at the end of January 2004: 730 classifications and 75 concordances

CLASSIFICATION SERVER Further development of the Classification server and e- government initiative  automatic load of dimensional tables in data warehouse  via the Internet: additional search facilities concordances available  e-government initiative: feasibility study  notifications revealing changes in classifications

METIS REPOSITORY  In production from 2003 and we are working on  Questionnaires and Methodology (variables)  Application functionality: import, preparation and export metadata for special use.  Standard solutions: technology, content  Standardized statistical process (standard tools for each sub process)  Metadata play a major role in dissemination of statistics (standard tools and approaches)  The other functions of metadata in statistical process

METIS REPOSITORY We started with detailed analysis of data collection sub process and revision of this module: Eliminations, The others (IQML, Blaise ) have some metadata we don’t have. We analyzed 7 typical SORS’s questionnaires from methodology definition to the FOR – micro database.

Variables, other definitions Auxiliary information Questionnaire Editing and validation rules Automation of editing rules Data editing 2. Survey design 4.1. Data editing Contents resources legal basis 1. Methodology Sampling design 3. Survey processing Questionnaires Questionnaire dispatch Raw data entry Raw data collection Data validation Aggregations – IT support Data quality assessment Data analysis Weighting Imputation Data analysis Estimated aggregations – requirements Steps in survey design and implementation - from general framework to edited (clean) microdata Specific processing in each survey instance Database loading 5. Final observation register (FOR) creation Files update Database creation Data matrix definition a) File systems update b) Microdata bases loading Documentation Directory of folders

METIS REPOSITORY  Some possibilities were offered: electronic form as option next to paper form, reduce periodicity, reduce frequency, improve/introduce explanations/instructions, simplifications, integrations, adapting forms to source administrations / common practices of enterprises.

METIS REPOSITORY  Then we found out a lot of problems: layout of questionnaires, general data on the questionnaire, lack of documentation available, difficult or impossible to map survey data and respondents list for the same reference period without personal contact.

FINDINGS  The statistical process needs to be constantly analyzed and modernized.  Regular training and cooperation in international exchange of knowledge and experience needs to be organized.  Documents processes and procedures need to be standardized.  Deadlines need to be set and monitored.  Users and producers satisfaction needs to be measured. We are working on the pilot from methodology definition to the FOR for one survey in witch we will try to define the metadata in each sub process.