Statistics Estonia's new system for statistical data activity processing (VAIS) ITDG Luxembourg 2010 Allan Randlepp.

Slides:

Advertisements

Similar presentations

Configuration management

Advertisements

Statistics 2020 and Platform Approach Te Käpehu Whetü May 2011.

Software Construction

APPLIED GSBPM IN GSO by Ha Do Statistical Standard Methodology and ITC Department General Statistic Office Vietnam 1 General statistic office Vietnam.

NESSTAR Limitedw w w. n e s s t a r. c o m DDI-Publishing Made Easy- the Nesstar Way Jostein Ryssevik Nesstar Ltd.

Software Factory Assembling Applications with Models, Patterns, Frameworks and Tools Anna Liu Senior Architect Advisor Microsoft Australia.

1 Business Exchange Structures Concepts.

by Ha Do Statistical Standard Methodology and ITC Department

UNIT-V The MVC architecture and Struts Framework.

United Nations Economic Commission for Europe Statistical Division Applying the GSBPM to Business Register Management Steven Vale UNECE

Open source administration software for education software development simplified KRAD Kuali Application Development Framework.

MDC Open Information Model West Virginia University CS486 Presentation Feb 18, 2000 Lijian Liu (OIM:

WP.5 - DDI-SDMX Integration

METS-Based Cataloging Toolkit for Digital Library Management System Dong, Li Tsinghua University Library

Katanosh Morovat.   This concept is a formal approach for identifying the rules that encapsulate the structure, constraint, and control of the operation.

WP.5 - DDI-SDMX Integration E.S.S. cross-cutting project on Information Models and Standards Marco Pellegrino, Denis Grofils Eurostat METIS Work Session6-8.

Survey Data Management and Combined use of DDI and SDMX DDI and SDMX use case Labor Force Statistics.

Zhonghua Qu and Ovidiu Daescu December 24, 2009 University of Texas at Dallas.

Metadata management and statistical business process at Statistics Estonia Work Session on Statistical Metadata (Geneva, Switzerland 8-10 May 2013) Kaja.

Development of metadata in the National Statistical Institute of Spain Work Session on Statistical Metadata Genève, 6-8 May-2013 Ana Isabel Sánchez-Luengo.

EMI INFSO-RI SA2 - Quality Assurance Alberto Aimar (CERN) SA2 Leader EMI First EC Review 22 June 2011, Brussels.

The Adoption of METIS GSBPM in Statistics Denmark.

Configuration Management (CM)

5 June 2013 SDMX Technical Working Group Luxembourg 1 5 June 2013 SDMX Technical Working Group Luxembourg 1 WP Item 6 The Expressions Language of Banca.

Current and Future Applications of the Generic Statistical Business Process Model at Statistics Canada Laurie Reedman and Claude Julien May 5, 2010.

1 Rake. 2 Automated Build Any non-trivial project needs facility to automate builds –Routine common tasks that need to be carried out several times a.

11 CORE Architecture Mauro Bruno, Monica Scannapieco, Carlo Vaccari, Giulia Vaste Antonino Virgillito, Diego Zardetto (Istat)

PHP Features. Features Clean syntax. Object-oriented fundamentals. An extensible architecture that encourages innovation. Support for both current and.

MODA-TEL An IST project on Model Driven Architectures for Telecommunications System Development and Operation.

Metadata driven application for data processing – from local toward global solution Rudi Seljak Statistical Office of the Republic of Slovenia.

United Nations Economic Commission for Europe Statistical Division Mapping Data Production Processes to the GSBPM Steven Vale UNECE

LCG-SPI: SW-Testing LCG AppArea internal review (20/10/03)

United Nations Economic Commission for Europe Statistical Division High-Level Group Achievements and Plans Steven Vale UNECE

ILO Department of Statistics Edgardo Greising

Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,

Open GSBPM compliant data processing system in Statistics Estonia (VAIS) 2011 MSIS Conference Maia Ennok Head of Data Warehouse Service Data Processing.

August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.

Toward an Open Source Textual Entailment Platform (Excitement Project) Bernardo Magnini (on behalf of the Excitement consortium) 1 STS workshop, NYC March.

RECENT DEVELOPMENT OF SORS METADATA REPOSITORIES FOR FASTER AND MORE TRANSPARENT PRODUCTION PROCESS Work Session on Statistical Metadata 9-11 February.

CoRD Meeting 12 March 2003 STIPES (Lot 4) STIPES = Statistical Inquiries from Popular European Software.

SEESCOASEESCOA SEESCOA Meeting Activities of LUC 9 May 2003.

TRIUMF HLA Development High Level Applications Perform tasks of accelerator and beam control at control- room level, directly interfacing with operators.

Introduction to Quality Management Frameworks Eurostat, Luxembourg, January 2016 Process quality Dr Johanna Laiho-Kauranne.

Viewpoint Modeling and Model-Based Media Generation for Systems Engineers Automatic View and Document Generation for Scalable Model- Based Engineering.

De Rigueur - Adding Process to Your Business Analytics Environment Diane Hatcher, SAS Institute Inc, Cary, NC Falko Schulz, SAS Institute Australia., Brisbane,

Esri UC 2014 | Technical Workshop | Address Maps and Apps for State and Local Government Allison Muise Nikki Golding Scott Oppmann.

Introduction to Statistics Estonia Study visit of the State Statistical Service of Ukraine on Dissemination of Statistical Information and related themes.

M&CML: A Monitoring & Control Specification Modeling Language

The Holmes Platform and Applications

Leveraging R and Shiny for Point and Click ADaM Analysis

Accelerate define.xml using defineReady - Saravanan June 17, 2015.

Session 8 Data Processing Estonian case study

S-DWH layered architecture – Statiscs Finland

Generic Statistical Business Process Model (GSBPM)

YTY − an integrated production system for business statistics

Tomaž Špeh, Rudi Seljak Statistical Office of the Republic of Slovenia

ESSnet on Data Warehousing 4th Workshop Maia Ennok 20th. of March 2013

Validation at Statistics Sweden

Metadata Framework as the basis for Metadata-driven Architecture

SDMX in the S-DWH Layered Architecture

Execute your Processes

Java Workflow Tooling (JWT) Release review: JWT v0

The JSF Tools Project – WTP (internal) release review

Mapping Data Production Processes to the GSBPM

Metadata used throughout statistics production

SOA initiatives at Istat

Work Session on Statistical Metadata (Geneva, Switzerland May 2013)

Technical Coordination Group, Zagreb, Croatia, 26 January 2018

Validation Activities in the ESS What you will hear today…

Integrated Statistical Production System WITH GSBPM

Presentation transcript:

Statistics Estonia's new system for statistical data activity processing (VAIS) ITDG Luxembourg 2010 Allan Randlepp

What it is? VAIS (pronounced as wise) is a collection of tools and technologies aimed at automating data processing (Phase 5 in GSBPM). In essence, the task of check, clean, and transforming statistical activity data can be identified as taking the raw data from one or more sources and transforming it to analytical system source data input data base structures. Statistics Estonia's new system for statistical data activity processing (VAIS)

Framework for … Integrate data Cassify & code Review, validate and edit Impute Derive new variables & statistical units Calculate weights Calculate aggregates Finalize data files Statistics Estonia's new system for statistical data activity processing (VAIS)

Metadata driven template based tool Template driven approach provides an universal solution for three main goals of the VAIS project: Create an easy to use statistical data processing tool requiring minimal programming skills for transformation package creation. Create a metadata driven process-oriented and automated statistical data processing tool. Create an extendable data transformation tool. Statistics Estonia's new system for statistical data activity processing (VAIS)

Data processing with VAIS Statistics Estonia's new system for statistical data activity processing (VAIS)

Balancing automation and manual intervention Statistics Estonia's new system for statistical data activity processing (VAIS) RAW data Metadata (validation and transformation rules) Automated data processing Manual data processing OK? Input data

Technical platform VAIS is built on open-sourced freely available technological components. XDTL (eXtensible Data Transformation Language – an XML based descriptional language designed for specifying data transformations, see run-time engine (XDTL RT). MMX Metadata Repository, part of Metadata Framework (a MOF compliant metadata management environment designed with a wide variety of metadata-driven applications in mind, see Apache Foundation's Velocity template engine ( is used as the template engine combining excellent template rendering functionality with very easy to use template language. The user applications are programmed in Java, based on Wicket MVC framework ( Quartz scheduling framework ( is used for execution scheduling. Statistics Estonia's new system for statistical data activity processing (VAIS)