Presentation is loading. Please wait.

Presentation is loading. Please wait.

S-DWH layered architecture – Statiscs Finland

Similar presentations


Presentation on theme: "S-DWH layered architecture – Statiscs Finland"— Presentation transcript:

1 S-DWH layered architecture – Statiscs Finland

2 NSI: Statistics Finland Name: Antti Santaharju
Date: Institutional Output Reporting Analysis Data mining Meta Data & Process Control System Macro Data Data Acces Layer GSBPM Phase 7 Disseminate Micro Data YTY S-DWH for validated micro data Data cubes Future plan Data cubes Interpretation and data analysis Layer GSBPM Phase 6 Analysis YTY operational database Reports Integration Layer GSBPM Phase 5 Process Statistics Finland’s other databases Sources Layer GSBPM Phase 4 Collect 11 Direct surveys 16 Administrative Data sources

3 YTY business statistics production system
”Dependent” statistics of the integrated YTY operational database Business Register Some SBS and other statistics: * Financial statement * Regional and industrial statistics * International trade in services * FATS statistics (inward and outward) * Industrial output /Commodity (PRODCOM) STS statistics (partial integration) Turnover indices Wage and salary indices Antti Santaharju

4 1. YTY Source layer Direct surveys are carried out by web application
Administrative datasets (sequential text files) are transmitted from other administrative institutes to Statistics Finland as sequential textfiles All data sources (and every variable) are described in the Statistics Finland’s metadata database Metadata descriptions are accessible by Variable Editor application Using the metadata received administrative datasets are converted into SAS datasets are technically validated All data sources (external and internal) are stored into YTY Source layer as SAS datasets (raw data warehouse) All the processing is controlled by tailor made process control system process engine X (procX) Antti Santaharju

5 YTY Integration layer & YTY Interpretation and data analysis layer
YTY Integration layer & YTY Interpretation and data analysis layer work in a single database called YTY operational database Microsoft SQL2012 database All database tables are described in the Statistics Finland’s metadata database All the processing is controlled by Statistics Finland’s tailor made process control system procX Antti Santaharju

6 2. YTY Integration layer Data sources are uploaded to YTY integration layer By SAS programs Some information is extracted directly from Statistics Finland’s other databases to YTY operational database Input for estimation Data update launches modular data integration, coding, validating, editing and imputing processes These process modules are SAS/.Net programs Erroneous units are flagged Flagged units are analyzed and edited manually by Statistics Finland's tailor made .Net application Tietopalveluyksikkö/Viestintä

7 3. Interpretation and data analysis layer
Data analysis and data mining is done by SSAS Database cubes MS Excel SAS EG Microsoft SQL2012 report builder reports Data analysis and data mining is based on real time data in YTY production database Tietopalveluyksikkö/Viestintä

8 4. Access layer All validated micro data are loaded to access layer
Microsoft SQL2012 server database for validated micro data Data analysis, data mining, dissemination and delivery is based on SSAS Database cubes Daily updated MOLAP cubes One data cube for each statistics Publication process creates frozen micro data version into database Publication process tools: SAS, Tau-Argus, PC-Axis, PX Web.. In future Some validated information is loaded directly from other production databases to YTY S-DWH for validated micro data (YTY Access layer) Tietopalveluyksikkö/Viestintä


Download ppt "S-DWH layered architecture – Statiscs Finland"

Similar presentations


Ads by Google