Pekka Korhonen - Statistical services The publishing system of statistics in Natural Resources Institute Finland (LUKE) Pekka Korhonen - Statistical services

Natural Resources Institute Finland (LUKE) Established 1.1.2015 Fusion: MTT (Agrifood Research Finland), Metla (Finnish Forest Research Institute), RKTL (Finnish Game and Fisheries Research Institute) & MMM Tike (Information Centre of the Ministry of Agriculture and Forestry) Statistical Services Unit Official statistical authority - Official Statistics of Finland (OSF) Statistics on agriculture, forestry and fisheries 62 statistics (1.11.2017) Agriculture: 27 statistics Forestry: 25 statistics Fisheries and game: 10 statistics Indicators Agrikaattori: The EU Common Agricultural Policy (CAP): 38 indicators Forest Indicators (coming) Total 422 statistical tables (1.11.2017) Pekka Korhonen - Statistical services 15.11.2017

The structure of the statistics publishing system of LUKE 2 separate parts those are linked to each other Statistics Portal (html site): http://stat.luke.fi/en basic maintenance, development and location purchased from outside normal html pages (drupal), tailored to TiPa TiPa has its own maintenance interface Statistics Database (PX-Web): http://statdb.luke.fi Virtual Machines - maintained by Government ICT Centre (VALTORI) PX-Web application, own installation and maintenance linked to the Statistics Portal: database, ready-made tables, dynamic images automatic maintenance (timing + scripts) API Interface: Conductive Data for 3rd Parties (open data) Three-language (Finnish, Swedish and English) User Management (separate PX Web installation) Pekka Korhonen - Statistical services 15.11.2017

The publishing system of the statistical services -unit of LUKE http://luke.fi STATISTICAL BULLETINS http://statdb.luke.fi http://stat.luke.fi LUKE- PORTAL PX-WEB AUTOMATIC GRAPHICS HOMEPAGE, INFO, QUESTIONNAIRES etc. STATISTICAL PAGES 0 min. valmis-taulut portaaliin STATISTICS DATABASE STATISTICS PORTAL LINKS: DATABASE READYMADE TABLES GRAPHICS Pekka Korhonen - Statistical services 15.11.2017

PX-Web specs in LUKE Statistics 2 virtual servers: Test / Production sites (2 virtual machines) (maintained by VALTORI) Windows Server 2012 R2 & IIS 8.5 remote access (RDC), disk allocation, admin rights, automation TEST: test PX-Web (Luke intra) maintenance of automation (scripts + data storage + backups + logs) PRODUCTION: production PX-Web (public) graphical automation (basic javascript + js- & html-files of graphics) Common network disk space PX-Web folder structure update and timing files (in the right folders) access by statisticians (statistics take care themselves) Pekka Korhonen - Statistical services 15.11.2017

Structure & operation of the PX-Web database INTRA WEB 7 9 8 6 TEST SERVER PRODUCTION SERVER 5 3 2 TEST PX-Web (check) PRODUCTION PX-Web (public) STATISTICS Release calender: update + timing 0 min. readymade tables to portal 5 COMMON DISKSPACE Update files (csv,px,txt) UPDATE AUTOMATION Scripts, Data storage, Logs, Backups GRAPHICS AUTOMATION Basic code, Page codes (js,html) 0 min. API 9:00 if update 15 min. 0 min. graphs to portal 15 min. Pekka Korhonen - Statistical services 15.11.2017

The automatic maintenance of PX-Web Update + timing files on a shared network disk update: px- or csv-file, timing: txt-file (eg 01.01.2018) A 15-minute script retrieves updates from the network disk to the test server and combines them with the original tables and moves the tables to the test-PX-Web and updates the menu -> Updates the Test-PX-Web every 15 mins (checking) Every morning script (mon-fri 9:00) combines updates with original production tables if the timing file is ok (same day or past) and moves tables to the production-PX-Web and updates the menu -> Updates the Production-PX-Web every working day at 9:00 am according to the release calendar (release) Scripts own production + PX-Job & Menubuilder (Statistics Finland) Timing with Task Scheduler (Windows) Pekka Korhonen - Statistical services 15.11.2017

Automatic graphics All the LUKE Statistics Portal graphics come through API-interface basic patterns + maps (over 250 graphs total) http://statdb.luke.fi/charts/api_testi.html dynamic links, downloaded to iFrame, updated from the database table linking with URLs to any website (including customers) languages parameterized (?lang = fi / sv / en) PX-Web API-interface (open data) API-address (url) JSON query (TiPa: json-stat - also xlsx, json, csv, sdmx, px) TiPa's own graphics application (charts) JavaScript: basic code + picture-specific js- & html-files libraries: JSON-stat Javascript Toolkit, Idescat Visual (flot) + own code (development & production in Git) Ready made tables (and pictures) with PX-Web Saved Search (statistical updaters / customers) Pekka Korhonen - Statistical services 15.11.2017

The last page Tools: PX-Edit: designing, creating and maintaining tables PX-Job: automatic updates (part of the script) PX-Win: classification (vs- & agg-files) Demo: Indexed images Nominal / real values Data from many sources, through two APIs LUKE / Statistic Finland Calculation on the fly http://stat.luke.fi/en/commercial-marine-fishery Pekka Korhonen - Statistical services 15.11.2017

15.11.2017