Working group “Maritime Transport Statistics” Luxembourg, April 2008 Data related issues Item 8 of the agenda

Introduction Data transmission format Compliance monitoring Timeliness in data transmission Data quality –Distinction between “zero” and “empty” –Current quality checks –Forthcoming activities

Data transmission format (1/3)

Information on General characteristics

Data transmission format (1/3) Information on General characteristics Information on data or metadata

Data transmission format (1/3) Information on General characteristics Information on data or metadata Data

Data transmission format (1/3) Information on General characteristics Information on data or metadata Data Mandatory fields

Data transmission format (1/3) Information on General characteristics Information on data or metadata Data Mandatory fields Optional fields

Data transmission format (1/3) Information on General characteristics Information on data or metadata Data Mandatory fields Optional fields Fields not relevant for the dataset

Data transmission format (2/3) The fields not relevant for the dataset should be provided: - as "empty space" : best solution –or not provided at all: acceptable temporary solution Example: –Best solution: A3;2007;0;CCPPP;1;;;;;;;;;3270;1500;;;;;;300;150 –Acceptable, for the time being: A3;2007;0;CCPPP;1;3270;1500;300;150

Data transmission format (3/3) Countries are warmly invited to use the standard structure ("best solution") for each dataset, when transmitting data to Eurostat.

Compliance monitoring (1/3) Decision 1578/2007 of the European Parliament and the Council of 11 December 2007 on the Community Statistical Programme 2008 to 2012 (OJ L 344 of ): –Reference to compliance monitoring –a vigorous and systematic monitoring of the application of the Community statistical legislation Two major dimensions a.Timeliness in data transmission b.Data quality

Compliance monitoring (2/3) Letter from the Director General of Eurostat to National Statistical Offices regarding the application of the Community legislation Thanks to a good cooperation, no mention of Directive 95/64 Countries are kindly invited to continue improving the timeliness in data transmission and the data quality, where necessary.

Compliance monitoring (3/3) The implementation of the Community legislation is a national competence However, close cooperation and understanding between Eurostat, NSA, data providers and users is fundamental Several initiatives have already contributed to a continuous improvement in the compliance with Community legislation In 2008, possible visits of Eurostat to a selected number of countries (volunteers?)

Timeliness in data transmission: Data reception status 09/04/2008 Countries are asked to check the table in the annex and, where necessary, provide comments during or in the margin of the meeting.

Data quality: Distinction between “zero” and “empty” (1/5) Important for the correct analysis and interpretation of data “Zero”: to be used when the information about the “no activity” is available (“positive information”) “Empty”: –The information was not available –The provision of a certain variable is not applicable in the frame of the Directive

Data quality: Distinction between “zero” and “empty” (2/5) Eurostat kindly invites countries to apply the above guidelines when transmitting those datasets including more than one statistical variable (datasets A3, C1, F1 and F2). For the other datasets the inclusion of records with the statistical variable equal to zero is recommended only in specific situations (temporary closure of port/ terminal, temporary stop of a usually standard activity, and so on).

Data quality: Distinction between “zero” and “empty” (3/5) A port PPP in the country CC is registering non-cruise passengers and cruise passengers movements but no cargo handling activity during the reference year: The gross weight of goods should be “0” A3;2007;0;CCPPP;1;;;;;;;;;0;1500;;;;;;300;150 (best solution) A3;2007;0;CCPPP;1;0;1500;300;150 (acceptable temporary solution)

Data quality: Distinction between “zero” and “empty” (4/5) The port has cargo handling activities but no facilities to welcome passenger ships.The number of the three passenger variables should be “0”. A3;2007;0;CCPPP;1;;;;;;;;;3270;0;;;;;;0;0 (best solution) A3;2007;0;CCPPP;1;3270;0;0;0 (acceptable temporary solution)

Data quality: Distinction between “zero” and “empty” (5/5) The port has cargo handling activities and registers non- cruise passengers activities but is not able to collect information on cruise passenger movements The "number of cruise passengers starting and ending a cruise" and the "number of cruise passengers on cruise passenger excursion (direction: inwards only)" should be left empty. A3;2007;0;CCPPP;1;;;;;;;;;3270;1500;;;;;;; (best solution) A3;2007;0;CCPPP;1;3270;1500;; (acceptable temporary solution)

Current quality checks: Overview Several types of quality checks Detection of potential errors or inconsistencies When reporting the results –Possible solution suggested to the countries to be approved –Ask for clarification or explanation of the problems/errors Enhancement of some checks Some new checks have been implemented

New Checks: Unknown share checks (1/2) Highlight important shares of data reported under the codes "Unknown" or "Other“ Quarterly datasets: A1, A2, C1, D1, F1 and F2 Detect the use of codes: –unknown Reporting Port (example BE888) –unknown Partner MCA (ZZ00, ZZ01, ZZ02) –unknown Type of Cargo (X) –unknown Size of Vessel (XX) –unknown Type of Vessel (XX)

New Checks: Unknown share checks (2/2) Additional checks on datasets A1 and D1 Detect the use of codes –unknown national partner port (example BE888 for BE) –unknown foreign partner port (example DE888 for BE) –unknown Partner MCA of countries with more than one MCA (example EG09) –non sea partner countries (example Afghanistan, Luxembourg) Results sent when important shares are detected In 2008, to be extended to annual datasets: A3, B1 and E1

New Checks: Self declaring ports Check if in a record the declaring port is the same a as the partner port Applied to all relevant variables and datasets Sent to the countries for explanations or corrections

Quality checks: Forthcoming activity Migration from “GENEDI” to “eDamis Validation Engine” (eVE) Mirror checks

Migration from “GENEDI” to eVE Some of the current quality checks included in GENEDI Some countries faced difficulties in using this functionality, especially complex tests Most of GENEDI validation rules converted to eVE Some complex validation rules not kept in eVE to facilitate the data transmission

Mirror checks (1/4) Last full exercise during 2005 –2003 data –EU-15 Member States and Norway Complex and require resources Should be used to detect only major and systematic problems During the WG in October 2006: –Full Exercise not to be repeated for 2004 data –Fully exploit the results of previous exercises

Mirror checks (2/4) Some initiatives to tackle the major identified “mirror problems” –Harmonisation of the list of ports –Clarification on the implementation of the classification by type of cargo –Check the use of “unknown” codes Implementation of “simplified” mirror checks –Detect few major structural inconsistencies –2004 data –EU-15 and Norway

Mirror checks (3/4) 2005 data –“Simplified” mirror checks for EU-15 and Norway –Full mirror checks for EU-12 Member States and Croatia Results to be sent in the coming weeks Activity based on the agreement by the WG to show their individual port data to other countries for this restricted use only (data checking)

Mirror checks (4/4) Eurostat require the delegates of participating countries in the Working Group (and in particular the countries for which this exercise is new) to confirm: –their agreement for such a distribution of their own data to the other countries by Eurostat –their engagement to use the received information only for the specific statistical purpose mentioned above (data checking), strictly avoiding in particular any re-dissemination of the received data to any third party.