1. OBSERVATIONS & PRÉVISIONS CÔTIÈRES www.seadatanet.org Training Workshop – Ostende – 26-29 October 2009 REFORMATTING TOOLS Med2MedSDN and NEMO M. Fichaut

2. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 2 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 CDI CSR EDMERP EDMED SEADATANET PORTALSEADATANET PORTAL European portal Collection of ASCII files Format X Data in Database Metadata in Database Data Input Metadata Input Metadata In Excel files Partner system : pilot data centre SeaDatanet Vocabulary NEMO ODV Med2MedSDN Coupling table Download Manager Collection of ASCII files Format SDN XML Metadata Files Local copy of data to download Data request Data download MIKADO XML Validator

3. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 3 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Overview Med2MedSDN Objectives and main features NEMO Objectives, main features and principles Technical characteristics Description of the different steps to follow to be able to reformat ASCII files to SeaDataNet format Link between NEMO and MIKADO Link between NEMO and SDN Download Manager – coupling table

4. OBSERVATIONS & PRÉVISIONS CÔTIÈRES www.seadatanet.org Training Workshop – Ostende – 26-29 October 2009 Med2MedSDN For MEDATLAS files

5. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 5 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Objectives of Med2MedSDN Translate MEDATLAS files to SeaDataNet MEDATLAS files - available under Windows Version 1.1.02 and user manual available at : http://www.seadatanet.org/standards_software/software/Med2MedSDN

6. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 6 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Med2MedSDN main features Reformats MEDATLAS files to MEDATLAS SeaDataNet format Java tool, bilingual (French, English) Adds the additional SeaDataNet information : mapping for parameters and LOCAL_CDI_ID and EDMO_CODE Can reformat one file or a large number of files (in one directory) Generates a coupling file for the SeaDataNet download manager Linked to SeaDataNet vocabularies through Web services for parameters mapping and for list of EDMO codes Need of internet connexion while updating lists

7. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 7 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Med2MedSDN Med2MedSDN vocabulary update

8. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 8 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Med2MedSDN main screen

9. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 9 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Med2MedSDN log files Errors (managed by Log4j) are registered in a log file in Med2MedSDN installation directory. One line in the log file is composed as following: Date, Name of the Software, Error severity level, Error message INFOInformative messages for starting of the conversion or successful conversion ERROR For conversion errors : conversion is cancelled on the current file but continues on the other files FATALFor conversion errors which stop the processing of the files

10. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 10 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Med2MedSDN - User manual User manual is provided : File : sdn_Med2MedSDN_UserManual_V1.0.pdf Detailed explanation for Med2MedSDN installation and use, lots of snapshots

11. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 11 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 ANY QUESTION?

12. OBSERVATIONS & PRÉVISIONS CÔTIÈRES www.seadatanet.org Training Workshop – Ostende – 26-29 October 2009 NEMO For ASCII files on any type

13. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 13 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Objectives NEMO is a reformatting software used to generate ASCII files at MEDATLAS or ODV formats which are defined as SeaDataNet formats for data exchange between SeaDataNet partners. Can be downloaded from SeaDataNet Web site : http://www.seadatanet.org/standards_software/software/nemo It was created in IFREMER for internal purposes, and adapted to take into account the SeaDataNet formats and vocabularies.

14. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 14 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Technical characteristics Written in Java Language (Version >= 1.6) Bilingual (French, English) Available under multiple environments : Microsoft : Windows 2000, XP, VISTA, APPLE Unix - Solaris Linux Use of SeaDataNet common vocabularies web services to update lists of values of the SeaDataNet common vocabularies need network connections in order to have up to date lists of values. But NEMO works offline once the lists are up-to-date

15. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 15 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 NEMO main features Reformat any ASCII file of vertical profiles, time-series or trajectories to a SeaDataNet ASCII format (ODV, MEDATLAS). The input ASCII files can be : one file per station for vertical profiles or time series one file for one cruise for vertical profiles, time series or trajectories Interact with Mikado, to be able to generate ISO-19115 XML descriptions of the data.

16. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 16 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 NEMO principles NEMO must be able to read almost any ASCII format to translate it to MEDATLAS, ODV Users of NEMO describe the entry files format so that NEMO is able to find the information which is necessary in the SeaDataNet formats One mandatory pre-requirement is that in the set of entry files the information about the stations must be located at the same position : same line in the file, same position on the line or same column if CSV format be in the same format For example : for all the stations the latitude is on line 3 on the station header, from character 21 to character 27, or 3 rd column in CSV the format is +DD.ddd

17. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 17 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Running NEMO 4 or 5 steps to convert the entry files to a SeaDataNet format 1.Describe the type of file(s) 2.[Describe the cruise, if the files are related to 1 cruise] NEMO is able to read XML CSR generated with MIKADO 3.Describe the station information 4.Describe the measured parameters 5.Convert the file One more step to Save the description of the format (Model)

18. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 18 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 NEMO - Description of entry files (1) Where are the files? Is it a cruise? Is it one file per cruise or one file per station ? Is it a collection of station files ? grouped by cruises or not? Are the files with separators? Tabulations? Semicolon? Comma ? Are they vertical profiles, time series or trajectories data?

19. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 19 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 NEMO - Description of entry files (2) File to convert is displayed in NEMO window The principle is to describe the input file by selecting the information in the file window

20. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 20 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station header : constant number of lines

21. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 21 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station header : number of lines not constant

22. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 22 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Cruise description Can be Manually input Imported from a SeaDataNet XML CSR Imported from a database (IFREMER only)

23. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 23 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description

24. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 24 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description Select the date in the file, in the left part of the screen

25. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 25 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description Input the date format

26. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 26 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description Click on the Set button to tell NEMO the start and end position of the date

27. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 27 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description Click on the Test button to verify your input

28. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 28 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Station description

29. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 29 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Give the position, the default value, the format of each parameter QC flags can also be described Pressure is mandatory for vertical profiles (MEDATLAS requirement) and cannot be deleted.

30. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 30 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Select « Parameter list » in the context menu of the measured parameters table (right click to open the menu)

31. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 31 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description

32. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 32 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Select the value of the measured parameter in the file, in the upper left part of the screen

33. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 33 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Select « Set start /end » in the context menu of the measured parameters table

34. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 34 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Start and End position of temperature are added in the table

35. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 35 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description Select « Update test » in the context menu of the measured parameters table

36. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 36 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Data description The values read by NEMO are displayed in the Test column of the table. User is able to check that his input is correct

37. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 37 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 File conversion Output formats MEDATLAS Multi-station ODV Mono-station Multi-station

38. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 38 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 While converting … NEMO is able to Generate a SeaDataNet CDI Summary Text file containing the minimum mandatory information needed in the CDI ISO-19115 description This file can be converted to an Excel file Which can be read by MIKADO to generate the XML CDI files Generate a coupling table that will be used by the Download manager of SeaDataNet It is the link between the LOCAL_CDI_ID and the file

39. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 39 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 CDI summary Further information must be added in NEMO settings (menu Options of NEMO)

40. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 40 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 CDI SUMMARY File Text file Open it with Excel Rename the sheet : STATION Save the file as an EXCEL file

41. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 41 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Interaction with MIKADO Collection of ASCII files ODV files CDI summary CSV file SeaDataNet CDI MIKADO XML CDI files summary_CDI_NEMO.xml

42. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 42 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Coupling table for Download Manager Further information must be added in NEMO settings (menu Options of NEMO) The coupling table makes the link between each LOCAL_CDI_ID and the file(s) which contains it

43. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 43 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Coupling table If the Generate mapping option is clicked : NEMO will insert a record in the coupling table for each vertical profile, time series or trajectory that it converts. The coupling table is managed by a local database imbedded in NEMO The coupling table can be : Edited (for modification, insertion or deletion) exported (to be used by the download manager) imported (from previous version of NEMO, for example)

44. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 44 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Coupling table content (1) LOCAL_CDI_ID Modus 1 : mono-station 3 multi-station Filename Date of creation

45. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 45 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Coupling table content (2) One unique LOCAL_CDI_ID in one mono-station ODV file The same LOCAL_CDI_ID in 2 different files : One multi-station MEDATLAS file One mono-station ODV file

46. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 46 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 Downloading the data CDI CSR EDMERP EDMED SEADATANET PORTALSEADATANET PORTAL European portal Collection of ASCII files Format SDN Local copy of data to download Data request Data download Coupling table SeaDataNet Data Centre List of LOCAL_CDI_IDs Download Manager

47. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 47 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 NEMO – User manual User manual is provided : File : sdn_Nemo_UserManual_V1.2.pdf Detailed explanation for NEMO installation and use, lots of snapshots

48. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 48 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 And dont forget that All software freely available at : http://www.seadatanet.org/standards_software/software FAQ on the web site for MIKADO and NEMO Dont forget to read the user manuals which are very detailed Any problems using the software Mail to sdn-userdesk@seadatanet.org

49. OBSERVATIONS & PRÉVISIONS CÔTIÈRES 49 www.seadatanet.org 3 rd SeaDataNet training course – Ostende – 27 October 2009 ANY QUESTION?