NSO8055 Okeanograafiline prognoos Jüri Elken Andmete haldamise küsimusi (väga lai temaatika, ühtne kontseptsioon puudub, siiski: ) ICES ajalooline andmehõive perfokaardi formaat (andmed) ROSCOP vorm (meta-andmed) geograafiline kodeering (Mardseni kvadraadid) Andmete haldamise tendentsid MyOcean, Sea-SEARCH & SeaDataNet Meta-andmed: EDIOS

Card from a Fortran program: Z(1) = Y + W(1) 1881, Herman Hollerith IBM 80 column punch card format kasutusel alates 1928 kuni ca 1980 andmestruktuur kanti üle magnetlintidele numbrid 1 auk, tähed mitu auku kasutati ka “overpunch” FORTRAN reeglid põhinevad perfokaardile Perfokaart

ICES ajalooline andmehõive ICES Punch Card format (Data) 1968 ROSCOP Cruise Report format (MetaData) ICES perfokaardid Hydro Master Card iga jaama kohta 1, sisaldab meteoandmeid Hydrochemistry Card iga sügavuse kohta 1 kaart, sh riigi, laeva, koordinaatide, aja andmed Hydrography Card iga sügavuse kohta 1 kaart, kohandatud CTD-le (rohkem tüvenumbreid) andmete lahutus asukoht minut sügavus m T, S sajandik biogeenide suured kontsentratsioonid “overpunch” abil NB! puudub info metoodika ja kvaliteedikontrolli kohta

Submitters of data should, if at all possible, use either of the formats described here. Receivers of data from ICES will receive it in only the first described format, the ICES Oceanographic 'punch card' format but software is normally supplied with any request to help the user read data sets prepared in this format, including an export facility to data bases and spreadsheets. The user should note that this format has been modified from that published by ICES in 1979 in several important respects. In particular provision was made to include position information to.01 of a degree, and time to the nearest minute. Other changes include a re-definition of the > (greater than) overpunch in the nutrient fields (type '56' chemistry record was replaced by type '76') and record type 'P6' was introduced to accommodate the very high nutrient levels reported from some coastal regions. In both of these record types chlorophyll 'a' is stored to only one decimal place (the '56' record type was 2 decimal places). From early 1994, additional features were added to the format (03 record) to accommodate extra decimal places common in CTD records. This affected only columns previously used for derived quantities (sigma-t, dynamic depth). For data received after ca 1997 parameters not supported by the above record types were included by the inclusion of the '0Z' record (Additional Parameter Record). This record type allows for any number of parameters, so long as it is specifified in the BODC/JGOFS data dictionary. ICES andmehõive kaasaegne juhend (ka HELCOM seire kasutab seda)

ICES perfokaardi näited: Hydro Master ja Hydrochemistry

ROSCOP (Cruise Summary Report) ROSCOP (Report of Observations/Samples collected by Oceanographic Programmes) kinnitati IOC poolt 1960-ndate lõpus Ekspeditsioonide lühiinfo mida millega mõõdeti kus (Marsden kvadraadid) kelle käest küsida mõõtmisandmeid kaasajal analoog: meta-andmed Kirjelduste grupid: Meteorology (6) Physical Oceanography (18) Chemical Oceanography (16) Marine Contaminants/Pollution (8) Marine Biology/Fisheries (27) Marine Geology/Geophysics (16) Other (>30)

Näide

Data. We make distinction between information and data. Information is describing a status or situation (e.g. today is cold); data quantify the status or situation (e.g. temperature is 10°C). We define data broadly to include experimental or in situ observations, model outputs and images. Visualization. The first significant examples of Marine Information Systems were developed during the '60s. Until the '80s the data management was composed of many independent processes: data collection; pre-processing; storage in files. Graphical representation and dissemination were part of the scientific study of the ocean dynamics. Today an efficient information strategy includes visual representation of data (graphs, maps,...), as a tool for dissemination of data among users and the public. Quality control. There are areas were the use of term 'data' is often controversial: a) processed versus raw measurements, b) model outputs versus observations, c) images versus digital underpinnings. Scientist strive to fully characterise their data to enable a better understanding of its limitations. Use of data can be limited by the lack of certain attributes such as: procedures for collection, conditions during collection, instrumentation, temporal and spatial referencing, error or uncertainty, indications on quality assurance procedures. Analysis. Working with data provides opportunities for quantitative analysis and reasoning, broad discussion and debate to evolve scientific understanding. Models. We define as model an idealisation that embodies certain aspects of the 'real ocean'. Models provide an experimental apparatus for the scientific rationalisation of the ocean phenomena. In the presentation of ocean model fundamentals, it is useful to start with a discussion on fluid kinematics. Data Tools and Models (SeaDataNet)

Kui andureid ja platvorme oli vähe, koguti andmed suurtesse keskustesse Kaasajal hoitakse andmed enamasti mõõtja juures, formaadid jms on kirjeldatud, ligipääs üle veebi, rahvuslikud andmekeskused on siiski tugevad: Meta-andmed kirjeldavad andmeid (parameetri definitsioon, metoodika, platvorm, kvaliteedikontroll jne) Andmete otsimine läbi meta-andmete kataloogi Andmeülekande protokollid (ftp, OpeNDAP jne) Operatiivne okeanograafia: mõõdetavaid parameetreid vähe, lihtsam suuri “süsteemide süsteeme” kokku panna Interdistsiplinaarne mereteadus: parameetreid tohutult, juba defineerimine keerukas Andmete haldamise tendentsid

Operatiivne süsteem

Andmete “otsimise” projekt (juba lõppenud)

SEADATANET has developed an efficient distributed Marine Data Management Infrastructure for the management of large and diverse sets of data deriving from in situ and remote observation of the seas and oceans. The on-line access to in-situ and remote sensing data, meta-data and products is provided through a unique portal interconnecting the interoperable node platforms constituted by the SeaDataNet data centres. The development and adoption of common communication standards and adapted technology ensure the platforms interoperability. The quality, compatibility and coherence of the data issuing from so many sources, is assured by the adoption of standardized methodologies for data checking, by dedicating part of the activities to training and preparation of synthesised regional and global statistical products from the most comprehensive in-situ and remote sensing data sets made available by the SeaDataNet partners.

The marine observing system is highly fragmented: more than 600 scientific data collecting laboratories from governmental organizations and private industry have been identified. They collect data by using various sensors on board of research vessels, submarines, fixed and drifting platforms, airplanes and satellites, to measure physical, geophysical, geological, biological and chemical parameters, biological species etc. The collected data are neither easily accessible, nor standardized. They are not always validated and their security and availability have to be insure in the future.

SeaDataNet üldine andmete haldamise kontseptsioon CDI Otsing Andmeid on veel vähe! väljavõte 2010

SeaDataNet produktid Soolsuse klimatoloogia JaanuarAprill JuuliOktoober Interpolatsioon: 4D Data-Interpolating Variational Analysis Soft: DIVA GHER (University of Liege)

vabalt kasutatav Windows (7, Vista, XP, 9x, Me, NT, 2000), Mac OS X, Linux, and UNIX (Solaris, Irix, AIX) oma andmete formaat, kuid loeb ka NetCDF rannajoon, sügavused

The BODC Parameter Dictionary In the 1980s, when we first started managing oceanographic data our dictionary contained less than twenty parameters. The BODC Parameter Dictionary is a collection of controlled vocabularies for parameter management. The BODC Parameter Usage Vocabulary (8 MB) contains almost 19,000 terms that are designed to label data values. These have been systematically constructed using a semantic model. Navigation through such a large number of parameters is a daunting task. To help with this, a 3-layer hierarchy of discovery keywords is provided. The top level is the SeaDataNet Parameter Disciplines, followed by the SeaDataNet Agreed Parameter Groups and the BODC Parameter Discovery Vocabulary. XML formaadis, kasutatav vastava tarkvaraga BODC = British Oceanographic Data Centre Milliseid andmeid tuleb käsitleda?

Termdefinition Administration and dimensions Parameters related to spatial and temporal co- ordinates, entity referencing (eg record numbering and keys) and access control AtmosphereThe atmospheric sciences domain Biological oceanographyThe biological oceanographic science domain Chemical oceanographyThe chemical oceanographic science domain Cross-disciplineNo specific association with an identified domain Cryosphere The cryosphere science domain, including ice on both land and sea Marine geologyThe marine geological science domain Physical oceanographyThe physical oceanographic science domain TerrestrialThe terrestrial science domain SeaDataNet Parameter Disciplines asendatud veebisõnastikuga

SeaDataNet Agreed Parameter Groups term AcousticsOptical properties Administration and dimensionsOther biological measurements Amino acidsOther inorganic chemical measurements Atmospheric chemistryOther organic chemical measurements Bacteria and virusesOther physical oceanographic measurements Biota compositionPCBs and organic micropollutants Birds, mammals and reptilesPhytoplankton Carbon, nitrogen and phosphorusPigments Carbonate system Rate measurements (including production, excretion and grazing) CryosphereRock and sediment age and dating CurrentsRock and sediment biota Dissolved gasesRock and sediment chemistry Fatty acidsRock and sediment lithology and mineralogy FishRock and sediment physical properties FluxesSea level Gravity, magnetics and bathymetrySediment pore water chemistry Halocarbons (including freons)Sedimentation and erosion processes HydrocarbonsSonar and seismics IsotopesSuspended particulate matter Metal concentrationsTerrestrial MeteorologyWater column temperature and salinity MicrozooplanktonWaves NutrientsZooplankton asendatud veebisõnastikuga

EDIOS Meta-andmete formaat (MIF) (1) asendatud veebivormiga

EDIOS Meta-andmete formaat (MIF) (2) asendatud veebivormiga

Marine Strategy Framework Directive foresees adoption of methodological standards for the assessment of the status of the marine environment, monitoring, environmental targets and the adoption of technical formats for the purposes of transmission and processing of data in line with INSPIRE Directive. In respect of each marine region or subregion, Member States shall make an initial assessment of their marine waters, taking account of existing data where available. Member States sharing a marine region or subregion shall draw up monitoring programmes and shall, in the interest of coherence and coordination, endeavour to ensure that: (a) monitoring methods are consistent across the marine region or subregion so as to facilitate comparability of monitoring results; (b) relevant transboundary impacts and transboundary features are taken into account. Uued arengud (1)

Lot 1 – Hydrographic data Lot 2 – Marine geological data Lot 3 – Chemical data Lot 4 – Biological data EMODNET European Marine Observation and Data NETwork EMODNET will improve availability of high quality data. EMODNET will provide data on scales defined by the regions and subregions of the Marine Strategy Framework Directive. The parameters to be collated are chosen to fit in with the requirements of the Directive. Four service contracts were launched for creating pilot components: Uued arengud (2)

EMODNET vision: components

Läänemere andmeid veebis Operatiivsed andmed (ainult näha)BOOS Ajaloolised andmebaasid ICES vabahttp:// BEDvabahttp://nest.su.se/models/bed.htm BALTEXparooligahttp:// FMI ja SYKE (endine FIMR) SMHI (SHARK)