2. The CERA2 Data Base Data input – Data output Hans Luthardt Model & Data/MPI-M, Hamburg Services and Facilities of DKRZ and Model & Data Hamburg, 29.3.2006

3. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 2 Data Input

4. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 3 Including data into CERA Data sets from users can be included into the data base and made accessible (by internet) Inclusion is done by Model & Data if following conditions are met: data are of general interest data should become available to a broader community data size is small relative to the data archive size a complete set of meta data (fitting the CERA standard) must be provided

5. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 4 Store data data:data must be provided in a structure that can be transferred into the data base without further processing meta data: XML-files for the experiment and each of the data sets must be provided XML-templates are available from : http://cera-www.dkrz.de/Meta_Fill/http://cera-www.dkrz.de/Meta_Fill/ or http://cera-www.dkrz.de/Meta_Fill/mod_metadata.html

6. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 5 Preferred storage structure for 3-d gridded data: single variable single level time series grib formatted / netCDF formatted (/ others) Data structures Application related data structure (2-D) (180 KByte/blob) original data structure (4-D) (3 MByte/blob)

7. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 6 Retrieve data sets

8. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 7 Retrieve data sets Database content: 220 TByte data volume 650experiments 80 000 data sets Database content: 220 TByte data volume 650experiments 80 000 data sets

9. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 8 Data and Meta data CERA Metadata Model

10. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 9 Search strategies Keyword Project Experiment Dataset Information Download Variable specific Dataset (by name) Postprocessing Code

11. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 10 Access to the data base Access to the CERA data base: by java servlet - using an internet browserjava servlet by java applet (java runtime required) - obselet!java applet by java command line tool (jblob )jblob Unrestricted browsing for meta data User account required for data download Start: http://www.mad.zmaw.de/http://www.mad.zmaw.de/

12. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 11 Access to the data base Retrieval by script : Java based – for Unix / Linux and Windows http and JDBC connection examples : jblob –datasetname -options jblob –datasetid -options jblob –showdatasets ” ” (% as wildcard)

13. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 12 Access to the data base Access to the CERA data base: by java servlet - using an internet browserjava servlet by java applet (java runtime required) - obselet!java applet by java command line tool (jblob )jblob Unrestricted browsing for meta data User account required for data download Start: http://www.mad.zmaw.de/http://www.mad.zmaw.de/

14. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 13 Access to the data base Access by java applet: (obsolet!) e.g. hierarchical topic search

15. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 14 Browse/Retrieve Metadata see wini.wdc-climate.de Metadata access via WWW: xsql query to DB xml output from DB xsl mapping to any metadata format raw xml xhtml ISO xml DC xml... various metadata formats http: XML xsl – mappi ng xsql –query lnternet Applicati on Server request: URL user applications

16. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 15 dynamic html pages http: html Servlet / JSP lnternet Applicatio n Server web browser Catalogue access via WWW URL parsed by JSP integrated DB retrieval by JSP response in standard html efficient administration of detailed meta information request: URL Access to the data base

17. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 16 Search for data The servlet interface currently allows search for: Project – experiment – data set (keyword) Data set names (or part of it) Data set associated with codes

18. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 17 Hierarchical search for data sets Project Experiment data set keywords

19. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 18 Access to the data base Access to the CERA data base: by java servlet - using an internet browser by java applet (java runtime required) - obselet! by java command line tool (jblob) Unrestricted browsing for meta data User account required for data download Start: http://www.mad.zmaw.de/http://www.mad.zmaw.de/

20. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 19 Search for data (hierarchical)

21. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 20 Search for data (hierarchical)

22. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 21 Search for data (hierarchical)

23. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 22 Search for data (hierarchical) Experiment decsription

24. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 23 Search for data (hierarchical) List of data sets

25. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 24 Search for data (hierarchical) Data set description

26. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 25 Search for data (hierarchical) Data set description: data organisation

27. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 26 Download data sets x Selecting data set for download (login required)

28. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 27 Search data sets (by Code)

29. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 28 Search data sets (by Code) Specifying code table

30. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 29 Search data sets (by Code) Specifying code number

31. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 30 Search data sets (by Code)

32. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 31 Search data sets (by dataset name)

33. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 32 Search data sets (by dataset name) Specifying name elemets

34. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 33 Search data sets (by dataset name)

35. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 34 Download data sets

36. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 35 Download data sets Process list: initiating data download

37. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 36 Download data sets Specifying time range for download

38. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 37 Download data sets Specifying processing options

39. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 38 Download data sets Starting download

40. H. Luthardt (M&D/MPI-M, Hamburg) 29.3.06 / 39 Contact URLs: M&D : http://www.mad.zmaw.de/ Data Base Portal : http://cera-www.dkrz.de/CERA/ WDCC : http://www.wdc-climate.de email: data@dkrz.de