1. Climate Data Formats Deniz Bozkurt
web.itu.edu.tr/bozkurtd
ITU Eurasia Institute of Earth Sciences
The International Training Course on
“Climate Analysis and Applications”
Alanya
10-19 October 2011

2. Climate Datasets
Datasets

3. Digital representation “11010001”
Climate Datasets
Raw data
“abcd12345”
Input system
Digital representation “ ”

4. Climate Datasets Modeled data: gridded multi-dimendisonal data
Consisting of variables, dimensions and attributes
Observations (station, reanalysis)

5. Climate Datasets
Climate data can be stored as different formats in different platforms:
Character format
Packed binary format

6. Climate Data Formats - Character Format
ASCII (American Standard Code for Information Interchange)
It refers to a “text” file that is readable by the naked eye (it only contains the letters a-z, numbers, carriage returns, and punctuation marks.
Each character is stored as digital =
Binary H e l o , w r d

7. Climate data formats – ASCII

8. Climate data formats – ASCII
Factors restrict the use of ASCII:
Weak data storage and huge data size:
“9” 1 byte, “679.43” 6 byte, E byte
No or not practical metadata

9. Climate data formats – Packed Binary
A common data format for saving and storing of large size of the data
“3.1” 24 bits (ASCII) can be packed as 5 bit
Can be processed very quickly and effectively in data analysis

10. Climate data formats – Packed Binary

11. Some standard data formats
NetCDF (network Common Data Form) Format
Self-describing data format
All the information about data is stored inside of the data
No need for an additional information for the identification of the contents of the file
Machine independent
Can be readable by various applications
NCL, IDL, GrADS, Matlab, Ferret, C, C++, Java, Fortran
Komut satırı operatörler: NCO, CDO
Most commonly used format in the oceanographic and atmospheric science for observational data and numerical modeling (IPCC datasets, NASA, NOAA, NCAR, UCAR…)

12. Climate data formats - NetCDF
metadata
dimensions
variables
Variable
properties
attributes

13. Climate data formats - NetCDF
Data storage in NetCDF format
3D (x,y,t) or 4D (x,y,z,t)

14. Climate data formats - NetCDF
Data storage in NetCDF format
netcdf mynetcdf {
dimesions:
x=4;
y=4;
time=UNLIMITED;
variables:
float x(x);
float y(y);
int time(time);
float temperature(time,x,y);
data:
x = 10, 20, 30, 40;
y = 110, 120, 130, 140;
time = 31, 59, 90; }

15. Climate data formats - NetCDF
Data storage in NetCDF format
netcdf mynetcdf {
dimesions:
x=4;
y=4;
time=UNLIMITED;
variables:
float x(x);
float y(y);
int time(time);
float temperature(time,x,y);
data:
x = 10, 20, 30, 40;
y = 110, 120, 130, 140;
time = 31, 59, 90;
Temperature = 111, 211, 311, 411; }
time = 1
x = 1 to 4
y = 1

16. Climate data formats - NetCDF
Data storage in NetCDF format
netcdf mynetcdf {
dimesions:
x=4;
y=4;
time=UNLIMITED;
variables:
float x(x);
float y(y);
int time(time);
float temperature(time,x,y);
data:
x = 10, 20, 30, 40;
y = 110, 120, 130, 140;
time = 31, 59, 90;
Temperature = 111, 211, 311, 411, 121, 221, 321, 421, 131, 231, 331, 431, 141, 241, 341, 441; }
time = 1
x = 1 to 4
y = 1 to 4

17. Climate data formats - NetCDF
Data storage in NetCDF format
netcdf mynetcdf {
dimesions:
x=4;
y=4;
time=UNLIMITED;
variables:
float x(x);
float y(y);
int time(time);
float temperature(time,x,y);
data:
x = 10, 20, 30, 40;
y = 110, 120, 130, 140;
time = 31, 59, 90;
Temperature = 111, 211, 311, 411, 121, 221, 321, 421, 131, 231, 331, 431, 141, 241, 341, 441, 112, 212, 312, 412, 122, 222, 322, 422, 132, 232, 332, 432, 142, 242, 342, 442, 113, 213, 313, 413, 123, 223, 323, 423, 133, 233, 433, 143, 243, 343, 443; }
time = 1 to 3
x = 1 to 4
y = 1 to 4

18. Climate data formats - NetCDF
4D (x,y,z,t)

19. Climate data formats - NetCDF
some NetCDF commands
ncdump file_name | less, to see the entire contents of the file
ncdump –h file_name | less, to see the front portion of the data
ncdump –v var_name file_name | less, to see the specific data within the file
ncl_filedump to see the contents of different data formats

20. Climate data formats - GRIB
GRIB (GRIdded Binary) Format
World Meteorological (WMO) standard, commonly used in meteorology
Bit-oriented data format: very efficient for transmission/archival
- netCDF (float) 2-4 times larger than corresponding GRIB
two dimensional horizontal grids only
- each grid has scale/offset for optimal precision
requires external parameter table (s)

21. Climate data formats - GRIB
Basically consists of:
Product definition section(PDS)
Creater of the data, unit of the data, time stamp, vertical system of the data
Grid description section (GDS), optional
Grid description, map projections
Bit map section (BMS), optional
Defines which grid points will be presented
Binary data section (BDS)
Contains the packed binary data
End section

22. Climate data formats - GRIB
Files are typically 1/2 to 1/3 of the size of normal binary files:
Y: gridded data
D: scale factor
R: reference value
E: binary scale factor

23. Climate data formats – ArcGIS Shapefile
NetCDF data:
Raster
Table
Feature
Directly readable

24. Climate data formats – ArcGIS Shapefile

25. Climate data formats – ArcGIS Shapefile
Raster
Feature
Table

26. Climate data formats – ArcGIS Shapefile

27. Climate data formats – ArcGIS Shapefile