1. http://www.ncl.ucar.edu/Training/Workshops/lectures.shtm http://www.ncl.ucar.edu/ Slides based on lectures from the NCL workshop Also see the NCL homepage http://www.atmos.albany.edu/student/janiga/temp/incoming _grad_lecture.ppt This ppt can be downloaded off the web at the below address

2. ncl-talk@ucar.edu For additional help… http://www.ncl.ucar.edu/Support/ncl_talk.shtml [haley@ucar.edu, shea@ucar.edu]

3. The NCL variable model is based on netCDF

4. netCDF files self describing – (ideally) all info contained within the file – no external information needed to determine file contents portable [machine independent] Similar to HDF (NASA) and GRIB (NCEP, ECMWF, etc) formats; all these are read by NCL using the same one line. Supported by many software tools / languages – NCL, IDL, Matlab, ferret, GrADS, F, C, C++,Java – command line file operators: NCO, CDO  perform various tasks (i.e. concatenation) very efficiently  http://nco.sourceforge.nethttp://nco.sourceforge.net  http://www.mpimet.mpg.de/fileadmin/software/cdo – ncview  quick visualization using gui if in COARDS format:COARDS  IDV, NCVIEW, Panopoly, etc.

5. Examining a netCDF file ncdump file_name – dumps the entire contents of a file; prints every value ncdump -h file_name – Dumps header information [most commonly used] – NCL equivalent: ncl_filedump file_name ncdump -v U file_name – NCL equivalent: ncl_filedump –v U file_name ncl_filedump file_name [ more general ] – netCDF3/4, GRIB-1,GRIB-2, HDF, HDF-EOS [HDF5] ncview file_name [visualize file contents] Now for an example….

6. Parts of netCDF file DIMENSION SIZES dimensions: lat = 64 lon = 128 time = 12 VARIABLES: Names, Types, Attributes, Coordinate Variables variables: float lat(lat) lat:long_name = "latitude" lat:units = "degrees_north" float lon(lon) lon:long_name = "longitude" lon:units = "degrees_east" int time(time) time:long_name = "time" time:units = "Month of Year" double T(time, lat, lon) T:long_name = “Temperature” T:units = “C" T:missing_value = 1.e+20f T:_FillValue = 1.e+20f FILE ATTRIBUTES global attributes: title = “Temp: 1999” source = “NCAR” conventions = "None”

7. Longitude coordinate variable (1D, &) Latitude coordinate variable (1D, &) attributes @: long_name units visual: simple 2D netCDF Variable coordinate variables (rectilinear grid)

8. Non-Traditional Grids or Arrays netCDF and NCL can handle various “other” types of data: – Spectral coefficents (YOTC) – Lambert conformal (WRF) http://www.ncl.ucar.edu/Applications/wrflc.shtml http://www.ncl.ucar.edu/Applications/wrflc.shtml – Radiosondes – Surface station data – Hurricane tracks http://www.ncl.ucar.edu/Applications/unique.shtml – Make your own! Tons of pre-defined functions for manipulating and converting between different formats as well as plotting. http://www.ncl.ucar.edu/Document/Functions/list_alpha.shtm

9. netCDF [NCL] Variable model f = addfile(“foo.nc”,”r”) ; grb/hdf x = f - >X X Scalar or Array attributes long_name _FillValue units add_offset scale_factor etc. values Scalar or Array attributes long_name _FillValue units add_offset scale_factor etc. accessed via @ accessed via & time lev lat lon etc. coordinates time lev lat lon etc. coord var NCL reads the scalar/array, attributes, and coordinate variables as an object X

10. Detailed Look netCDF Variable (NCL) ncl ; interactive mode ncl 0 > f = addfile ("UV300.nc", "r") ; open file ncl 1 > u = f->U ; import STRUCTURE ncl 2 > printVarSummary (u) ; overview of variable Variable: u Type: float Total Size: 65536 bytes 16384 values Number of Dimensions: 3 Dimensions and Sizes: [time | 2] x [lat | 64] x [lon | 128] Coordinates: time: [ 1.. 7 ] lat: [ -87.8638.. 87.8638 ] lon: [ 0.. 357.185] Number of Attributes: 5 _FillValue : 1e36 units : m/s long_name : Zonal Wind Component short_name : U missing_value : 1e36 Classic netCDF Variable Model NCL syntax/funcs query use modify add any aspect of data object

11. NCAR Command Language Complete Programming Language – data types – variables – operators – expressions – conditional statements – loops – functions/procedures/graphics Features – query / manipulate meta data – import data in a variety of formats – array syntax / operations – can use user fortran/C codes and commercial libraries – most functions/procedures ignore missing data

12. Running NCL Interactive Mode (Command line) – ncl [options][command-line-arguments] ncl> enter commands ncl> quit – can save interactive commands ncl> record “file_name” ncl> stop record Batch Mode [.ncl suffix is optional] – ncl [options][arguments] script.ncl – ncl < script.ncl [also acceptable] – nohup ncl [options][arguments] script.ncl > out.txt & – "&" means put in background – “nohup” means don’t quit when I logoff (ie long scripts).

13. NCL predefined options ncl –hnxV [predfined options are preceded by dash] may be used for interactive or batch mode informational – ncl –h [display predefined options and usage and exit] – ncl –V [print the NCL version and exit] action – ncl –x [echo statements as encountered (debug)] – ncl –n [don't enumerate dimensions of values in print() ] multiple predefined options – ncl –nx [ not ncl –n –x ] http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclCL O.shtml Variables can be set on the command line prior to a script. However, this tends to not be necessary. Input from a text file or systemfunc()

14. Sample Batch Script: sample.ncl load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl“ load “$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl“ f1 = addfile("TMP_58-97.nc", "r") ; open netCDF file T = f1->Tmp ; T(time,lev,lat,lon) =>(480,17,73,144) f2 = addfile(“P_1958-1997.grb", "r") ; open GRIB file P = f2->Pres ; P(time,lev,lat,lon) f3 = addfile(“Q_1958-1997.hdfeos", "r") ; open hdf –eos file Q = f3->Specific_Humidity ; Q(time,lev,lat,lon) wks = gsn_open_wks( " ps", “sample") ; open a graphic workstation gsn_define_colormap (wks,"gui_default") ; change from default plot = gsn_csm_contour_map_polar (wks,T(0,5,:,:),False) res = True ; change default plot res@cnFillOn = True ; use colors res@gsnSpreadColors = True ; use entire color map plot = gsn_csm_pres_hgt (wks,T(0,:,{50},:), res ) pot = T* (1.+0.622*Q) *(1000/P)^0.286 ; potential temperature (array) print( " pot: min= " + min(pot) + " max= " + max(pot) ) begin ; optional end ; only if begin is present

16. Outline: NCL Language Basics special syntax characters data types Variables  netCDF/NCL variable model attributes _FillValue named dimensions coordinate variables print and printVarSummary shaping subscripting

17. NCL Syntax Characters ; - comment [can appear anywhere] @ - reference/create attributes ! - reference/create named dimension & - reference/create coordinate variable {…} - coordinate subscripting $ - enclose strings when (im/ex)port variables via addfile (/../) - array construct characters : - array syntax | - separator for named dimensions \ - continue character [statement to span multiple lines] :: - syntax for external shared objects (eg, fortran/C) -> - use to (im/ex)port variables via addfile function “ “- use for a string

18. Data Types Numeric double (64 bit) float (32 bit) long (32 or 64 bit) integer (32 bit) short (16 bit) byte ( 8 bit, 0-255) complex NOT supported non-Numeric string character graphic file logical list Time coordinate variables frequently double. If IR data (K) was offset by 100 it could be stored in byte instead of double (8 times less disk space) or float (4 times less disk space). Data will often be packed into short or byte this way xFloat = short2flt(xShort) xShort@scale_factor xShort@add_offset

19. Attributes assign/access with @ character – T@units = “deg C” – T@long_name = “temperature” – T@whatever = an array of any dimension or type – T@_FillValue = -999. – plot title uses x@long_name by default attribute functions [isatt, getfilevaratts] – if (T@units.EQ.”K”) then delete can eliminate an attribute – delete(T@title) info about a variable or file [meta data] – attributes can be any data type except file – scalar, multi dimensional array (string, numeric)

20. _FillValue Unidata and NCL reserved attribute most NCL functions ignore _FillValue netCDF Operators [NCO] : _FillValue attribute ncview: missing_value attribute – best to create netCDF files with both NCL: don’t use commonly encountered values as a _FillValue! Note: “missing_value” attribute: no special status to NCL – if “T” has “missing_value” attribute but no “_FillValue ”  NCL creates: T@_FillValue = T@missing_value – use built-in function “ismissing” to check for _FillValue  if (any(ismissing(T))) then  T = where (ismissing(T),0,T)

21. Arrays row major ….. like C/C++  left dimension varies slowest; right varies fastest  dimension numbering left to right [0,1,..] indicies [subscripts] are zero based [0,N-1] Consider T(:,:,:) left dimension is 0 [ T!0 ] middle dimension is 1 [ T!1 ] right dimension is 2 [ T!2 ]

22. NCL (netCDF) Dimensions assigned with ! character {let T(:,:,:)} – T ! 0 = "time" ; leftmost [slowest varying] dim – T ! 1 = "lat“ – T ! 2 = "lon" ; rightmost [fastest varying] dim dim names may be renamed, retrieved – T ! 1 = "LAT" … dName = T ! 2 delete can eliminate: delete ( T ! 2 ) Named dimensions used to reshape may be “named” – provides alternative way to reference subscripts – recommendation: always name dimensions – use NCL syntax

23. NCL (netCDF) Coordinate Variables data coordinates [ eg: time, level, lat, lon ] – strict netCDF definition  1D numeric array  monotonically in[de]creasing numeric values  can only be assigned to a named dimension  1D array must be the same size as dimension  should not have _FillValue attribute – coordinate values assigned via & character  T&lat = latitude ; Here latitude is a 1D array used in graphics, coordinate subscripting coordinate functions – iscoord, isfilevarcoord delete can eliminate coordinate array – delete(T&time)

24. Create and Assign Coordinate Variables assign values to named dimension – time&time = time – lon&lon = lon let x be 2D: name dimensions – x ! 0 = “time” … x!1 = “lon” assign coordinate variables to x – x&time = time … x&lon = lon assign dimension name [same as variable name] – time!0 = “time” – lon!0 = “lon” create 1D array – time = (/ 1980, 1981, 1982 /) ; integer – lon = (/ -90,... 90 /) ; float

25. Access/Change/Create/Delete Meta Data @ attributes – u@long_name = "U" – lonName = u@long_name ! named dimensions – u!0 = "TIME" – tName = u!0 & coordinate variable – u&lat = (/ -90., -85,...., 85., 90. /) – latitude = u&lat $ substitute string – x = fin->$variable(n)$(date,{-30:60},:)

26. Variable Assignment Value-only assignment (no meta copy) – U multi-dimensional array with meta data  Uval = (/ U /) or Uval = (/ f->U/)  the (/... /) operator pair strips meta data Variable-to-Variable assignment – consider y = x where x is previously defined  if y not defined:  y has same type/shape/values/meta data as x  if y predefined:  y must have same shape and type  or, x must be coerceible to the type of y  y attributes, dimension names and coordinate variables, if they exist, will be over written

27. Variable Subscripting (1 of 3) Standard Array Subscripting ranges: start/end and [optional] stride indices separated by : omitting start/end index implies default begin/end Consider T(time,lat,lon) T  entire array [ don't use T(:,:,:) ] T(0,:,::5)  1 st time index, all lat, every 5 th lon T(0,::-1,:50)  1 st time index, reverse lat order, 1 st 51 lon T(:1,45,10:20)  1 st 2 time indices, 46 th index of lat, 10-20 indicies of lon

28. Variable Subscripting (2 of 3) Coordinate Variable Subscripting only applies to coordinate variables same rules apply for ranges, strides, defaults use curly brackets {…} standard and coordinate subs can be mixed [if no reorder] T(:,{-30:30},:)  all times/lon, lat -30° to +30° (inclusive) T(0,{-20},{-180:35:3})  1 st time, lat nearest - 20°, every 3rd lon between -180° and 35°

29. Variable Subscripting (3 of 3) Named Dimensions only used for dimension reordering indicated by | dim names must be used for each subscript named/coordinate subscripting can be mixed Consider T(time,lat,lon) t = T(lat|:, lon|:, time|:)  makes t(lat,lon,time) t = T(time|:,{lon|90:120},{lat|-20:20})  all times, 90-120° lon, -20-20° lat

30. Longitude coordinate variable (1D) Latitude coordinate variable (1D) Standard and Coordinate Subscripting Standard: T(9:13,1:8) Coordinate: T({-10:20},{-170:-110}) Combined: T({-10:20}, 1:8)

31. “printing” printVarSummary – gives gross overview of a variable print – same info as printVarSummary – prints values write_matrix – print to standard out or a file – format control of numerical output – can write to file also

32. print (1 of 2) Variable: T Type: float Total Size: 32768 bytes 8192 values Number of Dimensions: 2 Dimensions / Sizes: [lat | 64] x [lon | 128] Coordinates: lat: [-87.86379.. 87.86379] lon: [ 0. 0.. 357.1875] Number of Attributes: 2 long_name: Temperature units: C (0,0) -31.7 (0,1) -31.4 (0,2) -32.3 (0,3) -33.4 (0,4) -31.3 etc. [entire T array will be printed] Prints out all variable information including – meta data, values – T(lat,lon): print (T)

33. print (2 of 2) Variable: T (subsection) Type: float Total Size: 256 bytes 64 values Number of Dimensions: 1 Dimensions / Sizes: [lat | 64] Coordinates: lat: [-87.86379.. 87.86379] Number of Attributes: 3 long_name: Temperature units: C lon: 109.6875 [ added ] (0) -40.7 (1) -33.0 (2)-25.1 (3)-20.0 (4) -15.3 etc. can be used to print a subset of array – meta data, values – T(lat,lon): print( T(:,103) ) or print( T(:,{110}) )

34. Things that can be done (look up or ask) Write netCDF files (compression, making large files) Read and write binary and ascii Functions and procedures in NCL language (over 1000), fortran or C (most are earth science orientated; essentially Matlab or IDV for earth science). Loops, conditional statements. OPeNDAP support (query and read files directly from the web without downloading the whole file) http://www.ncl.ucar.edu/Document/Function s/index.shtml http://www.ncl.ucar.edu/Document/Function s/index.shtml