1. © Crown copyright Met Office An Introduction to Linux, PP utilities, CDO and NCL Bangkok Workshop, 26 th February– 1 st March 2013

2. © Crown copyright Met Office What is Linux? Useful commands Bash PP utilties CDO NCL Useful websites Contents

3. © Crown copyright Met Office Tux the penguin, mascot of Linux http://www.linux.org/ A form of the Unix operating system. Freely downloadable from the internet. What is Linux?

4. © Crown copyright Met Office Directory structure / bindevhome dir1dir2 rootusr

5. © Crown copyright Met Office Navigating the file system cd Change Directory cd home/dir1 to get to the dir1 directory cd.. to go up one directory (parent directory) cd $HOME or cd ~ to navigate back to your personal directory no matter where you are cd - goes to the former working directory pwd Print Working Directory to display the path of the current directory

6. © Crown copyright Met Office List directory contents ls Lists files and directories ls -a lists all files, including hidden ones. ls -l displays the file list in long format, including file details like size, time stamp, and owner. ls -F adds / after the name for directories, an * for executables, and @ for linked files. ls -r reverses the sort order (alphabetic or time). ls -t sorts the list by the time each file was created.

7. © Crown copyright Met Office Copying and renaming files cp Copy cp file1 file2 makes a copy of file1 in same directory cp file1 /dir2/ makes a copy of the file in dir2 mv To move or rename files mv file1 file001 to rename file1 to file 001 mv file001 dir2 to move file001 into dir2

8. © Crown copyright Met Office Deleting files and directories rm file1 removes the file. Directories mkdir dir3 makes a new directory. rmdir -r dir1 deletes the directory and the r flag is used to delete everything underneath it.

9. © Crown copyright Met Office Bash Some useful stuff Tab completion To copy double click with left mouse button and paste by pressing middle mouse button. Some keyboard shortcuts

10. © Crown copyright Met Office Regular expressions * ls e* lists all files that start with e ls *pdf lists all files that end in pdf ? ls file?.dat will list files such as file1.dat and file7.dat but will not list file001.dat

11. © Crown copyright Met Office Other Basic Commands catConcatenate and display lessCan move through a file when viewing it manManual touchMakes a new file clearClears the terminal screen emacsEditor picoEditor viEditor acroreadAcrobat Reader

12. © Crown copyright Met Office PP Utilities, CDO and NCL

13. © Crown copyright Met Office What are they? CDO, NCL and PRECIS utilities are tools for data analysis and visualisation. Different tool are useful for different activities e.g. CDO and PRECIS tools for analysis and NCL for visualisation. In the worksheets these tools are used together to process, analyse and visualise PRECIS output.

14. © Crown copyright Met Office PRECIS utilities Designed for use with PRECIS output (which is in Met Office PP format.) Can be used with Linux and Windows. Can be typed directly into the command line, or scripted. Extensive range of different utilities available, see http://www.metoffice.gov.uk/precis/tutorials http://www.metoffice.gov.uk/precis/tutorials For example – calculate the JJA mean: ppstats –H “mon>=6 and mon<=8” –o jja.mean.pp –t mean infile.pp

15. © Crown copyright Met Office CDO – Climate Data Operators (1) Developed by the Max Planck Institute for use with GRIB 1/2, netCDF 3/4, SERVICE, EXTRA and IEG data. Designed specifically for climate and NWP data analysis, there are more than 600 operators available. Can be run on Linux, Windows, MasOS and others. As with PP utilities can be typed directly into the command line, or scripted. For example – calculate the JJA mean: cdo timmean –selmon,6/8 infile.nc jja.mean.nc

16. © Crown copyright Met Office Free to download and documentation and support forums can be found at https://code.zmaw.de/projects/cdohttps://code.zmaw.de/projects/cdo CDO – Climate Data Operators (2)

17. © Crown copyright Met Office NCL – NCAR Command Language (1) Developed by NCAR (Nation Center for Atmospheric Research) for use with GRIB 1/2, netCDF 3/4, shape files and ascii data. Free interpreted language designed for scientific data processing and visualization. Can be run on Linux, Windows, MasOS and others. Can be typed directly into the NCL command line (very time consuming), or scripted. Produces plots of publishable quality.

18. © Crown copyright Met Office NCL – NCAR Command Language (2) Extensive documentation and hundreds of example scripts and plots at http://www.ncl.ucar.edu/index.shtmlhttp://www.ncl.ucar.edu/index.shtml

19. © Crown copyright Met Office NCL – NCAR Command Language (3) Looks more complex than it is.

20. © Crown copyright Met Office Using tools together The worksheets combine all three of these tools to post process and analysis PRECIS data. 1.Use PP utilities to remove rim and convert from PP to netCDF format. 2.Use CDO to analyse the netCDF data. 3.Use NCL to visualise (and process) the data.

21. © Crown copyright Met Office Worksheets 1 – File locations and pre-processing (into to Linux and using PP utilities.) 2a- Introduction to using CDO for analysis and NCL for visualisation (validation.) 2b- Creating future climate scenarios and analysing change (using CDO and NCL.) 3-Preparing data for GLAM using CDO.

22. © Crown copyright Met Office Some useful websites Doctor Bobs Lowfat Linux http://lowfatlinux.com/ http://lowfatlinux.com/ Getting started with Linux http://www.linux.org/lessons/beginner/toc.html http://www.linux.org/lessons/beginner/toc.html Unix Tutorial for beginners http://www.ee.surrey.ac.uk/Teaching/Unix/index.html http://www.ee.surrey.ac.uk/Teaching/Unix/index.html

23. © Crown copyright Met Office Questions