1. 10/15/05 http://www.wdssii.org/ 1 Making a build of WDSS-II from the source Jeff Brogden National Severe Storms Laboratory & University of Oklahoma

2. 10/15/05http://www.wdssii.org/ 2 Concurrent Versions System CVS – Concurrent Versions System CVS is a version control system which allows us to record the history of source file changes. Allows multiple developers to make changes to the code at the same time. Can access it across a network. WDSS-II's repository is on tensor.nssl.noaa.gov Not accessible from outside the lab Source code for infrastructure, algorithms is not freely distributed Make sure that you are allowed to put source code on a machine before you make a build on that machine No source code on machines belonging to private companies, for example. Make build on NSSL machines, then move the build over.

3. 10/15/05http://www.wdssii.org/ 3 CVS Access To view the repository and logs, you can use the web. http://tensor.nssl.noaa.gov/http://tensor.nssl.noaa.gov/ Two ways to access CVS Using pserver setenv CVSROOT :pserver:jbrogden@tensor:/var/cvsjbrogden@tensor export CVSROOT=:pserver:jbrogden@tensor:/var/cvsjbrogden@tensor Using ssh (preferred) setenv CVS_RSH ssh setenv CVSROOT jbrogden@tensor:/var/cvs Your LDAP username and password will work Use ssh techniques to give yourself password-less access Log into CVS cvs login

4. 10/15/05http://www.wdssii.org/ 4 Build Scripts Builds are best created with the perl script build_distribution.pl Interface to this script with build_distribution.sh Shell script to help make builds easier Allows you to set the appropriate options Always use the most current build_distribution.pl to create builds.

5. 10/15/05http://www.wdssii.org/ 5 Build script options Options are set in build_distribution.sh: perl build_distribution.tmp\ user=":pserver:jbrogden"\ loc="/mnt/home/jbrogden/CLEAN/WDSS2"\ packages="w2 w2algs w2tools"\ static=no 'perl build_distribution.pl -help' will give a list of options you can set.

6. 10/15/05http://www.wdssii.org/ 6 Options user (jbrogden) – user name on tensor (LDAP user name). loc (/home/jbrogden/WDSS2) – location of build in your directory. branch () – empty indicates latest. asof () – empty indicates latest, (any format that cvs -D can accept, including 20051017 or yesterday) static (yes) – or shared libraries. extras (--disable-orpg --disable-hires --disable-wish --enable- nexrad --enable-netssap)

7. 10/15/05http://www.wdssii.org/ 7 Options - cont... packages (gdal g2clib netcdf config orpginfr udunits w2 w2ext w2algs w2tools) optimized (no) – uses O2 for gcc debug (yes) – debug flag (-g) to compiler doc (no) – build the documentation tar (no) – make distribution tar file. cxxflags () – additional compile flags checkout (yes) – check out or just build

8. 10/15/05http://www.wdssii.org/ 8 Options cont.... researchonly (no) – if turned on, this build is created without realtime support, for research users only. The free version of WDSS-II is created with researchonly gdal (yes) – build applications that rely on gdal. numcpus (1) – number of CPUs your machine has. distclean (no) – do a 'make distclean' in the directory to remove detritus from earlier builds.

9. 10/15/05http://www.wdssii.org/ 9 Using CVS directly Documentation available on tensor's web page. http://tensor.nssl.noaa.gov/cvs.html Basic Commands checkout 'cvs checkout w2' Creates a directory called 'w2' and populates it with files. Committing changes in existing file cvs commit filename starts an editor so you can enter a log message. use '-m "message" ' to avoid starting editor

10. 10/15/05http://www.wdssii.org/ 10 Using CVS - cont... viewing differences cvs diff filename runs the diff command to compare your copy to latest checked in copy of file. tkdiff filename will launch graphical diff If you don’t have tkdiff, Google it and install it (just a script) adding files to existing directories – two step process cvs add filename use '-kb' if it's a binary file cvs commit filename actually check file in and make it available remove files – three step process cvs remove filename rm filename cvs commit filename

11. 10/15/05http://www.wdssii.org/ 11 Using CVS - cont... checking the status of files cvs status [filename] cvs log [filename] updating and merging cvs update filename – updates a specified file cvs update – recursively update all files from current directory cvs update directory – update a specific directory Works pretty well, but there can be conflicts. saves original file in '.#filename.version (ie.#driver.c.1.4) new file contains conflict markers to show where conflicts exist. ' >>>>>>'.

12. 10/15/05http://www.wdssii.org/ 12 WDSS-II Packages gdal – for converting to GeoTIFF (optional) g2clib – for converting to GRIB2 (optional) netcdf – for reading/writing NetCDF files config – used to get autoconfig stuff working orpginfr – used for LB and event-handling infrastructure udunits – unit conversion package w2 – all WDSS-II infrastructure w2ext – extra data formats (Level-III, ORPG...) (optional) w2algs – algorithm development w2tools – display and its supporting infrastructure & a few other data source management tools

13. 10/15/05http://www.wdssii.org/ 13 Adding an algorithm Writing algorithms covered in earlier session. We recommend: make most of code a library have apps as a subdirectory use an existing algorithms Makefile.am file as a template for your own modify w2algs/configure.in to add your Makefile add all your code to the CVS repository run./autogen.sh and make to be sure things work correctly. Look in w2/w2config/algcreatorfiles/README for more information.

14. 10/15/05http://www.wdssii.org/ 14 Doxygen Doxygen allows us to use code comments to auto-generate documentation. Current documentation is available at http://tensor.nssl.noaa.gov/cgi-bin/doxlist.pl http://tensor.nssl.noaa.gov/cgi-bin/doxlist.pl Example of documentation: /** SampleClass * Implements a series of funcitons that operates on Samples * @see BigClass * @author LastName */ class SampleClass : public BigClass { public: /** Construct a SampleClass with the number of parameters. * @param num How many samples there are. * @param val The default value. */ SampleClass(int num, double val); }

15. 10/15/05http://www.wdssii.org/ 15 Environment Variables If, for some reason, you have things installed in non-standard places, you can use environment variables to inform the autoconfig process of the build where they exist. Some examples: GLDIR – location of GL libraries FREETYPEDIR – location of FreeType Font library NCDIR – location of NetCDF libraries For the most part, you shouldn't ever need to do this. If you are porting to a different platform where this is where these files will always be, add the directory to config/w2.m4.

16. 10/15/05http://www.wdssii.org/ 16 Authorization files WDSS-II has an authorization mechanism that allows you to control access to different features. You can distribute authorization files to users that "turn-off" capabilities or "turn-on" things without having to recompile. For algorithms that wish to restrict users, there is a config file. each line states the restriction. (ie "realtime false") All algorithm's authorization keys are collected into single key file. The authorization key value (or the file) should NOT be distributed Keep the authorization key file safe.

17. 10/15/05http://www.wdssii.org/ 17 Use W2Authorization class just after you get all the startup options #ifndef algorithm_AUTH_KEY W2Authorization auth("alg_name", algorithm_AUTH_KEY); // Make sure authorized for more than 3 tiles if (auth.getString(“numTiles”) != “ANY” && num_tiles> atoi(auth.getString(“numTiles”).c_str()) ) throw std::string(“Exceeded tiles limit!”); #endif Make a new file called w2/w2config/auth/default/alg_name Run w2auth to create a default auth for this file w2auth -i w2config/auth/default -o w2config/auth \ -k w2config/auth/default/key Check the added/modified files in the auth and default directories into CVS How to restrict code

18. 10/15/05http://www.wdssii.org/ 18 Format of Authoriztion file Field-name values-seperated-by-commas no extra spaces in the values field true/false are special literals ANY is a special value domainnssl.noaa.gov,ou.edu userANY untilANY realtimetrue numTiles 5

19. 10/15/05http://www.wdssii.org/ 19 Making a build for external users Generate authorization files Use w2auth to generate 128-bit keys that are hard to break Keep keyfile in a safe place – no way to recreate it if you lose it. Use different keyfiles for different licensees Write plain-text authorization files (format on previous slide) Run w2auth to create encrypted.auth files Make a build Specify tar=yes Specify keyfile=location_to_key Replace auth directory in tar file Replace the w2config/w2auth directory in temp_build Recreate tar file – this is what you need to distribute No source code to be distributed outside NSSL Binary distribution outside NSSL must be approved

20. 10/15/05http://www.wdssii.org/ 20 Extracting an algorithm? The algorithm scientific part is separate from the WDSS-II infrastructure is dependent on it for input, output, data formats, notification, factories, data objects, etc. This common infrastructure Makes algorithm development easy and productive Makes “extracting” the scientific part almost impossible. You will have to code up the WDSS-II services again Not recommended Unless you can code up the services based on similar facilities that already exist in another system (e.g: AWIPS) If this is the case, we can help you create the appropriate adapters.

21. 10/15/05http://www.wdssii.org/ 21 Developer services Changes checked into cvs since the previous day are e-mailed out to all users of cvs Tell ITS that you need to be added to the “cvs” group Name of file, and message listing reason for change Make sure to explain changes Use this log to keep up to date on changes to entire system Documentation automatically built on tensor The documentation on tensor is up-to-date daily Document all classes that others will use! The documentation on wdssii.org reflects the latest research build.