Web-Based Open Source GIS: Decision Support Tools Explaining the Software Stack Presented by Aaron Racicot – GIS Programmer April 19th, 2006 A Citizen of Salmon Nation

Outline  Introduction - Ecotrust  Benefits/Limitations of Open Source  OCEANSystem  Problem description  Technology Overview - Software Stack  FORESTSystem  Problem description  Technology Overview - Software Stack  Where is it all going?

Who am I… B.S. Computer Science Split Personality Telemark/BC Skiing Rafting Guide Mountain Rescue M.S. Environmental Science Open Source User/Developer GIS Programmer

Ecotrust - Salmon Nation

Benefits/Limitations of Open Source Benefits  Software cost = $0  Source code is available and modifiable  User and development communities flourish  Development cycles are VERY fast Limitations  Total cost is NOT = $0  Many applications are not as polished as the proprietary counterparts  Compatibility with proprietary software can be an issue

Software Stacks… Desktop Generic StackFOSS4G StackGrouping ApplicationQGis, Grass, OSSIM, JUMP, UDig User Interface Application Dev. Environment Eclipse, QT, OpenGL High Level UtilitiesGeoTools, PostGISData Serving High Level Scripting Languages PHP, PERL, Python Low Level UtilitiesShplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT Data Processing Low Level Languages C, C++, Java, FortranSystem Software Operating System Linux, Darwin, Cygwin Hardware Drivers bit processor drivers Desktop Software Stack

Software Stacks... Server Server Software Stack Generic StackFOSS4G StackGrouping Client Side BrowserFirefox, Safari, NetscapeUser Interface Client Side Scripting Java Script, Java Applets ~~~~~~~~~~~WEB~~~~~~~~~~~ Server Side Application Chameleon, Cartoweb3, CustomData Serving High Level UtilitiesMapserver, PCO, Grass Low Level UtilitiesShplib, GEOS, OGR/GDAL, PostGIS, R-Statistics, GMT Data Processing High Level Scripting Languages PHP, PERL, Python Low Level Languages C, C++, Java, FortranSystem Software Operating System/Drivers Linux, Darwin, Cygwin Hardware Drivers32-64 bit processor drivers

Software Stack... DSTs (Ecotrust) DST Software Stack Generic StackFOSS4G StackGrouping Client Side Browser Firefox, Safari, Netscape User Interface Client Side Scripting Chameleon/Custom ~~~~~~~~~~~WEB~~~~~~~~~~~ Server Side Application Chameleon Data Serving High Level Utilities Mapserver, GRASS, PostGIS Low Level Utilities Shplib, GEOS, R-Statistics, GMT, OGR/GDAL Data Processing High Level Scripting Languages PHP, PERL Low Level Languages C, C++, Java, Fortran System Software Operating System/Drivers Linux (Fedora) Hardware Drivers 32 bit processor drivers

Work Flow – Real-Time Web DST Web User Request Web User Response Static Data Storage Web Services Data Gathering Data Formatting Data Processing Map Formatting Map Production Real-Time Web Decision Support Tools

Work Flow – What OSGIS is good at Web User Request Web User Response Static Data Storage Web Services Data Gathering Data Formatting Data Processing Map Formatting Map Production The Problem Is The Arrows! Connecting a web request to server side GIS analysis is tricky Real-Time Web Decision Support Tools

Data Flow... DSTs (Ecotrust)

Raster based DST Siuslaw Watershed Restoration Initiative Data Gathering Data Formatting Data Processing PHP Mapscript GRASS

Raster based DST - Code class Grass_GIS { var $data_dir; // Where the GRASS data dir is var $rc_dir; // Where the GRASS rc dir is var $bin_dir; // Where the GRASS bin dir is var $location; var $mapset; function Grass_GIS($data_dir, $rc_dir, $bin_dir, $location, $mapset) { $this->data_dir = $data_dir; $this->rc_dir = $rc_dir; $this->bin_dir = $bin_dir; $this->location = $location; $this->mapset = $mapset; } function Grass_GIS_copy($grass_gis) { $this->data_dir = $grass_gis->data_dir; $this->rc_dir = $grass_gis->rc_dir; $this->bin_dir = $grass_gis->bin_dir; $this->location = $grass_gis->location; $this->mapset = $grass_gis->mapset; } function set_location($location) {$this->location = $location;} function set_mapset($mapset) {$this->mapset = $mapset;} function set_data_dir($data_dir) {$this->data_dir = $data_dir;} function set_rc_dir($rc_dir) {$this->rc_dir = $rc_dir;} function set_bin_dir($bin_dir) {$this->bin_dir = $bin_dir;} function run_command($cmd){ $std_output = ""; $grass_exec = ""; // Check that all the required info is available.. if ($this->data_dir != NULL && $this->rc_dir != NULL && $this->bin_dir != NULL && $this->location != NULL && $this->mapset != NULL){ // Here we will write out the script to run... $fp_data = fopen($this->data_dir."/grass_run.sh","w"); // write out text header fwrite($fp_data,"export GISBASE=".$this->bin_dir." \n"); fwrite($fp_data,"export GISDBASE=".$this->data_dir." \n"); fwrite($fp_data,"export GISRC=".$this->rc_dir."/.grassrc6 \n"); fwrite($fp_data,"export ETC=".$this->bin_dir."/etc \n"); fwrite($fp_data,"export PATH=".$this->bin_dir."/bin:".$this->bin_dir."/scripts:".$this->bin_dir. "/garden/bin:\$PATH:/usr/bin:/usr/local/bin/ \n"); fwrite($fp_data,"export LOCATION_NAME=".$this->location." \n"); fwrite($fp_data,"export MAPSET=".$this->mapset." \n"); fwrite($fp_data,"export LOCATION=".$this->data_dir."/".$this->location."/".$mapset." \n"); fwrite($fp_data,"\n"); fwrite($fp_data,$cmd."\n"); fflush($fp_data); fclose($fp_data); $fp_data = fopen($this->rc_dir."/.grassrc6","w"); fwrite($fp_data,"GISDBASE: ".$this->data_dir."\n"); fwrite($fp_data,"LOCATION_NAME: ".$this->location."\n"); fwrite($fp_data,"MAPSET: ".$this->mapset."\n"); fwrite($fp_data,"PAINTER: ppm\n"); fwrite($fp_data,"MAPLP: stuff.ppm\n"); fflush($fp_data); fclose($fp_data); $grass_exec = sprintf("chmod u+x ".$this->data_dir."/grass_run.sh"); exec($grass_exec,$std_output,$std_error); $grass_exec = sprintf($this->data_dir."/grass_run.sh"); exec($grass_exec,$std_output,$std_error); }

Raster based DST - Code define("GRASSDATA_DIR", "/var/www/html/apps/siuslaw_target/data/GRASSDATA/"); $location_name = "OR_siuslaw“; $mapset_name = "land_targets“; $session_grass_dir = $session_tmp_dir."/GRASSDATA"; // Make a Dir to work in echo " Make a directory to work in inside this session "; $dir_exec = sprintf("mkdir ".$session_grass_dir); exec($dir_exec,$arr,$err1); $dir_exec = sprintf("mkdir ".$session_grass_dir."/rc"); exec($dir_exec,$arr,$err1); $template_path = GRASSDATA_DIR."siuslaw_epa_template"; echo " Copying over new GRASS GIS template "; $dir_exec = sprintf("cp -R ".$template_path." ".$session_grass_dir."/".$location_name); exec($dir_exec,$arr,$err1); // Make a MAPSET echo " Setting up the siuslaw specific area "; $dir_exec = sprintf("mkdir ".$ session_grass_dir."/".$location_name."/".$mapset_name); exec($dir_exec,$arr,$err1); // Copy over the projection info from the PERMANENT area echo " Setting projections "; $dir_exec = sprintf("cp ".$template_path."/PERMANENT/PROJ* ".$ session_grass_dir."/".$location_name."/".$mapset_name."/"); exec($dir_exec,$arr,$err1); $grass_session = new Grass_GIS($session_grass_dir, $session_grass_dir."/rc", "/usr/local/grass-6.0.0", $location_name, $mapset_name); $grass_session->run_command("g.list type=rast");

Vector based DST Mapserver PHP -> PHPMapscript -> PostGIS PostGIS

Vector based DST - Code // Now connect to the db $host_str = "host=localhost"; $db_str = "dbname=shelf_closure_db"; $usr_str = "user=web_user"; $string = $host_str." ".$db_str." ".$usr_str; $connection = pg_connect($string); if (!$connection){ pg_close($connection); echo "Error: Cannot connect to the database \n"; } else { echo "Connected to the SCA database \n"; } // Create a dynamic VIEW in PostGIS echo " * Create postgresql VIEW * \n"; $exec_str = sprintf("CREATE VIEW ". $previous_session_name."_".$previous_session_id." AS ". "SELECT bk.the_geom, sum(tr.total_lbs*tb.percent) as total_lbs ". "FROM tow%02d_3kblk_intersect as tb, trawl%02d". " as tr, cablk3km as bk, port as pt ". "where tb.blk_gid=bk.gid ".$species_sql." ".$port_sql." ".$gear_sql. " and tb.tow%02d_id=tr.tow_id2 ". "GROUP BY bk.the_geom HAVING count(tr.tow_id2)>2", $start_year_short,$start_year_short,$start_year_short); $result_pg_exec = pg_exec($connection, $exec_str); if (!$result_pg_exec) { printf(" %s \n", pg_errormessage());}

Vector based DST - Code // Execute the pgsql2shp to create the output echo " * Execute pgsql2shp create the shapefile in session directory * "; $dir_exec = sprintf("/usr/local/pgsql/bin/pgsql2shp -u web_user -f ". $session_tmp_dir."data/target_areas/test_output.shp shelf_closure_db ". "\"SELECT * from ".$previous_session_name."_".$previous_session_id."\""); exec($dir_exec,$arr,$err1); for ($ii=0;$ii<count($arr);$ii++) { echo $arr[$ii]." \n"; } // Drop the dynamic table echo " * DROP the postgresql VIEW * \n"; $result_pg_exec = pg_exec($connection, "DROP VIEW ". $previous_session_name."_".$previous_session_id); if (!$result_pg_exec) { printf(" %s \n", pg_errormessage()); }

13 Step Tool Install 1) First install Fedora Core4 with updates 2) Ming wget tar -xzvf ming-0.3beta1.tar.gz cd ming-0.3beta1 make make static cd php_ext/ make make install echo "Don't forget to add 'extension=php_ming.so' to php.ini!" 3) Pdflib wget tar -xzvf PDFlib-Lite tar.gz cd PDFlib-Lite /configure make make install 4) Update ldconfig Add /usr/local/lib to /etc/ld.so.conf Run : /sbin/ldconfig 5) Proj.4 tar -xzvf proj tar.gz cd proj cd nad/ ls cp../../proj-nad tar.gz. tar -xzvf proj-nad tar.gz cd.../configure make make install 6) GDAL Installed the ECW SDK (do a configure, make and install) Installed the MrSid SDK (moved to /usr/local) wget tar -xzvf gdal tar.gz cd gdal /configure --with-ecw --with mrsid=/usr/local/GeoExpressSDK/ make ogr-all make install which gdalinfo 7) GEOS wget tar -xjvf geos tar.bz2 cd geos /configure make make install 8) POSTGRESQL tar -xzvf postgresql tar.gz cd postgresql NOTE the LDFLAGS here... this is to support GEOS for postGIS LDFLAGS=-lstdc++./configure --with-perl --prefix=/usr/local/pgsql_8_1_2 gmake gmake install Add the shared library path to /etc/ld.so.conf : /usr/local/pgsql_8_1_2/lib Run /sbin/ldconfig Run make install in the./doc directory to install the documentation Finally add the following to the profile file to make paths available: PATH=$PATH:/usr/local/pgsql/bin MANPATH=$MANPATH:/usr/local/pgsql/man export MANPATH Add the postgis user useradd postgis passwd pgsql]$ postmaster -D./data/ >./data/logfile.txt 2>&1 &

13 Step Tool Install 9) POSTGIS Just make sure that the Postgresql is configured with the LDFLAGS variable set and that the GEOS software is downloaded and installed from Refractions../configure --with-proj --with-geos --with-pgsql make make install Now we can create the database: createdb -O aaronr test_db createlang -U aaronr plpgsql test_db psql -f /usr/local/pgsql_8_1_2/share/postgresql/contrib/lwpostgis.sql -d test_db 10) PHP./configure --prefix=/usr/local/php4 --program-suffix=4 --enable-force-cgi-redirect --with- config-file-path=/etc/httpd/ --with-gd=/usr/local/ --with-jpeg --with-png --with-tiff --with-zlib --with-freetype-dir --without-ttf --with-mysql --with-regex=system -- enable-dbase --enable-dbx --enable-versioning --with- pgsql=/usr/local/pgsql_8_1_2/ make make install strip sapi/cgi/php cp sapi/cgi/php /var/www/cgi-bin/php4 cp php.ini-dist /etc/httpd/php.ini Made the following mod to /etc/httpd/php.ini: ; Directory in which the loadable extensions (modules) reside. ;extension_dir = "./" extension_dir = "/etc/httpd/php_mods" mkdir /etc/httpd/php_mods Added the following to /etc/httpd/conf/httpd.conf # # For PHP scripts as CGI-BIN # AddType application/x-httpd-php-cgi.php4.phtml Action application/x-httpd-php-cgi /cgi-bin/php4 Now re-start the server: /etc/rc.d/init.d/httpd restart Stopping httpd: [ OK ] Starting httpd: [ OK ] 11) MAPSERVER./configure --with-jpeg --with-gd --with-freetype --with-zlib --with-png --with-pdf -- without-tiff --with-proj --with-threads --with-ogr --with-gdal --with-postgis --with- wfs --with-wmsclient --with-wfsclient --enable-debug --with-php=/src/php/php make cp legend mapserv scalebar /var/www/cgi-bin/ cp mapscript/php3/php_mapscript.so /etc/httpd/php_mods/ 12) GRASS CFLAGS="-g -Wall"./configure --with-gdal=/usr/local/bin/gdal-config --with-postgres- includes=/usr/local/pgsql/include/ --with-postgres-libs=/usr/local/pgsql/lib/ make make install 13) R-Statistics tar -xzvf R tar.gz cd R /configure make make install

Where to go for more info OSGIS  Maptools -  FreeGIS -  Open Source GIS - Standards  OGC - Desktop  GRASS -  QGIS -  UDIG -  JUMP –  OpenEV - Server/Web  Mapserver -  GRASS -  PostGIS - Tools  Remote Sensing -  GDAL/OGR -  PROJ  R-Statistics -  GMT -

The End Tool Screen Shots Follow

Backup - GRASS

Backup – R-Statistics

Backup – PostGIS Geometry WKT Geometry

Backup - QGIS