UGIC 2009 Introduction to Open Source RS/GIS programming with Python Chris Garrard RS/GIS Laboratory Utah State University

UGIC 2009 Why bother with FOSS? Free and Open Source Software (FOSS) Affordable (free, as in free beer) Fits the budget of small companies or individuals Open (as in free speech) Use it for whatever you want Distribute it if you want Modify it if you want

UGIC 2009 Why bother with FOSS? Helpful user community Including the developers themselves Fast bug fixes Not limited to Windows Lightweight

UGIC 2009 Cons Lightweight Not the ESRI geoprocessor Smaller user community than some other packages

UGIC 2009 Free != inferior Apache, MySQL, Linux, Firefox, etc. ESRI likes Python GDAL is used by popular commercial packages ArcGIS, Erdas Imagine, ER Mapper

UGIC 2009 Python Free, even for resale Powerful and has extensive libraries Easy to integrate with other tools Runs on many different operating systems Easy to learn

UGIC 2009 GDAL and OGR libraries Free, even for resale Available for many different operating systems Written in ANSI C and C++ Bindings exist for Python, C#, Perl, Java, Ruby, R, VB6

UGIC 2009 OGR OGR Simple Features Library Vector data access Software-specific formats shapefiles, personal geodatabases, ArcSDE, MapInfo, GRASS, Microstation Open formats TIGER/Line, SDTS, GML, KML Databases MySQL, PostgreSQL, Oracle Spatial, Informix, ODBC

UGIC 2009 GDAL Geospatial Data Abstraction Library Raster data access Supports about 100 different formats ArcInfo grids, ArcSDE raster, Imagine, Idrisi, ENVI, GRASS, GeoTIFF HDF4, HDF5 USGS DOQ, USGS DEM ECW, MrSID TIFF, JPEG, JPEG2000, PNG, GIF, BMP

UGIC 2009 Real-world examples Drill through a few hundred raster images and pull out cell values at several hundred points (~3.5 million values in output) Batch compute NDWI (Normalized Difference Water Index) for Landsat imagery Batch clip extraneous data from edges of images

UGIC 2009 Real-world examples Created a text file with info (bounding coordinates, dimensions, etc) about several hundred MSS images which was then used to input data into a database Created polygons showing bounding coordinates of actual data in several hundred raster images Logistic regression

UGIC 2009 Location Calculator

UGIC 2009

def getdata (keys=''): ''' possible ways to call this function: value=getdata('key') dict=getdata(('key1','key2')) #get subset dict=getdata() #return everything It assumes you don't have the same key for GET and POST methods ''' import types key_type=type(keys) d_data={} #initialize dictionary if keys=='': #if they don't supply keys then return everything for key in Request.Form: d_data[key]=Request.Form(key) for key in Request.QueryString: d_data[key]=Request.QueryString(key) return d_data elif key_type == types.StringType: #if they provide a single string value=Request.Form(keys) if (value != 'None') and (str(value) == 'None'): return Request.QueryString(keys) else: return value #if they provide a list then return a dictionary with all the key/values elif key_type == types.TupleType or key_type == types.ListType: for key in keys: value=Request.Form(key) #now check if the data was empty, if so look at QueryString if (value != 'None') and (str(value) == 'None'): value=Request.QueryString(key) data[key]=value return d_data if str(key) == 'utm': X_UTM=getdata('UTMx') Y_UTM=getdata('UTMy') datum=getdata('datumutm')

UGIC 2009 UTM NAD27 Zone 12 North Projects to:

UGIC 2009 UTM NAD27 Zone 12 North Projects to:

UGIC 2009 UTM NAD27 Zone 12 North Projects to:

UGIC 2009 Geocoding

UGIC 2009 Shapefile from SQL

UGIC 2009 Ex. 1: Reading vector data Open a point shapefile For each point: Get the “id” and “cover” attributes Get the x,y coordinates for the point Print “id x y cover”

UGIC 2009 Ex. 2: Extracting vector data Extract all “Quacking aspen woodland alliance” SWReGAP field sites in Wasatch county from the statewide data set Open a point shapefile with SWReGAP sample sites Open a Utah counties polygon shapefile Create a new point shapefile Use the county file to limit the SWReGAP shapefile to Wasatch County

UGIC 2009 Ex. 2: Extracting vector data Limit SWReGAP to “Quaking aspen woodland alliance” For each point in SWReGAP (remember it is now filtered spatially and by attribute): Copy the feature into the new shapefile Close all of the files

UGIC 2009 Ex. 3: Generate Near Table tool Given a point shapefile (SWReGAP field sites) and a line shapefile (roads), calculate the distance from each point to the nearest road Open both shapefiles For each point: Limit the search to the provided radius by buffering the point and using that to set a spatial filter on the roads

UGIC 2009 Ex. 3: Generate Near Table tool For each point, continued: Get the distance to the first line and its FID Loop through the rest of the lines and if the distance is less than the “remembered” distance, then remember the new distance and FID Write out the point FID and the remembered line FID and distance to a text file Close the shapefiles and the text file Needs an ArcInfo license in ArcMap (can probably write your own script, though)

UGIC 2009 Ex. 4: Extract pixel values Open a point shapefile Open a 3-band raster file For each point in the shapefile: Print out the pixel values at that location for all three bands “Sample” tool requires Spatial Analyst (to be fair, I think I may have done this with the arcgisscripting module without SA)

UGIC 2009 Ex. 5: Compute NDVI Create a Normalized Difference Vegetation Index for an ASTER image Open the image Get the image dimensions and block size Use this info to create a new image Read in one block at a time from the image Compute the NDVI: (band3-band2)/(band3+band2) Write the block to the new image Compute the statistics for the new image

UGIC 2009 Ex. 5: Compute NDVI Set the georeferencing and projection for the new image Build pyramids for the new image Needs the Spatial Analyst extension, and I don’t know if you could do it with anything in the geoprocessor without SA – I’m sure you could do it with ArcObjects

UGIC 2009 Ex. 6: NDVI in ArcMap Run the NDVI script in ArcMap Change the script to accept parameters Change the script to send error messages to the geoprocessor Add the script to a toolbox

UGIC 2009 Thanks! Course materials at I can be contacted at