MapServer and SDE Howard Butler Center for Survey Statistics & Methodology Iowa State University Beginning Middle End

National Resources Inventory National longitudinal survey of multiple natural resources Estimates of status and change over time on nonfederal lands –Land cover/use changes –Soil erosion trends –Urbanization and loss of prime farmland –Wetland dynamics Beginning Middle End

National Resources Inventory > 300,000 area segments –160 acres each –70,000 samples / year –Low altitude photography Beginning Middle End

National Resources Inventory 1/3 digital -- 2/3 analog > 18 gb of support data –Federal land –Water –Sample location layers –Support imagery Beginning Middle End

NRI GIS Data History Recently migrated to ESRI SDE Oracle database for tabular data Changes/edits tracked with ArcGIS editing tools Beginning Middle End

Use of MapServer in NRI The need for speed –ArcIMS doesn’t even compare on the same hardware –We can get by with cheaper hardware because of improved performance –Lower software licensing costs Beginning Middle End

Use of MapServer in NRI Still need SDE features, though –Versioned queries Edits to data are tracked through a versioned editing workflow. Web mapping applications must be able to see these various versions –Joins Spatial and tabular joins Beginning Middle End

2001 Survey year Survey edits Revision edits Frozen edit state N 2002 version of the 2000 control data NRI Versioned Data Tree

Use of MapServer in NRI MapServer brings features, too –Fluent WMS allows us to leverage other data sources that we don’t have to manage –Bulletproof projection on-the-fly support –MapScript is simpler, more robust, more flexible (supports many languages), and faster than many of the ArcIMS connector technologies such as ActiveX and JSP Beginning Middle End

Recent improvements in MapServer SDE Support Support for Versioned Queries –Can specify the version on the CONNECTION string to query from Auto-sensing of SDE table structure –SDE maintains a unique ID for each table. The user can change the name of this column at will, and many applications within the ESRI family are not consistent. Beginning Middle End

Recent improvements in MapServer SDE Support New internal query syntax –ArcSDE 8.x changed the query syntax in its C API. The new methods allow the user to pre- join feature and business tables, making the queries much faster. –This change also will allow us to support user- specified joins similar to XBASE joins in the future. Beginning Middle End

Compiling in SDE Support Linux –Easy. Just turn on the configure line parameters related to SDE (--with-sde, - -with-sde-version) Solaris –Same as Linux, except that make sure everything is compiled with the same compiler Beginning Middle End

Compiling in SDE Support Windows –Harder. Configure all of your settings in nmake.opt –Ensure that the.def table for all of the C API functions is available to map the function names to the Windows calling convention Beginning Middle End

How to Connect to SDE Define a Layer in your MapFile Beginning Middle End LAYER NAME states TYPE POLYGON CONNECTION "sdemachine.iastate.edu,port:5151,sde,username,password,SDE.DEFAULT" CONNECTIONTYPE SDE DATA "HOBU.STATES_LAYER,SHAPE" TEMPLATE '/where/the/template/file/is/located' CLASS STYLE SYMBOL 'circle' SIZE 3 COLOR OUTLINECOLOR END More detail at:

How to Connect to SDE Beginning Middle End CONNECTIONTYPE SDE MapServer will report an error of SDE Support not available if SDE has not been properly compiled-in

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -Machine Name -Can be an IP Address -Often needs to be the FQDN

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -SDE Service location -port: is important -esri_sde for systems specified in /etc/services

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -The database name in which SDE is stored -This is often “SDE” -Not required by all DBMS, but some text is required by MapServer

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -User must be granted rights to select the table in the DATA directive using either ArcCatalog or the SDE command-line utilities

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -The user’s password -CASE SENSITIVE

How to Connect to SDE Beginning Middle End CONNECTION "sdemachine.iastate.edu, port:5151, sde, username, password, SDE.DEFAULT" -The version to connect to -SDE.DEFUALT is used if none is specified -CASE SENSITIVE

How to Connect to SDE Beginning Middle End DATA "HOBU.STATES_LAYER,SHAPE" SchemaTable NameSpatial Column No spaces! Oracle and MSSQL usually require the schema name – use the full name shown in ArcCatalog if not sure

How to Connect to SDE Beginning Middle End TEMPLATE '/where/the/template/file/is/located' Must be specified to query attributes Unnecessary performance hog if you are only drawing layers – especially if the layer has a lot of attributes

An Example Application NRI DataView ID-Based Python MapScript TerraServer WMS pyTerra for image dates PIL for post-processing image manipulation Beginning Middle End

An Example Application 300,000 PSUs –QueryByAttribute would require a sequential scan of the entire table –The ID has natural hierarchy 19001_010101R –First two digits are the state FIPS code –Second three digits are the county FIPS code Use QueryByAttribute on the USSTATES table to limit the extent of the query of the USCOUNTIES table Use QueryByAttribute on the USCOUNTIES table to limit the extent of the query on the PSU table Beginning Middle End

An Example Application Beginning Middle End

An Example Application Beginning Middle End

Upcoming MapServer SDE Improvements QueryByShape –Currently only extents are supported –SDE has support for advanced operations Buffer select, touching, common edge Beginning Middle End

Upcoming MapServer SDE Improvements Joins –Will be implemented similar to XBASE and MySQL –As fast or as slow as your database –We get around this limitation with database views right now Beginning Middle End

Upcoming MapServer SDE Improvements SDE Raster Support –Been talked about and requested for a long time –No one has the time or desperate need for it to fund development –Could be implemented through GDAL Beginning Middle End

The End Questions? Beginning Middle End