Traditional Statistics Mean, StDev (Normal Curve) Mean, StDev (Normal Curve) Central Tendency Central Tendency Typical Response (scalar) Typical Response (scalar) Minimum= 5.4 ppm Maximum= ppm Mean= 22.4 ppm StDEV= 15.5 Spatial Statistics Map of the Variance (gradient) Map of the Variance (gradient) Spatial Distribution Spatial Distribution Numerical Spatial Relationships Numerical Spatial Relationships Spatial Distribution (Surface) Traditional GIS Points, Lines, Polygons Points, Lines, Polygons Discrete Objects Discrete Objects Mapping and Geo-query Mapping and Geo-query Forest Inventory Map Grid-based Map Analysis (Spatial Analysis/Statistics) Spatial Analysis Cells, Surfaces Cells, Surfaces Continuous Geographic Space Continuous Geographic Space Contextual Spatial Relationships Contextual Spatial Relationships Erosion Potential (Surface)

Grid-Based Map Analysis Spatial analysis investigates the “contextual” relationships in mapped data… Reclassify — reassigning map values (position; value; size, shape; contiguity) Reclassify — reassigning map values (position; value; size, shape; contiguity) Overlay — map overlay (point-by-point; region-wide; map-wide) Overlay — map overlay (point-by-point; region-wide; map-wide) Distance — proximity and connectivity (movement; optimal paths; visibility) Distance — proximity and connectivity (movement; optimal paths; visibility) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) (Berry) Data Mining investigates the “numerical” relationships in mapped data… Descriptive — aggregate statistics (e.g., average/stdev, similarity, clustering) Descriptive — aggregate statistics (e.g., average/stdev, similarity, clustering) Predictive — relationships among maps (e.g., regression) Predictive — relationships among maps (e.g., regression) Prescriptive — appropriate actions (e.g., optimization) Prescriptive — appropriate actions (e.g., optimization) Surface modeling maps the “spatial distribution” and pattern of point data… Map Generalization — characterizes spatial trends (e.g., titled plane) Map Generalization — characterizes spatial trends (e.g., titled plane) Spatial Interpolation — deriving spatial distributions (e.g., IDW, Krig) Spatial Interpolation — deriving spatial distributions (e.g., IDW, Krig) Other — roving window/facets (e.g., density surface; tessellation) Other — roving window/facets (e.g., density surface; tessellation) Spatial Statistics

Point Density Analysis Point Density analysis identifies the number of customers with a specified distance of each grid location Roving Window (count) (Berry)

Identifying Unusually High Density Pockets of unusually high customer density are identified as more than one standard deviation above the mean (Berry)

Surface Modeling (Density Surface) (Berry) Discrete Map Surface 2 Hugags every 30 min for 30 days HugagCounts Hugag Hugag Activity draped over Elevation Continuous Map Surface Most of the activity is in the NE Hugag Density Surface Avg StDev= 15.0 Roving Window Total number of counts within 6-cell radius Density Surface Modeling “Counts” the number of occurrences within a specified “roving window” reach— higher values indicate concentrations of occurrence …from discrete observations to continuous spatial distribution (Short Exercise #6)

Spatial Interpolation (Smoothing the Variability) The “iterative smoothing” process is similar to slapping a big chunk of modeler’s clay over the “data spikes,” then taking a knife and cutting away the excess to leave a continuous surface that encapsulates the peaks and valleys implied in the original field samples …repeated smoothing slowly “erodes” the data surface to a flat plane = AVERAGE (Berry) (digital slide show SSTAT2) SSTAT2

Inverse Distance Weighted Approach (Berry) Tobler’s First Law of Geography — nearby things are more alike than distant things 1/D Power

Spatial Autocorrelation (Kriging) Tobler’s First Law of Geography — nearby things are more alike than distant things Variogram — plot of sample data similarity as a function of distance between samples (Berry) …Kriging uses regional variable theory based on an underlying variogram to develop custom weights based on trends in the sample data (proximity and direction) …uses Variogram Equation instead of a fixed 1/D Power Geometric Equation

Surface Modeling Methods (Surfer) Inverse Distance to a Power — weighted average of samples in the summary window such that the influence of a sample point declines with “simple” distance Modified Shepard’s Method — uses an inverse distance “least squares” method that reduces the “bull’s-eye” effect around sample points Radial Basis Function — uses non-linear functions of “simple” distance to determine summary weights Kriging — summary of samples based on distance and angular trends in the data Natural Neighbor —weighted average of neighboring samples where the weights are proportional to the “borrowed area” from the surrounding points (based on differences in Thiessen polygon sets) Minimum Curvature — analogous to fitting a thin, elastic plate through each sample point using a minimum amount of bending (Spatial Interpolation) Nearest Neighbor — assigns the value of the nearest sample point Triangulation — identifies the “optimal” set of triangles connecting all of the sample points (Geometric Facets) Polynomial Regression — fits an equation to the entire set of sample points (Map Generalization) Thiessen Polygons (Berry) Geometric facets Map Generalization Spatial Interpolation

…all interpolation algorithms assume that… 1) “nearby things are more alike than distant things” (spatial autocorrelation), 2) appropriate sampling intensity (ample number of samples), and a 3) suitable sampling pattern …the interpolated surfaces “map the spatial variation” in the data samples …the interpolated surfaces “map the spatial variation” in the data samples Spatial Interpolation is similar to throwing a blanket over the “data spikes” to conforming to the geographic pattern of the data. (Berry)

Comparing Spatial Interpolation Results Comparison of the IDW interpolated surface to the whole field average shows LARGE differences in localized estimates (Berry) Comparison of the IDW and Krig interpolated surfaces shows small differences in in localized estimates

Surface Modeling (Full Exercise #6) (Berry) Use Surfer to interpolate a continuous surface… …and generate contour and solid surface plots Spatial Interpolation Use MapCalc to create a density surface (total count) Density Surface Derivation (Use MapCalc to derive a customer density surface) SCAN Total_Customers TOTAL WITHIN 6 FOR Customer_density6 SCAN Total_Customers TOTAL WITHIN 6 FOR Customer_density6 RENUMBER Customer_density6 ASSIGNING 0 TO 0 THRU 33.7 ASSIGNING 1 TO 33.7 THRU 1000 FOR Customer_highDensity RENUMBER Customer_density6 ASSIGNING 0 TO 0 THRU 33.7 ASSIGNING 1 TO 33.7 THRU 1000 FOR Customer_highDensity

Spatial Interpolation Techniques (Berry) Characterizes the spatial distribution by fitting a mathematical equation to localized portions of the data (roving window) AVG= 23 everywhere Spatial Interpolation techniques use “roving windows” to summarize sample values within a specified reach of each map location. Window shape/size and summary technique result in different interpolation surfaces for a given set of field data …no single techniques is best for all data. Inverse Distance Weighted (IDW) technique weights the samples such that values farther away contribute less to the average …1/Distance Power

AVG= 23 Spatial Interpolation (Evaluating performance) (Berry) Assessing Interpolation Results (Residual Analysis) …the best map is the one that has the “best guesses”

Spatial Interpolation (Spatially characterizing error) A Map of Error (Residual Map) …shows you where your estimates are likely good/bad (Berry)

Spatial Dependency (Spatial Autocorrelation & Correlation) the conditions of that variable at nearby locations, termed Spatial Autocorrelation (intra-variable dependence for Surface Modeling) the conditions of that variable at nearby locations, termed Spatial Autocorrelation (intra-variable dependence for Surface Modeling) (Berry) Spatial Variable Dependence — what occurs at a location in geographic space is related to: the conditions of other variables at the conditions of other variables at that location, termed Spatial Correlation (inter-variable dependence for Spatial Data Mining) …understanding relationships within a single map layer Basis for… SurfaceModeling …understanding relationships among map layers Basis for… Spatial Data Mining

Grid-Based Map Analysis Data Mining investigates the “numerical” relationships in mapped data… Descriptive — aggregate statistics (e.g., average/stdev, similarity, clustering) Descriptive — aggregate statistics (e.g., average/stdev, similarity, clustering) Predictive — relationships among maps (e.g., regression) Predictive — relationships among maps (e.g., regression) Prescriptive — appropriate actions (e.g., optimization) Prescriptive — appropriate actions (e.g., optimization) (Berry) Spatial analysis investigates the “contextual” relationships in mapped data… Reclassify — reassigning map values (position; value; size, shape; contiguity) Reclassify — reassigning map values (position; value; size, shape; contiguity) Overlay — map overlay (point-by-point; region-wide; map-wide) Overlay — map overlay (point-by-point; region-wide; map-wide) Distance — proximity and connectivity (movement; optimal paths; visibility) Distance — proximity and connectivity (movement; optimal paths; visibility) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) Surface modeling maps the “spatial distribution” and pattern of point data… Map Generalization — characterizes spatial trends (e.g., titled plane) Map Generalization — characterizes spatial trends (e.g., titled plane) Spatial Interpolation — deriving spatial distributions (e.g., IDW, Krig) Spatial Interpolation — deriving spatial distributions (e.g., IDW, Krig) Other — roving window/facets (e.g., density surface; tessellation) Other — roving window/facets (e.g., density surface; tessellation) Spatial Statistics

Visualizing Spatial Relationships (Berry) What spatial relationships do you see? Interpolated Spatial Distribution Phosphorous (P) …do relatively high levels of P often occur with high levels of K and N? …how often? …where?

Identifying Unusually High Measurements …isolate areas with mean + 1 StDev (tail of normal curve) (Berry)

Level Slicing …simply multiply the two maps to identify joint coincidence 1*1=1 coincidence (any 0 results in zero) (Berry)

Multivariate Data Space …sum of a binary progression (1, 2,4 8, 16, etc.) provides level slice solutions for many map layers (Berry)

Calculating Data Distance …an n-dimensional plot depicts the multivariate distribution— the distance between points determines the relative similarity in data patterns …the closest floating ball is the least similar (largest data distance) from the comparison point (Berry)

Identifying Map Similarity (Berry) …the green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar areas …the relative data distance between the comparison point’s data pattern and those of all other map locations form a Similarity Index

Clustering Maps for Data Zones …groups of “floating balls” in data space identify locations in the field with similar data patterns– data zones …a map stack is a spatially organized set of numbers (Berry) …fertilization rates vary for the different clusters “on-the-fly” (Cyber-Farmer, Circa 1990) Variable Rate Application

Evaluating Clustering Results (Berry) …graphical and statistics procedures assess how “distinct” clusters are— Clustering Performance …distinct in K and N (less), but not distinct in P

Spatial Data Mining (Full Exercise #7) (Berry) Similarity Map Cluster Map Regional Average Composite Scatter Plot Univariate Regression Multivariate Regression Spatial statistics … Spatial statistics … use MapCalc to implement derive relationships among P, K, N and Yield in a farmer’s field MapCalc (Short Exercise #7) DescriptivePrescriptive

The Precision Ag Process (Fertility example) As a combine moves through a field it 1) uses GPS to check its location then 2) checks the yield at that location to 3) create a continuous map of the 2) checks the yield at that location to 3) create a continuous map of the yield variation every few feet. This map is yield variation every few feet. This map is 4) combined with soil, terrain and other maps to 4) combined with soil, terrain and other maps to derive 5) a “Prescription Map” that is used to derive 5) a “Prescription Map” that is used to 6) adjust fertilization levels every few feet 6) adjust fertilization levels every few feet in the field (variable rate application). in the field (variable rate application). Farm dB Step 4) Map Analysis On-the-Fly Yield Map Steps 1) – 3) Prescription Map Step 5) Zone 1 Zone 3 Zone 2 Step 6) Variable Rate Application Cyber-Farmer, Circa 1992 (Berry)

Spatial Data Mining Precision Farming is just one example of applying spatial statistics and data mining techniques (Berry) Mapped data that exhibits high spatial dependency create strong prediction functions. As in traditional statistical analysis, spatial relationships can be used to predict outcomes… …the difference is that spatial statistics predicts where responses will be high or low Geo-business SDM

Continuous Spatial Distribution (Detailed) Map Analysis Spatially Interpolated data (Geographic Space — Spatial Statistics) Data Analysis Perspectives (Review) Identifies the Central Tendency Maps the Variance Central Tendency Average = 22.0 StDev = 18.7 Typical How Typical Discrete Spatial Object (Generalized) Traditional Analysis Field Data Standard Normal Curve fit to the data (Data Space — Non-spatial Statistics) (Berry)

Grid-Based Map Analysis (Review) Data Mining investigates the “numerical” relationships in mapped data… Descriptive — aggregate statistics (e.g., average/stdev, similarity) Descriptive — aggregate statistics (e.g., average/stdev, similarity) Predictive — relationships among maps (e.g., regression) Predictive — relationships among maps (e.g., regression) Prescriptive — appropriate actions (e.g., optimization) Prescriptive — appropriate actions (e.g., optimization) Surface Modeling maps the “spatial distribution” and pattern of point data… Map Generalization — characterizes spatial trends (e.g., titled plane) Map Generalization — characterizes spatial trends (e.g., titled plane) Spatial Interpolation — deriving spatial distribution (e.g., IDW, Krig) Spatial Interpolation — deriving spatial distribution (e.g., IDW, Krig) Other — roving window (e.g., density surface; tessellation) Other — roving window (e.g., density surface; tessellation) Spatial Analysis investigates the “contextual” relationships in mapped data… Reclassify — reassigning map values (position; value; size, shape; contiguity) Reclassify — reassigning map values (position; value; size, shape; contiguity) Overlay — map overlay (point-by-point; region-wide; map-wide) Overlay — map overlay (point-by-point; region-wide; map-wide) Distance — proximity and connectivity (movement; optimal paths; visibility) Distance — proximity and connectivity (movement; optimal paths; visibility) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) Neighbors — ”roving windows” (slope/aspect; diversity; anomaly) (Berry) Spatial Analysis Spatial Statistics

More Map Analysis Experience ( MapCalc & Surfer) (Berry) NEW BOOK — seethe description of the Map Analysisbook (Berry, 2007; GeoTec Media)at… NEW BOOK — see the description of the Map Analysis book (Berry, 2007; GeoTec Media) at… …develops a structured view of the important concepts, considerations and procedures involved in grid-based map analysis. …the companion CD contains further readings and software for hands-on experience with the material presented. Complete Experience See Default.htm Workshop CD Surfer Tutorial MapCalc Tutorial Tutorial Exercises Workshop Exercises Short & Full Exercise Sets

…but before we leave Spatial Statistics (Surface Modeling and Spatial Data Mining—descriptive, predictive and prescriptive) to tackle GIS Modeling, any… Questions? Questions? (Berry)