Analyzing Precision Ag Data : Intermediate workshop on what is needed to move Precision Agriculture beyond mapping Joseph K. Berry W. M. Keck Visiting Scholar in Geosciences, Geography, University of Denver Principal, Berry & Associates // Spatial Information Systems — Web Geographic Information and Spatial Technologies Workshop Ag Canada — October, 2006 — Winnipeg, Manitoba, Canada

Map Data Visualization and Summary (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps Overview …keynote presentation

Map Data Visualization and Summary (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

MAP Analysis Framework (Berry) Continuous, regular grid cells (objects) : --, --, --, --, --, 1944, --, --, --, --, --, :GridTable Click on… Zoom Pan Rotate DisplayShadingManager Now for some Map Analysis… GridAnalysis …calculate a slope map and drape on the elevation surface

Grid-based Map Data Visualization (Berry) Thematic Mapping Display and Data Types Lattice versus Grid 2D Map versus 3D Surface Discrete (Qualitative) versus Continuous (Quantitative) Continuous (Quantitative)

Investigating and Normalizing Mapped Data (Berry) GridMath equation Norm_GOAL = (mapValue / 250) * 100 Data Vales (matrix) Drill down

Comparing Map Data (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

Visually Comparing Mapped Data (Berry) What differences do you see? “How different are the maps?” “How are they different?” “Where are they different?” MUST have a common legend Class 5 Class 4 Class 3 Class 2 Class 1 DISCRETE integer values 1-5 CONTINUOUS ratio values 2.33 – Class Class Class Class Class 1 …you don’t “see” the data values– just the COLORS

Quantitative Comparison Align grid maps and count Class Class Class Class Class 1 Comparing Discrete Maps (Joint coincidence) (Berry) …a Coincidence Table reports the number of cells for each joint condition with diagonal cells identifying agreement (no change) = 1850/3289= overall agreement

Comparing Continuous Surfaces (Difference map) 1997_Yield_Volume 1997_Yield_Volume _Yield_Volume Yield_Diff Map Variables … map values within an analysis grid can be mathematically and statistically analyzed (Berry) …green indicates areas of increased production …yellow indicates minimal change …red indicates decreased production

Spatial Interpolation (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

The geo-registered soil samples form a pattern of “spikes” throughout the field. Spatial Interpolation is similar to throwing a blanket over the spikes that conforms to the pattern. Spatial Interpolation (Mapping spatial variability) (Berry) All interpolation algorithms assume that— 1) “nearby things are more alike than distant things” (spatial autocorrelation), 2) appropriate sampling intensity, and 3) suitable sampling pattern. …the continuous surfaces form a “map” of the spatial variation in the data samples.

Spatial Interpolation (Average vs. IDW) Comparison of the interpolated surface to the whole field average shows large differences in localized estimates Difference Map (Berry)

Spatial Interpolation (Compare maps) Comparison of the IDW and Krig interpolated surfaces shows small differences differences in in localized estimates Difference Map (Berry)

Spatial Interpolation Techniques (Berry) Characterizes the spatial distribution by fitting a mathematical equation to localized portions of the data (roving window) AVG= 23 everywhere Spatial Interpolation techniques use “roving windows” to summarize sample values within a specified reach of each map location. Window shape/size and summary technique result in different interpolation surfaces for a given set of field data …no single techniques is best for all data. Inverse Distance Weighted (IDW) technique weights the samples such that values farther away contribute less to the average …1/Distance Power

Assessing Spatial Interpolation Results Residual Analysis …the best map is the one that has the best “guesses” (Berry)

A Map of Error (Residual Map) …shows you where your estimates are likely good/bad (Berry)

Spatial Interpolation Techniques (Berry) A variogram depicts the relationship between the distance between sample points and the difference between the measurement values All interpolation algorithms assume that— 1) “nearby things are more alike than distant things” (spatial autocorrelation), 2) appropriate sampling intensity, and 3) suitable sampling pattern.

Basic Point Sampling Design Concerns Stratification— appropriate groupings for sampling Sample Size— appropriate sampling intensity for each stratified group (N) Sampling Grid— appropriate analysis grid for locating point samples (Berry) All interpolation algorithms assume that— 1) “nearby things are more alike than distant things” (spatial autocorrelation), 2) appropriate sampling intensity, and 3) suitable sampling pattern.

Point Sampling Design Concerns (Sampling Pattern) Sampling Pattern— appropriate arrangement of samples considering both spatial interpolation and statistical inference (Berry) All interpolation algorithms assume that— 1) “nearby things are more alike than distant things” (spatial autocorrelation), 2) appropriate sampling intensity, and 3) suitable sampling pattern.

Characterizing Data Groups (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

Visualizing Spatial Relationships (Berry) What spatial relationships do you see? Interpolated Spatial Distribution Phosphorous (P) …do relatively high levels of P often occur with high levels of K and N? …how often? …where?

Identifying Unusual Areas …locations that are more than one standard deviation above the mean are identified as unusually high (Berry)

Calculating Data Distance …an n-dimensional plot depicts the multivariate distribution; the distance between points determines the relative similarity in data patterns (Berry) …the closest floating ball is the least similar (largest data distance) from the comparison point

Identifying Map Similarity (Berry) The green tones indicate field locations with fairly similar P, K and N levels; red tones indicate dissimilar areas. …the relative data distance between the comparison point’s data pattern and those of all other map locations form a Similarity Index

Clustering Maps for Data Zones (Berry) (Cyber-Farmer, Circa 1992) Variable Rate Application …fertilization rates vary for the different clusters “on-the-fly” …groups of “floating balls” in data space identify locations in the field with similar data patterns– data zones …a map stack is a spatially organized set of numbers

Evaluating Clustering Results (Berry) …if the boxes do not overlap (much), the data clusters are distinct

Developing Predictive Models (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

NIR (R) Red (G) Green (B) (Beyond our sight) Color Infrared P K ph etc. RS Imagery as GIS Data Layers Remote sensing images are composed of numbers, numbers, just like any other map in a grid-based GIS… “Map-ematical Processing” A RS image is just a “shishkebab of numbers” like any other of numbers” like any other grid map (raster) grid map (raster) Image (Berry)

Creating Prediction Models (Scatter Plot) …a …a Scatter Plot identifies the “joint condition” at each map location; the trend in the plot forms a prediction equation (Berry) Map Set  New Graph  Scatter Plot

Deriving a Predictive Index (NDVI) …an index combining the Red and NIR maps can be used to generate a better predictive model Normalized Difference Vegetation Index NDVI= ((NIR – Red) / (NIR + Red)) for the sample grid location NDVI= (( ) / ( ))= / 135.7=.783 (Berry)

Evaluating Prediction Maps (Spatial error analysis) …the regression equation is evaluated and the predicted map is compared to the actual measurements to generate an error map Error = Predicted - Actual for the sample grid location Y est = 55 + (180 *.783) = 196 …error is 196 – 218 = 22 bu/ac Note that the average error is 2.62 and 67% of the predictions are within +/- 20 bu/ac Also, most of the error is concentrated along the edge of the field (Berry) Error = Predicted - Actual

Stratifying Maps for Better Predictions (Berry) Stratifying by Error Zones Other ways to stratify mapped data— 1) Geographic Zones, such as proximity to the field edge; 2) Dependent Map Zones, such as areas of low, medium and high yield; 3) Data Zones, such as areas of similar soil nutrient levels; and 4) Correlated Map Zones, such as micro terrain features identifying small ridges and depressions. The Error Zones map is used as a template to identify the NDVI and Yield values used to calculate three separate prediction equations. A Composite Prediction map is created by applying the equations to the NDVI data respecting the template map zones.

Assessing Prediction Results (Berry) Whole Field Prediction Stratified Prediction Actual Yield none Error Map for Stratified Prediction 80% Error Map

The Precision Ag Process (Fertility example) (Berry) As a combine moves through a field 1) it uses GPS to check its location then 2) checks the yield at that location to 3) create a continuous map of the yield variation every few feet (dependent map variable). On-the-Fly Yield Map Steps 1)–3) Variable Rate Application Step 5) 5) …that is used to adjust fertilization levels every few feet in the field (action). Intelligent Implements Derived Nutrient Maps Step 4) Prescription Map Zone 3 Zone 2 Zone 1 The yield map 4) is analyzed in combination with soil, terrain and other maps (independent map variables) to derive a “Prescription Map” …

Analyzing Spatial Context (Berry) Workshop Topics Overview (keynote) Mapped Data Visualization and Summary Comparing Mapped Data Spatial Interpolation Characterizing Data Groups Developing Predictive Models Generating Prescription Maps

Generating Prescription Maps (Step & Equations) (after Elaine McCallum, Red hen Farming) RENUMBER 1996_Fall_P ASSIGNING 50 TO 0 THRU 4 ASSIGNING 30 TO 4 THRU 8 ASSIGNING 15 TO 8 THRU 12 ASSIGNING 0 TO 12 THRU 500 FOR P_application Prescription maps of P and K are based on decision rules Prescription Mapping If then If then …this is where science and technology meet Prescription map for N is based on equation

Economic Maps (after Elaine McCallum, Red hen Farming) A total costs map is derived by summing the variable and fixed cost maps. Calculate ( (P_lbs_per_cell *.697) / (1997_Yield_Volume * 2.75) ) * 100 For Revenue_%P

On-Farm Testing (site-specific studies/research) (Berry) On-farm studies, such as seed hybrid performance, can be conducted using actual farm conditions.

Is GIS Technology Ahead of Science? (Berry) 1) Is the “scientific method” relevant in the data-rich age of knowledge engineering? 2) Is the “random thing” pertinent in deriving mapped data? 3) Are geographic distributions a natural extension of numerical distributions? 4) Can spatial dependencies be modeled? 5) How can “site-specific” analysis contribute to the scientific body of knowledge? …the bottom line is that modern maps are numbers first, pictures later Five critical questions underlying Precision Agriculture…

Where To Go From Here… Who’s Minding the Farm, GeoWorld, Adams Business Media, Chicago, Illinois, Feb 1998, 11: J.K. Berry. Applying Spatial Analysis for Precision Conservation Across the Landscape, J. of Soil and Water Conservation, Nov/Dec 2005, 60: J.K. Berry, J. A. Delgado, R. Khosla and F.J. Pierce. Precision Conservation for Environmental Sustainability, J. of Soil and Water Conservation, Nov/Dec 2003, 58: J.K. Berry, J. A. Delgado, R. Khosla and F.J. Pierce. Analyzing Precision Ag Data : book published by BASIS Press, Fort Collins, Colorado, 2003, 84 pages, 49 illustrations with exercises and companion software. J.K. Berry. Companion Software Analyzing Precision Ag Data : workshop this afternoon 1:30-4:30pm and repeated tomorrow morning 8:45-11:45am __________________________________