Variance of Similar Neighbors compared to Random Imputation Nearest Neighbor Conference August 28-30, 2006 Kenneth B. Pierce Jr and Janet L. Ohmann Forestry Sciences Lab, PNW Research Station, Corvallis

Project Objectives Map fuels and vegetation using Gradient Nearest Neighbor (GNN) imputationMap fuels and vegetation using Gradient Nearest Neighbor (GNN) imputation Produce maps of plot-level tree attributes as complete coveragesProduce maps of plot-level tree attributes as complete coverages Provide a high degree of analytical flexibility for end-usersProvide a high degree of analytical flexibility for end-users Provide robust accuracy assessmentProvide robust accuracy assessment California Sierra (Mediterranean) Coastal Oregon (Maritime) Eastern Washington (Temperate steppe)

Presentation Objectives Give an brief overview of Gradient Nearest Neighbor (GNN) imputation as a techniqueGive an brief overview of Gradient Nearest Neighbor (GNN) imputation as a technique Describe the use of imputation for mapping natural variabilityDescribe the use of imputation for mapping natural variability Describe the use of imputation for mapping sampling sufficiencyDescribe the use of imputation for mapping sampling sufficiency Examine the variability among nearest neighbors in gradient space versus a random set of neighborsExamine the variability among nearest neighbors in gradient space versus a random set of neighbors Examine the change in variability when restricting plot selection to those well represented in gradient spaceExamine the change in variability when restricting plot selection to those well represented in gradient space

Major Steps in GNN Imputation mapping: 1) Assembling Data1) Assembling Data 2) Statistical Modeling (CCA)2) Statistical Modeling (CCA) 3) Imputation/Map Creation3) Imputation/Map Creation 4) Accuracy Assessment4) Accuracy Assessment 5) Applications and Risk Assessment5) Applications and Risk Assessment

Statistical Modeling: Canonical Correspondence Analysis Multivariate statistical methodMultivariate statistical method –results in a weight for each spatial variable as to its relationship with the multiple response variables Modeling Variables-used as model Y’sModeling Variables-used as model Y’s –Structure models (BAC, BAH, STPH, CWD) –Species models Mapping Variables-retained with plot-map linkMapping Variables-retained with plot-map link

Neighbors in Gradient Space Direct gradient analysis allows assignment of a multi- dimensional location to each predicted pixelDirect gradient analysis allows assignment of a multi- dimensional location to each predicted pixel

A Pixel in Plotland (example 0.5 * elevation * precip)

A Pixel in Plotland Sample plot locations in gradient space (example 0.5 * elevation * precip)

A Pixel in Plotland Target Location in Gradient Space Sample plot locations in gradient space (example 0.5 * elevation * precip)

A Pixel in Plotland Five closest neighbors (example 0.5 * elevation * precip)

A Pixel in Plotland Twenty closest neighbors (example 0.5 * elevation * precip)

A Pixel in Plotland Interplot Distances (example 0.5 * elevation * precip)

How far is far in gradient space?

Major Steps in GNN mapping: 1) Data Preparation/Screening1) Data Preparation/Screening 2) Statistical Modeling2) Statistical Modeling 3) Imputation/Map Creation3) Imputation/Map Creation 4) Accuracy Assessment4) Accuracy Assessment 5) Applications and Risk Assessment5) Applications and Risk Assessment

Imputing/Assigning plot id’s Nearest neighbor (single neighbor, retains covariance, MSN-like)Nearest neighbor (single neighbor, retains covariance, MSN-like) Summary statistic of multiple neighbors (single value, kNN-like)Summary statistic of multiple neighbors (single value, kNN-like) Etc. (i.e. many other contortions possible)Etc. (i.e. many other contortions possible)

Process Uncertainty/Natural VariabilityProcess Uncertainty/Natural Variability –Uncontrollable (often unmeasurable) Natural disturbances Demographic stochasticity Anthropogenic disturbances Sampling UncertaintySampling Uncertainty –Not entirely uncontrollable Limited sampling Spatial averaging Temporal sample variation Sources of Uncertainty For Ecological Detectives Hilborn & Mangel 1997

Map integral (Value of Map)Map integral (Value of Map) –Confusion matrices/Kappa (local) –Correlation statistics (local) –Regional histograms (regional) Map explicit (Map of Values)Map explicit (Map of Values) –Confidence maps (Process) –Support (Sampling) Accuracy assessments “obsessive transparency”

Overview of maps Vegetation mapVegetation map –the predicted value Neighbor Count mapNeighbor Count map –a measure of sampling sufficiency for a specific ecological location Natural Variability mapNatural Variability map –the variability in response at the most similar locations

Natural variability maps Variability maps are created by calculating the variance for the 5 nearest neighbors at each location (a value other than 5 could certainly be used)Variability maps are created by calculating the variance for the 5 nearest neighbors at each location (a value other than 5 could certainly be used)

Sampling sufficiency maps Centile thresholds are selected from the histogram of interplot distancesCentile thresholds are selected from the histogram of interplot distances Gradient distance grids are retained for the 20 nearest neighbors during imputationGradient distance grids are retained for the 20 nearest neighbors during imputation The 20 distance grids are compared to the threshold values and a count grid is created where a value of 20 indicates 20 plots were within the threshold valueThe 20 distance grids are compared to the threshold values and a count grid is created where a value of 20 indicates 20 plots were within the threshold value

4m Aerial Photo

Expected value Basal Area m 2 /ha 0 61

10 th Quantile Threshold map 0 20 Neighbors out of 20 within the threshold distance

20th Quantile Threshold map 0 20 Neighbors out of 20 within the threshold distance

50th Quantile Threshold map 0 20 Neighbors out of 20 within the threshold distance

Natural Variability Standard deviation of 5 nearest neighbors for BA (m2/ha)

“Premise of Imputation” TheoremTheorem –Places similar in X-values should be similar in Y-values. PostulatePostulate –The 5 plots most similar to a location in X-values should have reduced variance in Y-values compared to 5 random plots

Methods 1.Create 1000 random spatial locations 2.Sample the plot ids from the 5 nearest neighbors and the 10-th and 20-th centile sufficiency grids 3.Select an attribute and query the plot data with the five nearest neighbor ids 4.Calculate the variance for the five nearest neighbors at each of the 1000 sample points 5.Plot the density of the variance values (Black line)

data Random sets of 5 values

Methods continued Create 1000 sets of 5 random plots and repeat the variance calculation and density plot (Open circles)Create 1000 sets of 5 random plots and repeat the variance calculation and density plot (Open circles) Subset the random locations and plot data sets into groups based on their sufficiency scores: 0, 5, >15 [# of 20 nearest neighbors w/in the threshold value]Subset the random locations and plot data sets into groups based on their sufficiency scores: 0, 5, >15 [# of 20 nearest neighbors w/in the threshold value] Plot densities by subgroupsPlot densities by subgroups Create and plot random sets from appropriate subgroupsCreate and plot random sets from appropriate subgroups

data Random sets of 5 values Bootstrap set All imputed Neighbors >=15 Neighbors >5 Neighbors <15 Neighbors <=5

Bootstrap set All imputed Neighbors >=15 Neighbors >5 Neighbors <15 Neighbors <=5

Is this a general result? Sorta.Sorta.

Conclusions