1. 1 Las Vegas G. P. Patil

2. 2 This report is very disappointing. What kind of software are you using?

3. 3 Space Age and Stone Age Syndrome Data:Space Age/Stone Age Analysis:Space Age/Stone Age DataSpace AgeStone Age Analysis Space Age++ Stone Age+

4. 4

5. 5 Sampling and Modeling for Accuracy Assessment Nested Area Sampling Frame Design Accuracy Assessment Bivariate Raster Map Modeling Conditional Simulation for Accuracy Assessment

6. 6 Sampling for Overall Accuracy Assessment Thematic Accuracy Full Error Matrix (category by category) Overall Accuracy (correct or incorrect) Binary Map (0: correct; 1: incorrect) Parameter p = proportion incorrect Sampling for Estimation of p

7. 7 Go Where They Are Neyman Coefficient of Variation Stratification Disproportionate Sampling

8. 8 Thematic Accuracy Assessment Sampling Designs for Reference Data Spatially uniform vs. spatially focused sampling intensity Multi-stage nested area sampling frame designs (usually spatially uniform) Adaptive sampling designs (spatially focused with no prior information) Disproportionate sampling designs (spatially focused with prior information, e.g., near patch boundaries, in particular patch-types, etc.)

9. 9 Where Are They? No prior information available Two stage design Stage 1: Find out where they are Stage 2: Intensive localized sampling

10. 10

11. 11 Two Stage Adaptive Sampling Design Region is partitioned into A equal size cells One binary observation x is made from each cell (Stage 1) If x=0, no second stage sampling in the cell If x>0, n 2 additional binary observations at random in the cell

12. 12 Estimation of p

13. 13 Best Unbiased Estimation

14. 14 A Startling Result

15. 15 Go Where They Are Any Contradiction? Instead of a binary map –Leave the 0s alone –Replace the 1s by iid observations on a positive random variable Y with mean μ and variance σ 2 The mean response across the region is pμ Use the same two-stage sampling design to estimate this mean response Examples: contamination field; animal abundance Binary map results when μ=1 and σ 2 → 0

16. 16 Performance of the Two Stage Design for Continuous Responses i 1 = information per observation in Stage 1 i 2 = information per observation in Stage 2 Relative cost effectiveness is Large e means second stage beneficial

17. 17 Behavior of Relative Efficiency e depends upon two features of the map –Presence-absence spatial pattern: {p a : a=1,2,…,A} –Continuous response CV: CV = σ/μ e→0 as CV→0 (binary map in the limit) But e large as soon as CV positive (traditional adaptive sampling context)

18. 18

19. 19 Larger First Stage Sample First-stage sample size: n 1 =2 Decision rule for second sage sampling: x > 0 Second-stage sample size: n Unbiased variance estimation possible e ≠ 0 but e ≤ 1 More cost effective to increase the number of first stage grid cells, rather than do two-stage sampling

20. 20 Relative Cost Effectiveness Per Cell (n = second-stage sample size)

21. 21 Further Possible Generalizations Increase n 1 Change the decision rule to trigger second- stage sampling However, for other decision rules, the sufficient statistic may not be complete

22. 22 An Interesting Decision Rule Let r be a fixed integer with 0 ≤ r ≤ n 1 Let x be the number of 1s in the first stage sample Do second-stage sampling if and only if x = r The Rao-Blackwellized estimator turns out to be exactly the same as the first-stage sample estimator Zero information in the second-stage observations for this decision rule.

23. 23 Landscape Pattern Extraction Spectral data Empirical extraction Thematic data Empirical extraction Spectral data Model-based extraction Thematic data Model-based extraction

24. 24

25. 25 Model-based Pattern Extraction Pattern = Spatial variability in thematic maps Proposed research limited to raster maps Possible Parametric Models: –Geostatistics (Multi-indicator) –Markov Random Fields –Hierarchical Markov Transition Matrix models (HMTM)

26. Geostatistical Models

27. Disjunctive Indicator Geostatistical (DIG) Model Motivation Overcomes limitations of multiple indicator approach, i.e., Marginal analyses (one category at a time) are too limiting Modeling and fitting of cross-variograms (for joint analyses) is difficult Simulation requires probability rather than moment model

28. DIG Model Model Description-1 Z Latent Surface gets green gets red lattice points

29. DIG Model Model Description-2 Grid with lattice points t Standard normal gaussian process Z(t) on the grid with correlation function  (h) Partition A1, A2, …, Ak of Z-axis Replace Z(t) by the disjunctive indicators of A1, A2, …, Ak, which determine a unique category (color code) for grid point t

30. DIG Model Accuracy Assessment Two overlaid maps –t: true/reference values/categories –d: declared categories Same latent surface Z(s), same correlogram  (h) Separate overlaid transitionograms

31. 31 DIG Model Accuracy Assessment Red gets Red Yellow gets Red Z(s)Z(s) dt Compiling t, d matches/mismatches in overlaid transitionograms gives model-predicted error/confusion matrix.

32. HMTM Model Employs a duality between Spatial Transitions in the raster map and Hierarchical Transitions in the model

33. Spatial Transitions Scan map to obtain a matrix of transition frequencies between pixels a fixed distance apart (horiz. or vert.)

34. Auto-Association Matrices Analogue of variogram for categorical responses Symmetric, k x k where k = number of mapping categories One auto-association matrix for each distance scale : adjacent pixels 2 pixels apart 4 pixels apart

35. HMTM Model Hierarchical Transitions Model generates a hierarchical sequence of raster maps, all having the same spatial extent Hierarchical Level 0 Hierarchical Level 1 Hierarchical Level 2

36. Assigning Categories to Pixels Assignment at coarsest scale is a random draw from the marginal land-cover distribution:

37. Assigning Categories to Pixels Assignment at finer scales is via k by k row stochastic matrices G i Mother cell4 daughter cells The transition is determined by 4 draws from the ith row of G:

38. HMTM as (hidden) MRF - 2 Neighborhood of each node: mother and 4 daughters Neighbors of node s are: s 0 and s 1, s 2, s 3, s 4 s0s0 s s 1 s 2 s 3 s 4

39. HMTM as (hidden) MRF - 1 Floor resolution pixels in HMTM model are leaf nodes in quadtree MRF on entire quadtree. All but leaf nodes are hidden from observation Benefit of MRF approach: Conditional simulation via MCMC---no obvious way to do conditional simulation directly in HMTM Conditional simulation especially needed for thematic accuracy assessment

40. 40 Thematic Accuracy Assessment Model-based Error Maps Thematic raster data consists of declared category (d) on each grid cell and reference (“true”) categories (t) on selected cells Model for joint (d,t) map is fitted. Repeated conditional simulation of fitted model yields a simulated probability distribution for each missing t-value. Models under consideration: MIG, MRF, HMTM

41. 41 Thematic Accuracy Assessment Model-based Error Maps Issues to be addressed: Model specification. The (d,t) map has a large number of categories and correspondingly many model parameters. Plausible parametric dimension reduction desirable. Model fitting issues. –How to impose reduced parameter space restrictions ? One possibility is to minimize a criterion function. In case of HMTM model, minimize discrepancy between observed and expected 4-tuple frequency tables measured by, e.g., Kullback-Liebler distance or chi- square distance.

42. 42 Thematic Accuracy Assessment Model-based Error Maps Issues to be addressed: Model fitting issues (cont’d). –What to do about excessive missing data arising from the sparsity of reference data (t-values). Conditional simulation. The simulation should reproduce any known d and t values but fill in any missing t values. How to do this for each of the three models ? Reference data. Probability vs. non-probability sample. Reuse of training data vs. independent reference data.

43. 43 An Example of a Single Date Error Matrix

44. 44 An Example of a Change Detection Error Matrix