1. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 1 Geographic Information Processing Spatial Interpolation Freed, et al, Magnetometer field readings of single dipole.

2. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 2 Wide Area Attribute Data Attributes of data are measured over a wide area at regular grid points –elevations, area surveys, etc. Attributes are needed at locations other than where measured or to fill data gaps Processing of data requires regular data grid –Filtering, smoothing, feature extraction, etc.

3. Sparsely Measured Data Attributes of data are widely distributed but sparsely measured –Weather gauges (rainfall, temperature, wind velocity, etc.) –Mineralization (lab analysis) Attributes are needed at locations other than where measured –Weather, geological, etc. attributes 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 3

4. Intensively Measured Data Attributes of data are widely distributed but intensively measured limited areas –Area surveys, location of buried objects, etc. Attribute spatial distribution model needed covering locations other than where measured Model will be processed to draw conclusions about attribute field 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 4

5. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 5 General Problems Attribute estimates may be needed at points OTHER than those for which measurements are available Grid of data may be needed for processing Model parameters may be needed to fit data Noise and outliers in data may need to be removed

6. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 6 Spatial Data Processing Interpolation Extrapolation Resampling Smoothing Filtering Gradient calculation Parameter estimation (model fitting)

7. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 7 Spatial Interpolation Estimating attributes at points between those where attributes have been measured Types: –Linear –Polynomial –Splines (a succession of cubic curves) –Filtering (typically Gaussian) –Use of physical model

8. Piecewise Linear Interpolation Known points: {x 1,y 1 }, {x 3,y 3 }, x 2 Determine: {x 2,y 2 } Method: Compute slope: m = (y 3 -y 1 )/(x 3 -x 1 ) Compute new point: y 2 = y 1 + m*x 2 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 8

9. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 9 Simple Interpolation Method Fit polynomial model to known data points Apply model to other points Simple Example: Model:f(x) = a + b x Data:f(5) = 1, f(6) = 3 Calculation: 1 = a + 5 b, 7 = a + 6 b a = -9, b = 2 Interpolation: f(7) = -9 + 2*7 = 5

10. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 10 Two-Dimensional Simple Model Model: z(x,y) = a 0 + a 1 x + a 2 y Data:

11. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 11 Least Squares Modeling We want to find the line, Which minimizes the squared estimation error at n measured (control) points, given by,

12. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 12 Solution It is well known that the unknown coefficients a and b are the solution of the simultaneous equation set:

13. Smoothing Use of smooth functions (with continuous derivatives) to fit, and thus to estimate, data –Example: Polynomial curves Cubic splines Use of low-pass digital filters to reduce noise associated with data 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 13

14. Polynomial Interpolation Given data points: {x 1,y 1 }... {x n,y n } Determine coefficients, a i, such that, y(x) = a 0 + a 1 *x 1 +a 2 *x 2 +... +a n *x n fits the data points as closely as possible (minimum sum of squared errors, for instance) Note: Polynomials for interpolation do not work well for extrapolation 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 14

15. Example: Terrain Cross-Section A terrain cross-section has been captured as points in a data file. It is desired to make a polynomial fit to the terrain height points, making a smooth curve. The cross-section points appear on the figure to follow. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 15

16. Terrain Cross-Section Pixels 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 16

17. Note Distances to the terrain from above ground level cannot be extracted because the pixel density is too low Normal line to the terrain might not intersect a terrain point Smooth curve interpolation needed 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 17

18. 30 th -Order Polynomial Fit y = 247.219 - 599.194x + 7554.89 x^2 - 53551.8 x^3 + 220798. x^4 - 575104. x^5 + 998935. x^6 - 1.19577*10^6 x^7 + 1.00308*10^6 x^8 - 589715. x^9 + 237018. x^10 - 60051.4 x^11 + 6953.51 x^12 + 676.484 x^13 - 282.826 x^14 – 2.39785 x^15 + 8.66168 x^16 + 0.0551704 x^17 - 0.264777 x^18 - 0.0111781 x^19 + 0.00773041 x^20 + 0.000778205 x^21 - 0.00020054 x^22 - 0.0000367033 x^23 + 4.73382*10^-6 x^24 + 1.40748*10^-6 x^25 - 1.46579*10^-7 x^26 - 4.64781*10^-8 x^27 + 1.06387*10^-8 x^28 - 8.36776*10^-10 x^29 + 2.41143*10^-11 x^30 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 18

19. Polynomial Fit to Terrain Pixels 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 19

20. Comments: Polynomial curve deviates from likely terrain Degree of polynomial would have to be raised significantly to get better fit Extrapolation at ends would be poor 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 20

21. Example: Magnetic Field Readings Readings from a fixed magnetometer will be used to remove time variations of the magnetic field of the earth from field survey measurements to locate buried objects The survey and fixed instruments are not synchronized to each other Interpolation of fixed measurements to survey measurement times will be needed 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 21

22. Magnetic Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 22

23. Preliminary Evaluation of Data Data set contains a few spurious data points that need to be removed Measurements contain quite a bit of noise, some of which should be removed by digital filtering 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 23

24. Filtered Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 24

25. Comments: Outlier points have been reduced in amplitude to within acceptable limits Random noise has been greatly reduced Points shown are still pixels, and interpolation will be needed to estimate magnetic field at survey time instants. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 25

26. 60 th -Order Polynomial Fit poly1 = Fit [outbl, {1,x, x^2, x^3, x^4, x^5, x^6, x^7, x^8, x^9, x^10, x^11, x^12, x^13, x^14, x^15, x^16, x^17, x^18, x^19, x^20, x^21, x^22, x^23, x^24, x^25, x^26, x^27, x^28, x^29, x^30, x^31, x^32, x^33, x^34, x^35, x^36, x^37, x^38, x^39, x^40, x^41, x^42, x^43, x^44, x^45, x^46, x^47, x^48, x^49, x^50, x^51, x^52, x^53, x^54, x^55, x^56, x^57, x^58, x^59},x]; 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 26

27. Polynomial Fit to Magnetic Data 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 27

28. Comments: 60th-order polynomial was needed to get an adequate fit to the data Polynomial interpolating function deviates unacceptably from magnetic field data after 3.5 hours Polynomial fit effectively removed spurious outlier points 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 28

29. Cubic Spline Interpolation Uses multiple cubic equations of the form, y1 = a 0 + a 1 *x 1 + a 2 *x 2 + a 3 *x 3 Cubic equations are generated for each set of three successive points Derivatives are continuous (the cubic equations are each twice differentiable) 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 29

30. Terrain Cross-Section Pixels 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 30

31. Spline Fit to Terrain Points 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 31

32. Magnetic Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 32

33. Filtered Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 33

34. Spline Fit to Magnetic Data 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 34

35. Comments: Spline curve fits measured data over entire epoch Spline curve does not remove artifact of spurious data points A more “relaxed” spline function, or further bandwidth reduction by digital filtering, would remove the evident short-term fluctuations. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 35

36. Extrapolation Estimating data attributes outside the field over which they have been measured Can be used to extend measured data to edge points 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 36

37. Digital Convolution Filters c = {1., 4., 8., 10., 8., 4., 1.}; css = 1.0*Total[c]; cn = c/css; filtbl = ListConvolve[cn, fieldtbl]; Produces low-pass filtered version of data in Table: fieldtbl 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 37

38. Unfiltered Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 38

39. Filtered Field Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 39

40. Resampling Changing the spatial sampling rate or origin to conform to a regular (periodic) “grid” from data gathered at other locations or intervals (spatial frequencies) Used for reducing the number of points in a data field or changing their locations 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 40

41. Example – Magnetic Measurements Magnetometer readings are taken periodically during traversals over a field of measurement. Data points are gathered intensively, every 0.2 seconds), but there are areas with no measurements. Course grid squares will have many measurements in them A fine grid wil have many squares containing no directly measured data points 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 41

42. Field and Measurements 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 42

43. Coarse Measurement Grid 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 43

44. Averaged Field Strengths – Coarse Grid 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 44

45. Pixel-Averaging Code Array[griddata, {200, 200}]; ninc = 200; npt = 100; nzro = 0; gridinc = 0.000001; (* Degrees *) For[i = 1, i grid3[[1, 1]] + i*gridinc && flux[k, 4] grid3[[1, 2]] + j*gridinc && flux[k, 5] 0, griddata[i, j] = griddata[i, j]/n] ] ] ]; 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 45

46. Comments: Averaging such high-quality data does not improve the result – unmeasured regions Even on this course grid, many pixels have no measured data Something must be done to interpolate data into all grid squares from the locations at which measurements exist 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 46

47. Resampling with Interpolation Used for estimating data on a spatially regular (periodic) “grid” from data gathered at irregular locations or intervals Used to regularize a grid of data pursuant to further processing, for instance by convolution methods, requiring regularized data input 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 47

48. Approaches: Fitting spline surface to the existing measurements (independent of the physics) Calculating field in missing pixels by some relationship with nearest known measurements (interpolation) Fitting a physical model (3D magnetic field distribution in this example) to the data points (incorporates know physical model) 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 48

49. Fitting Data Models Approximation of measured data by means of a physical or mathematical model Fitting is the process of selecting model parameters so that the model will fit the data points optimally, according to some criterion Least-squares fitting is common 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 49

50. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 50 Fitting Surfaces The method also applies to fitting 2-dimensional functions of the form: Using the set of data points: Higher order functions can also be treated in this manner.

51. Spline Surface Model 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 51 100 th degree spline fit to magnetometer data of Slide 44. Even with high degree, spline surface shows experimental scanning artifact, not indicative of single magnetic dipole.

52. Filtering Use of information from multiple measurements to refine an attribute estimate at a given point May be effectively applied to a spatial grid of points by the use of convolution methods (Discussed previously in Lectures 4 and 5) 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 52

53. Convolution Filtering of Magnetic Data 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 53 Convolution filtering of magnetometer data of Slide 44 using 15 x 15 kernel Magnetic hotspot is clearly shown but artifact of field scan is still visible

54. 15 x 15 Convolution Filter Kernel Array[c, 16, 16]; n = 15; sm = 4661.0; c = {{1, 2, 3, 5, 6, 7, 8, 9, 8, 7, 6, 5, 3, 2, 1}, {2, 3, 5, 8, 10, 13, 14, 15, 14, 13, 10, 8, 5, 3, 2}, {3, 5, 8, 12, 16, 20, 22, 23, 22, 20, 16, 12, 8, 5, 3}, {5, 8, 12, 18, 23, 29, 32, 34, 32, 29, 23, 18, 12, 8, 5}, {6, 10, 16, 23, 31, 38, 43, 45, 43, 38, 31, 23, 16, 10, 6}, {7, 13, 20, 29, 38, 47, 53, 55, 53, 47, 38, 29, 20, 13, 7}, {8, 14, 22, 32, 43, 53, 60, 62, 60, 53, 43, 32, 22, 14, 8}, {9, 15, 23, 34, 45, 55, 62, 65, 62, 55, 45, 34, 23, 15, 9}, {8, 14, 22, 32, 43, 53, 60, 62, 60, 53, 43, 32, 22, 14, 8}, {7, 13, 20, 29, 38, 47, 53, 55, 53, 47, 38, 29, 20, 13, 7}, {6, 10, 16, 23, 31, 38, 43, 45, 43, 38, 31, 23, 16, 10, 6}, {5, 8, 12, 18, 23, 29, 32, 34, 32, 29, 23, 18, 12, 8, 5}, {3, 5, 8, 12, 16, 20, 22, 23, 22, 20, 16, 12, 8, 5, 3}, {2, 3, 5, 8, 10, 13, 14, 15, 14, 13, 10, 8, 5, 3, 2}, {1, 2, 3, 5, 6, 7, 8, 9, 8, 7, 6, 5, 3, 2, 1}}/sm; 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 54

55. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 55 Gaussian Model Concept: To model the rainfall over a region in which there are NWS weather stations To sample the model at the location of weather stations To modify the Gaussian model until the samples represent actual rainfall data from the weather stations. To use the model to predict regional rainfall.

56. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 56 Measured Rainfall [cm] ##LATTITUDELONGITUDERAINQUADRANGLE 0140.01117-75.44232.8Valley Forge 0240.01617-75.505833.8Malvern 0340.04367-75.498833.6Valley Forge 0439.98483-75.379332.5Media 0539.97583-75.510333.8West Chester 0639.9505-75.510333.6 West Chester 0739.97867-75.5433.6 West Chester 0839.96533-75.5433.6 West Chester 0940.13117-75.509673.1Malvern 1040.1125-75.551333.8 Malvern 1140.00833-75.62252.5Malvern 1239.89117-75.5883.8 West Chester

57. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 57 Gaussian 2D-Model

58. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 58 Gaussian Model for Rainfall Rainfall [cm] as a function of distance [km] from Lat 40.0000, Lon -75.5000.

59. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 59 Sampled Values Coordinates of weather stations taken These are referenced to central point (e.g. Lat: 40.0000, Lon: -75.5000) Gaussian model is sampled at these points Averages are calculated for each station

60. 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 60 Example Data for x=1, y=1 [km] These are statistical samples of the Gaussian model for a single (x, y) point. An average of similar data would be formed for each weather station. 0.817652.519341.58060 0.077162.235611.47360 0.977962.311091.06013 1.661840.498361.20778 0.724582.427280.32148 0.630971.438010.56210 2.84448

61. Gradient Calculation Calculation of spatial derivatives, usually over a regular field of regular data points Requires well filtered data, because derivatives amplify data variations (noise) Convolution methods can be used for the calculations 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 61

62. Discussion 8/23/2015© 2009 Raymond P. Jefferis III Lect 11 - 62