1. SEMANTIC FEATURE ANALYSIS IN RASTER MAPS Trevor Linton, University of Utah

2. Acknowledgements  Thomas Henderson  Ross Whitaker  Tolga Tasdizen  The support of IAVO Research, Inc. through contract FA9550-08-C-005.

3. Field of Study  Geographical Information Systems  Part of Document Recognition and Registration.  What are USGS Maps?  A set of 55,000 – 1:24,000 scale images of the U.S. with a wealth of data.  Why study it?  To extract new information (features) from USGS maps and register information with existing G.I.S and satellite/aerial imagery.

4. Problems  Degradation and scanning produces noise.  Overlapping features cause gaps.  Metadata has the same texture as features.  Closely grouped features makes discerning between features difficult.

5. Problems – Noisy Data Scanning artifact which introduces noise

6. Problems – Overlapping Features Metadata and Features overlap with similar textures. Gaps in data.

7. Problems – Closely Grouped Features Closely grouped features make discerning features difficult.

8. Thesis & Goals  Using Gestalt principles to extract features and overcome some of the problems described.  Quantitatively extract 95% recall and 95% precision for intersections.  Quantitatively extract 99% recall and 90% precision for intersections.  Current best method produces 75% recall and 84% precision for intersections.

9. Approach  Gestalt Principles  Organizes perception, useful for extracting features.  Law of Similarity  Law of Proximity  Law of Continuity

10. Approach – Gestalt Principles  Law of Similarity  Grouping of similar elements into whole features.  Reinforced with histogram models.

11. Approach – Gestalt Principles  Law of Proximity  Spatial proximity of elements groups them together.  Reinforced through Tensor Voting System

12. Approach – Gestalt Principles  Law of Continuity  Features with small gaps should be viewed as continuous.  Idea of multiple layers of features that overlap.  Reinforced by Tensor Voting System.

13. Approach – Framework Overview

14. Pre-Processing  Class Conditional Density Classifier  Uses statistical means and histogram models.  μ = Histogram model vector.  Find class k with the smallest δ is the class of x.

15. Pre-Processing  k-Nearest Neighbors  Uses the class that is found most often out of k closest neighbors in the histogram model.  Closeness is defined by Euclidian distance of the histogram models.

16. Pre-Processing  Knowledge Based Classifier  Uses logic that is based on our knowledge of the problem to determine classes.  Based on information on the textures each class has.

17. Pre-Processing  Original Image with Features Estimated

18. Pre-Processing  Original Image with Roads Extracted Class condition classifier k-Nearest Neighbors Knowledge Based

19. Tensor Voting System  Overview

20. Tensor Voting System  Uses an idea of “Voting”  Each point in the image is a tensor.  Each point votes how other points should be oriented.  Uses tensors as mathematical representations of points.  Tensors describe the direction of the curve.  Tensors represent confidence that the point is a curve or junction.  Tensors describe a saliency of whether the feature (whether curve or junction) actually exists.

21. Tensor Voting System  What is a tensor?  Two vectors that are orthogonal to one another packed into a 2x2 matrix.

22. Tensor Voting System  Creating estimates of tensors from input tokens.  Principal Component Analysis  Canny edge detection  Ball Voting

23. Tensor Voting System  Voting  For each tensor in the sparse field  Create a voting field based on the sigma parameter.  Align the voting field to the direction of the tensor.  Add the voting field to the sparse field.  Produces a dense voting field.

24. Tensor Voting System  Voting Fields  A window size is calculated from  Direction of each tensor in the field is calculated from  Attenuation derived from

25. Tensor Voting System  Voting Fields (Attenuation)  Red and yellow are higher votes, blue and turquoise lower.  Shape related to continuation vs. proximity.

26. Tensor Voting System  Extracting features from dense voting field.  determines the likelihood of being on a curve.  determines the likelihood of being a junction.  If both λ 1 and λ 2 are small then the curve or junction has a small amount of confidence in existing or being relevant.

27. Tensor Voting System  Extracting features from dense voting field.  Original Image Curve Map Junction Map

28. Post-processing  Extracting features from curve map and junction map.  Global Threshold and Thinning  Local Threshold and Thinning  Local Normal Maximum  Knowledge Based Approach

29. Post-processing  Global threshold on curve map. Applied Threshold Thinned Image

30. Post-processing  Local threshold on curve map. Applied Threshold Thinned Image

31. Post-processing  Local Normal Maximum  Looks for maximum over the normal of the tensor at each point. Applied Threshold Thinned Image

32. Post-processing  Knowledge Based Approach  Uses knowledge of types of artifacts of the local threshold to clean and prep the image. Original Image Knowledge Based Approach

33. Experiments  Determine adequate parameters.  Identify weaknesses and strengths of each method.  Determine best performing methods.  Quantify the contributions of tensor voting.  Characterize distortion of methods on perfect inputs.  Determine the impact of misclassification of text on roads.

34. Experiments  Quantitative analysis done with recall and precision measurements.  Relevant is the set of all features that are in the ground truth.  Retrieved is the set of is all features found by the system.  tp = True Positive, fn = False Negative, fp = False Positive  Recall measures the systems capability to find features.  Precision characterizes whether it was able to find only those features.  For both recall and precision, 100% is best, 0% is worst.

35. Experiments  Data Selection  Data set must be large enough to adequately represent features (above or equal to 100 samples).  One sub-image of the data must not be biased by the selector.  One sub-image may not overlap another.  A sub-image may not be a portion of the map which contains borders, margins or the legend.

36. Experiments  Ground Truth  Manually generated from samples.  Roads and intersections manually identified.  Ground Truth is generated twice, those with more than 5% of a difference are re-examined for accuracy. Ground truth Original Image

37. Experiments  Best Pre-Processing Method  All pre-processing methods examined without tensor voting or post processing for effectiveness.  Best window size parameter for k-Nearest Neighbors was qualitatively found to be 3x3.  The best k parameter for k-Nearest Neighbors was quantitatively found to be 10.  The best pre-processing method found was the Knowledge Based Classifier

38. Experiments  Tensor Voting System  Results from test show the best value for σ is between 10 and 16 with little difference in performance.

39. Experiments  Tensor Voting System  Contributions from tensor voting were mixed.  Thresholding methods performed worse.  Knowledge based method improved 10% road recall, road precision dropped by 2%, intersection recall increased by 22% and intersection precision increased by 20%.

40. Experiments  Best Post-Processing  Finding the best window size for local thresholding.  Best parameter was found between 10 and 14.

41. Experiments  Best Post-Processing  The best post-processing method was found by using a naïve pre-processing technique and tensor voting.  Knowledge Based Approach performed the best.

42. Experiments  Running the system on perfect data (ground truth as inputs) produced higher results then any other method (as expected).  Thesholding had a considerably low intersection precision due to artifacts produced in the process.

43. Experiments  Best combination found was k-Nearest Neighbors with a Knowledge Based Approach.  Note the best pre-processing method Knowledge Based Classifier was not the best pre-processing method when used in combinations due to the type of noise it produces.  With Text:  92% Road Recall, 95% Road Precision  82% Intersection Recall, 80% Intersection Precision  Without Text:  94% Road Recall, 95% Road Precision  83% Intersection Recall, 80% Intersection Precision

44. Experiments  Confidence Intervals (95% CI, 100 samples)  Road Recall:  Mean: 93.61% CI [ 92.47%, 94.75% ] ± 0.14%  Road Precision:  Mean: 95.23% CI [ 94.13%, 96.33% ] ± 0.10%  Intersection Recall:  Mean: 82.22% CI [ 78.91%, 85.51% ] ± 3.29%  Intersection Precision:  Mean: 80.1% CI [ 76.31%, 82.99% ] ± 2.89%

45. Experiments  Adjusting parameters dynamically  Dynamically adjusting the σ between 4 and 10 by looking at the amount of features in a window did not produce much difference in the recall and precision (less than 1%).  Dynamically adjusting the c parameter in tensor voting actually produced worse results because of exaggerations in the curve map due to slight variations in the tangents for each tensor.

46. Future Work & Issues  Tensor Voting and thinning tend to bring together intersections too soon when the road intersection angle was too low or the roads were too thick.  The Hough transform may possibly overcome this issue.

47. Future Work & Issues  Scanning noise will need to be removed in order to produce high intersection recall and precision results.

48. Future Work & Issues  Closely grouped and overlapping features.

49. Future Work & Issues  Developing other pre-processing and post-processing techniques.  Learning algorithms  Various local threshold algorithms  Road following algorithms