1. Business Identification: Spatial Detection Alexander Darino Week 8

2. Weaknesses to Current Approach Latitude Longitude Geocoding Reverse Geocoding Nearby Businesses Image OCR Detected Text Business Name Matching Business Identification Business Spatial Detection 2

3. STR Implementation STR Implementation: “Automatic Detection and Recognition of Signs From Natural Scenes” Multiresolution- based potential characters detection Character/layout geometry and color properties analysis Local affine rectification Refined Detection

4. Multiresolution-based potential characters detection

7. STR Implementation Original Next Step: Replace with readily available text detector Text detectors are not readily available (Will revisit later)

8. TEMPLATE-IMAGE SIFT MATCHING After many technical difficulties…

9. Template Name George Font Trebuchet MS # Levels 3 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 20-24.1(95%) Image Name George … # Levels 3 Peak Threshold 0 Edge Threshold 10 Statistics Good 1 Bad 0 Total (% G) 1 (100%)

10. Template Name George Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 25.60547(95%) Image Name George … # Levels 3 Peak Threshold 0 Edge Threshold 10 Statistics Good 1 Bad 0 Total (% G) 1 (100%)

11. Template Name George Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 25.60547(95%) Image Name George … # Levels 1 Peak Threshold 0 Edge Threshold 10 Statistics Good 2 Bad 0 Total (% G) 2 (100%)

12. Template Name George Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 25.60547(95%) Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 10 Statistics Good 2 Bad 0 Total (% G) 2 (100%)

13. Template Name George Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 25.60547(95%) Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 2 Bad 0 Total (% G) 2 (100%)

14. Template Name Aiken Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 24.439163(95%) Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 3 Bad 0 Total (% G) 3 (100%)

15. Template Name Delicious Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 26.656116 Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

16. Template Name Prepared Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 26.656116 Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

17. Template Name Foods Font Trebuchet MS # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 26.656116 Image Name George … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

18. Template Name Bruegger’s Font Arial Rounded MT Bold # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 32.288651 Image Name Bruegger’s … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

19. Template Name Bakery Font Arial Rounded MT Bold # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 29.145470 Image Name Bruegger’s … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

20. Template Name Bakery Font Arial Rounded MT Bold # Levels 5 Peak Threshold 0 Edge Threshold 10 Scale Cutoff 29.145470 Image Name Bruegger’s … # Levels 1 Peak Threshold 20 Edge Threshold 6 Statistics Good 0 Bad 0 Total (% G) 0 (0%)

21. SCENE TEXT RECOGNITION Moving away from SIFT and revisiting

22. Scene Text Recognition Did not hear back from individuals contacted for STR implementation Returning to STR Implementation – Further reading indicates that patches are necessary for subsequent algorithms – Text detection is not enough: need to implement specified text detector Multiresolution- based potential characters detection Character/layout geometry and color properties analysis Local affine rectification Refined Detection

23. Color Properties Analysis Implemented Gaussian Mixture Model (GMM) to obtain μ and σ of foreground/background for: R/G/B/H/I Calculated Confidences that component (RGBHI) can be used to recognize characters Multiresolution- based potential characters detection Character/layout geometry and color properties analysis Local affine rectification Refined Detection

24. Original

25. Red

26. Green

27. Blue

28. Hue

29. Intensity

30. Evaluation The highest confidence was found in Intensity even though most letters vanish, vs Hue where letters are easily distinguisible This suggests text recognition should occur individually per character The paper further suggests it needs the patches around the individual characters

31. Next Step

33. Thank You