1. Masters ProjectFall 2010Joe B. Taylor Pedestrian Navigation Using Mobile Devices Georeferencing Photographs of Publicly Posted Maps Masters Project

2. Fall 2010Joe B. Taylor Overview Introduction Georeferencing Terminology and Examples Approaches Explored Challenges Future Work Summary Prototype Demonstration Question and Answer References

3. Masters ProjectFall 2010Joe B. Taylor Introduction Mobile devices such as the iPhone with GPS & camera are popular Mobile device pedestrian navigation maps are rare http://en.wikipedia.org/wiki/File:IPhone_sales_per_quarter_simple.svg

4. Masters ProjectFall 2010Joe B. Taylor Why are local maps more useful? Google Maps TomTom Posted UCCS Map

5. Masters ProjectFall 2010Joe B. Taylor Relate a photo to the real world? Related Work –Two Point Entry [Schöning, et al, 2009] User moves to and marks two locations –Smart Align [Schöning, et al, 2009] User aligns photo over an existing map Proposed by this project –GPS Track Alignment User aligns recently traveled path over photo

6. Masters ProjectFall 2010Joe B. Taylor Georeferencing Terminology and Examples

7. Masters ProjectFall 2010Joe B. Taylor Terminology Map - In this context, the map is a photo Pan - Move left / right and up / down Scale - Zoom in or out Rotate - Pivot around a point Northing - Direction on the map that is North Track - Set of GPS coordinates represented as a line Align - Calibrate the photo to Earth in this project’s context this means to align the photo with the track

8. Masters ProjectFall 2010Joe B. Taylor Example based on [Schöning, et al, 2009] Start position indicated by user with orange cross hair GPS location associated to map User physically moves to a new location and indicates with the blue cross hair GPS location associated to map Requires North up map orientation Two Point Entry

9. Masters ProjectFall 2010Joe B. Taylor Smart Align Example based on [Schöning, et al, 2009] Photo of map (green area) displayed over a Google map Photo of map is partially transparent User rotates, resizes and moves photo map to align with Google map Requires data access or preloaded maps and in the case of Google maps this method violates the Google maps API agreement

10. Masters ProjectFall 2010Joe B. Taylor GPS Track Alignment User walks within the map area while the GPS locations are recorded every few seconds An image of the GPS locations is overlaid on the photo of the map simulated here with orange dots The GPS track image is moved, resized and rotated by the user to match the actual path taken In field testing, route selection is important. Easily identifiable map locations and curves help

11. Masters ProjectFall 2010Joe B. Taylor Advantages of GPS Track Alignment Less error prone than specifying two pixels on the photo map Manually aligning a line of dots over a path will be easier than trying to line up two maps No additional data or maps required

12. Masters ProjectFall 2010Joe B. Taylor Approaches Explored Which moves the track, the map or both?

13. Masters ProjectFall 2010Joe B. Taylor Moveable Track Line Track line adjusted by –panning, scaling, rotating Gestures anywhere on screen move the track Map is fixed and scaled to show the entire map Works well until the track gets scaled down too small to easily manipulate

14. Masters ProjectFall 2010Joe B. Taylor Moveable Track & Map Track line adjusted by –panning, scaling, rotating Map is full size and pannable Gestures must be executed over map or track to move that particular portion Gestures are difficult to execute if all three are available for both the map and the track line

15. Masters ProjectFall 2010Joe B. Taylor Moveable Map Track line scaled to fit the screen and fixed in place Map is adjusted by –panning, scaling, rotating Gestures anywhere on screen move the map Track always stays large enough to improve alignment Found simplest method for aligning the track and map

16. Masters ProjectFall 2010Joe B. Taylor Challenges Projections, Transformations & Multiple Interface Approaches

17. Masters ProjectFall 2010Joe B. Taylor Cartographic Projections One-to-one ratio for mapping longitude, latitude to x,y was originally assumed Sinusodial projection required to deliver accurate reproduction of track lines

18. Masters ProjectFall 2010Joe B. Taylor Cartographic Projection Impact Higher line is 1:1 mapping of Lon / Lat Lower line is using sinusoidal projection to correct the perspective Variance is sufficient to greatly hinder alignment

19. Masters ProjectFall 2010Joe B. Taylor Affine Transformations a and d control the scale of the transform b and c control the rotation of the transform t x and t y control the panning or translation Calculating these by hand is straight-forward Interpreting these values from the Quartz 2D API is a different story

20. Masters ProjectFall 2010Joe B. Taylor iOS and Affine Transformations iOS provides the Quartz 2D API 2D API is powerful but complex Typical iPhone transformations include rotating text and moving a game piece This project needs to translate between 3 different coordinate systems: Lat, Lon; Lat, Lon on a flat plane; X,Y on the photo Appearance on screen is not the only consideration

21. Masters ProjectFall 2010Joe B. Taylor iOS and Affine Transformations iOS treats all images and drawings as individual views Transformations must be in the correct view context to be valid Views are managed in contexts as well –User context is device independent –Device context is device specific Frames, Bounds and Transforms –Frames become undefined when transforms are used

22. Masters ProjectFall 2010Joe B. Taylor Multiple Interface Approaches A single alignment interface implementation was originally planed and estimated. Three implementations were built and tested Each new approach required completely rethinking the relationships and alignment process Dropbox.net used for file backups contains 380 versions of the alignment code file Approximately 90 hours invested in analyzing and reworking the alignment calculations

23. Masters ProjectFall 2010Joe B. Taylor Calculation Code

24. Masters ProjectFall 2010Joe B. Taylor Causes for Calculation Complexity Navigation requires –Transform Latitude, Longitude to X,Y –Transform X,Y for map scale, rotation & pan Alignment requires reversing this process New transform matrix perspective per approach Transform API documentation inconsistently states that transforms are relative to top left, bottom left and center. Could not find consistent transform point.

25. Masters ProjectFall 2010Joe B. Taylor Causes for Calculation Complexity Manual calculations frustrated by API implementation –Manipulating only scale alters translation –Manipulating only rotation alters translation –Manipulating only translation works as expected API documentation mentions that order in which multiple translations are applied is critical –Example shown with pan, rotate and scale producing very different results depending on the order of execution

26. Masters ProjectFall 2010Joe B. Taylor Causes for Calculation Complexity Two different sets of pixels / degree values –Original track generation pixel ratios –Map adjusted pixel ratios –Each set of ratios has different values for latitude and longitude Working between fourth quadrant of a grid for positive latitude and negative longitude further complicates working in the first or second quadrant for X,Y depending on translation pivot point

27. Masters ProjectFall 2010Joe B. Taylor Conclusion Lessons Learned, Future Work, Summary & Demo

28. Masters ProjectFall 2010Joe B. Taylor Lessons Learned Mobile device touch screens limit design options due to space requires for touch gestures Projecting spheroid coordinates to a flat plane correctly is important even in small areas Georeferencing photos as an end user is more complex than originally assumed Translation techniques such as Affine Transformations are complicated by iOS framework implementation choices

29. Masters ProjectFall 2010Joe B. Taylor Future Work Survey average smart phone users for usability feedback and measure alignment accuracy for these users Assumption of map northing and typical scale could be used to generated the track lines closer to their expected scale and orientation Map photos could be analyzed for likely paths and compared to the GPS track lines for possible matches

30. Masters ProjectFall 2010Joe B. Taylor Summary Reviewed photo georeferencing research Researched map projections to planes Researched Affine Transformations Field tested the prototype Delivered working prototype and code Delivered written project report Delivered a video prototype demonstration Invested considerable hours beyond the original estimates

31. Masters ProjectFall 2010Joe B. Taylor Prototype Demonstration via Video

32. Masters ProjectFall 2010Joe B. Taylor

33. Masters ProjectFall 2010Joe B. Taylor Questions?

34. Masters ProjectFall 2010Joe B. Taylor References Schöning, J., Krüger, A., Cheverst, K., Rohs, M., Löchtefeld, M., and Taher, F. 2009. PhotoMap: using spontaneously taken images of public maps for pedestrian navigation tasks on mobile devices. In Proceedings of the 11th international Conference on Human-Computer interaction with Mobile Devices and Services (Bonn, Germany, September 15 - 18, 2009). MobileHCI '09. ACM, New York, NY, 1-10. Cheverst, K., Schöning, J., Krüger, A., and Rohs, M. Photomap: Snap, grab and walk away with a “you are here” map. In Proc. of MobileHCI ’08 : Workshop on Mobile Interaction with the Real World, September 2008. Smets, N. J., te Brake, G. M., Neerincx, M. A., and Lindenberg, J. 2008. Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 73-80.

35. Masters ProjectFall 2010Joe B. Taylor References Burigat, S., Chittaro, L., and Parlato, E. 2008. Map, diagram, and web page navigation on mobile devices: the effectiveness of zoomable user interfaces with overviews. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 147-156. Setlur, V., Kuo, C., and Mikelsons, P. 2010. Towards designing better map interfaces for the mobile: experiences from example. In Proceedings of the 1st international Conference and Exhibition on Computing For Geospatial Research & Application (Washington, D.C., June 21 - 23, 2010). COM.Geo '10. ACM, New York, NY, 1-4. S. Erle, R. Gibson and J. Walsh, Mapping Hacks. Sebastopol, CA:O’Reilly Media, Inc. 2005, pp. 120-140, 156-160.

36. Masters ProjectFall 2010Joe B. Taylor