1. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 1 Frank Allan Hansen, fah@daimi.au.dk Integrating the Web and the World: Contextual Trails on the Move Torben B. Pedersen Frank Allan Hansen Niels Olof Bouvin Bent G. Christensen Kaj Grønbæk Jevgenij Gagach Department of Computer Science, University of Aarhus, Denmark Department of Computer Science, Aalborg University, Denmark Euman Ltd., Denmark

2. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 2 Frank Allan Hansen, fah@daimi.au.dk Overview Motivation –Hypermedia in the physical world –Utilizing context information HyCon: Contextual Hypermedia –HyCon Architecture and Data Model –The HyConExplorer Prototype The SVG-based Client Framework Conclusion

3. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 3 fah@daimi.au.dk Motivation

4. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 4 Frank Allan Hansen, fah@daimi.au.dk Motivation Techniques for augmenting the physical world with digital structures Linking objects in physical and digital space –Supporting access to digital information related to the user’s context –Supporting production of information and documentation in the user’s context Beyond the PC: –Supporting interaction through heterogeniuos mobile devices

5. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 5 Frank Allan Hansen, fah@daimi.au.dk Open Hypermedia Structures How can Open Hypermedia structures be used to Link objects in physical and digital space? –OH structures are not embedded in the linked resources! –Externally defined, out-of-line links –Write access to the resources is not required –Support multiple sets of structures on the same corpus of documents or objects Hypermedia structures: –Links Multi-headed, bi-directional, typed, generic –Annotations (user created comments and footnotes) –Guided tours and information trails –... If you can identify an object you can link it!

6. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 6 Frank Allan Hansen, fah@daimi.au.dk Context and Hypermedia Utilizing the part of the surroundings (the context) which is relevant to the user’s task Context-aware application support: –Presentation of information and services Trails, annotations and links –Automatic execution of services Geo Based Search, GBS –Tagging of context to information to support later retrieval Trails, annotations and links

7. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 7 Frank Allan Hansen, fah@daimi.au.dk Hypermedia in the Physical World Location based browsing and search relative to the users context (time, place, activity,...) –Geo-tagged information –Unstructured Web information (GBS/Google) Digital graffiti –Location based annotations: The user as producer of information (Photo, video, speak, text notes) –Leaving trails of information ”Social computing” Digital graffiti: augmenting the environment without spoiling it

8. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 8 fah@daimi.au.dk HyCon: Contextual Hypermedia

9. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 9 Frank Allan Hansen, fah@daimi.au.dk The HyCon Platform BlipNode GPS Camera

10. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 10 Frank Allan Hansen, fah@daimi.au.dk The HyCon Data Model

11. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 11 fah@daimi.au.dk The HyConExplorer Prototype

12. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 12 Frank Allan Hansen, fah@daimi.au.dk The HyConExplorer Prototype –Prototype developed at University of Aarhus, Denmark –Application of the HyCon framework –Implemented in Java, SOAP, and SVG –Runs on commercially available tablet PCs connected to the network through WLAN or GRPS connections on cell phones –The tablet has a mounted camera for capturing low resolution images, video, and audio in the application. –The user’s physical location is registered through a Bluetooth enabled GPS unit. Tablet PC

13. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 13 Frank Allan Hansen, fah@daimi.au.dk Context Based Browsing Navigating resources by browsing –Browsing the digital –context… by changing parameters in the physical context Direct Physical Navigation –Browsing with your feet –Physically walking or driving from one location to another affects the time and location parameters Indirect Representational Navigation –Allows users to investigate information structures in the system even though they are not actually in the physical context associated with the structures.

14. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 14 Frank Allan Hansen, fah@daimi.au.dk Context Based Annotations Going beyond the classical context- aware browsing and navigation systems Support users as active information producers –Documenting or commenting on objects or situations –Linking Web resources to places Combining context sensors with mobile devices equipped with built-in cameras and microphones –Supports creation of video, audio, photo, and text annotations and automatic tagging with context information –Take photo in context –Taking notes about a site

15. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 15 Frank Allan Hansen, fah@daimi.au.dk Context Based Trails Creating structure! Organizing infor- mation as you go Leaving (user created) guided tours through collections of “points of interest” –No requirement for pre-defined tourist guides “Social computing”

16. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 16 Frank Allan Hansen, fah@daimi.au.dk Context Based Trails (2)

17. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 17 Frank Allan Hansen, fah@daimi.au.dk Geo Based Search (GBS) Search for information associated with the user’s context Geo Based Search: Web searches are augmented with context information or information derived from sensor data –Searching for shops or restaurants nearby which are open now –Requires information about the user’s (and shops) location and the time of day The benefit of adding criteria based on context information is of course highly dependent on the search engine used and how the context data can be formulated as an appropriate criteria…

18. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 18 Frank Allan Hansen, fah@daimi.au.dk Geo Based Search (GBS) The user’s position can be acquired from GPS –Search engines are not indexing pages by GPS coordinates  (geo- tagging is not very popular) –Derive GPS coordinates to postal addresses! Businesses often provide their postal addresses on their Web sites GBS: user supplied search terms are augmented with computed postal addresses

19. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 19 fah@daimi.au.dk The SVG-based Client Framework

20. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 20 Frank Allan Hansen, fah@daimi.au.dk The SVG-based Client Framework –Prototype developed at Aalborg University, Denmark –Implemented in Symbian C++ and using the Bitflash SVG component on Symbian Creating user interfaces for heterogeneous mobile devices is challenging! The client framework implements a “scriptable SVG browser”

21. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 21 Frank Allan Hansen, fah@daimi.au.dk The SVG Browser …

22. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 22 fah@daimi.au.dk Conclusion