1. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 1 Frank Allan Hansen, fah@daimi.au.dk Integrating the Web and the World: Contextual Trails on the Move Torben B. Pedersen Frank Allan Hansen Niels Olof Bouvin Bent G. Christensen Kaj Grønbæk Jevgenij Gagach Department of Computer Science, University of Aarhus, Denmark Department of Computer Science, Aalborg University, Denmark Euman Ltd., Denmark

2. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 2 Frank Allan Hansen, fah@daimi.au.dk Overview Motivation –Hypermedia in the physical world –Utilizing context information HyCon: Contextual Hypermedia –HyCon Architecture and Data Model –The HyConExplorer Prototype: Browsing and Searching Producing content: Annotations, Links, and Trails The SVG-based Client Framework Conclusion

3. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 3 fah@daimi.au.dk Motivation

4. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 4 Frank Allan Hansen, fah@daimi.au.dk Motivation Techniques for augmenting the physical world with digital structures Linking objects in physical and digital space –Supporting access to digital information related to the user’s context –Supporting production of information and documentation in the user’s context Beyond the PC: – Supporting interaction through heterogeniuos mobile devices

5. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 5 Frank Allan Hansen, fah@daimi.au.dk Open Hypermedia Structures How can Open Hypermedia structures be used to Link objects in physical and digital space? –OH structures are not embedded in the linked resources! –Externally defined, out-of-line links –Write access to the resources is not required –Support multiple sets of structures on the same corpus of documents or objects Hypermedia structures: –Links Multi-headed, bi-directional, typed, generic –Annotations (user created comments and footnotes) –Guided tours and information trails –... If you can identify an object you can link it!

6. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 6 Frank Allan Hansen, fah@daimi.au.dk Defining Context DEY (2001), Understanding and using context: “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” “A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.” Utilizing the part of the surroundings (the context) which is relevant to the user’s task

7. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 7 Frank Allan Hansen, fah@daimi.au.dk Defining Context (2) Context-aware application support: –Presentation of information and services –Automatic execution of services –Tagging of context to information to support later retrieval

8. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 8 Frank Allan Hansen, fah@daimi.au.dk Hypermedia in the Physical World Location based browsing and search relative to the users context (time, place, activity,...) –Geo-tagged information –Unstructured Web information (GBS/Google) Digital graffiti –Location based annotations: The user as producer of information (Photo, video, speak, text notes) –Leaving trails of information ”Social computing” Digital graffiti: augmenting the environment without spoiling it

9. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 9 fah@daimi.au.dk HyCon: Contextual Hypermedia

10. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 10 Frank Allan Hansen, fah@daimi.au.dk The HyCon Platform

11. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 11 Frank Allan Hansen, fah@daimi.au.dk The Sensor Layer Local sensors Remote sensors

12. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 12 Frank Allan Hansen, fah@daimi.au.dk The HyCon Data Model

13. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 13 Frank Allan Hansen, fah@daimi.au.dk The HyConExplorer Prototype –Implemented in Java, SOAP, and SVG –Runs on commercially available tablet PCs connected to the network through WLAN or GRPS connections on cell phones –The tablet has a mounted camera for capturing low resolution images, video, and audio in the application. –The user’s physical location is registered through a Bluetooth enabled GPS unit. Tablet PC

14. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 14 Frank Allan Hansen, fah@daimi.au.dk Context Based Browsing Navigating resources by browsing Context based browsing –Browsing the digital context… –by changing parameters in the physical context Direct Physical Navigation –Browsing with your feet –Physically walking or driving from one location to another affects the time and location parameters –User “bump into” information –Used by tourist guide systems, navigation systems, etc. Indirect Representational Navigation –It’s not always desirable nor feasible to have to physically move around to browse information. –Allows users to investigate information structures in the system even though they are not actually in the physical context associated with the structures.

15. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 15 Frank Allan Hansen, fah@daimi.au.dk Context Based Search (GBS) Search for information associated with the user’s context Geo Based Search: Web searches are augmented with context information or information derived from sensor data –Searching for shops or restaurants nearby which are open now –Requires information about the user’s (and shops) location and the time of day The benefit of adding criteria based on context information is of course highly dependent on the search engine used and how the context data can be formulated as an appropriate criteria…

16. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 16 Frank Allan Hansen, fah@daimi.au.dk Context Based Search (GBS) The user’s position can be acquired from GPS –Search engines are not indexing pages by GPS coordinates  (geo-tagging is not very popular) –Derive GPS coordinates to postal addresses! Businesses often provide their postal addresses on their Web sites GBS: user supplied search terms are augmented with computed postal addresses => The matching Web pages presumably covers the topic of interest and are associated to the given geographical area

17. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 17 Frank Allan Hansen, fah@daimi.au.dk Context Based Search (GBS)

18. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 18 Frank Allan Hansen, fah@daimi.au.dk Context Based Search (GBS)

19. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 19 Frank Allan Hansen, fah@daimi.au.dk Context Based Search (GBS)

20. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 20 Frank Allan Hansen, fah@daimi.au.dk Context Based Annotations Going beyond the classical context-aware browsing and navigation systems Support users as active information producers –documenting or commenting on objects or situations Combining context sensors with mobile devices equipped with built-in cameras and microphones –Supports creation of video, audio, photo, and text annotations and automatic tagging with context information –Take photo in context –Taking notes about a site –Using browsing and searching the information can later be retrieved by the user, friends, or colleagues

21. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 21 Frank Allan Hansen, fah@daimi.au.dk Context Based Trails Creating structure! Organizing information as you go Leaving (user created) guided tours through collections of “points of interest” –No requirement for pre-defined tourist guides “Social computing” –Combined with the annotation facilities trails may serve as personalized impressions of a given area and even allow users to share their impressions through discussions linked to points of interests along the trail.

22. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 22 Frank Allan Hansen, fah@daimi.au.dk Context Based Trails

23. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 23 fah@daimi.au.dk The SVG-based Client Framework

24. ISIS Katrinebjerg i n t e r a c t i v e s p a c e s. n e t 24 fah@daimi.au.dk Conclusion