1. Dept. of Computer & Information Science, Korea University Part I Surveys of Context-aware Systems 2008.01.28 임영희 http://idb.korea.ac.kr DB & Mining Lab. Korea University

2. DB & MINING Lab., Dept. of Computer & Information Science 2 Contents Definition of Context & Context-aware Application Architecture of Context-aware System Context Models Existent Context-aware Applications - Office and Meeting Tools - Tourist Guide - Memory Aids - Context-aware Home Essential Technologies for Context-aware Services

3. DB & MINING Lab., Dept. of Computer & Information Science 3 Definition of Context Context is - situational information - any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves. - the set of environmental states and settings that either determines an application’s behavior or in which an application event occurs

4. DB & MINING Lab., Dept. of Computer & Information Science 4 Categories of Context(1) Categories of Context - Primary context :  Location (where)  Identity (who)  Time (when)  Activity (what) - Secondary context  can be indexed by primary context  Ex) given a person’s identity,  phone number, addresses, birthdate given an entity’s location,  what object are near, what activity is occurring near the entity

5. DB & MINING Lab., Dept. of Computer & Information Science 5 Categories of Context(2) Another categories of Context - external (physical) context :  can be measured by h/w sensors  location, light, sound, movement, touch, temperature… - internal (logical) context:  be specified by users or captured monitoring the user’s interaction  user’s goal, tasks, user’s emotional state…

6. DB & MINING Lab., Dept. of Computer & Information Science 6 Context-aware Application Definition of Context-Aware - A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task Characteristics of Context-Aware application - presentation of information and services to a user - automatic execution of a service - tagging of context to information for later retrieval

7. DB & MINING Lab., Dept. of Computer & Information Science 7 Context-aware Application Categories of Context-Aware applications - Active context awareness:  An application automatically adapts to discovered context, by changing the application behavior. - Passive context awareness:  An application presents the new or updated context to an interested user or makes the context persistent for the user to retrieve later

8. DB & MINING Lab., Dept. of Computer & Information Science 8 Architecture of Context-aware Systems Different approaches context-aware systems - Direct sensor access:  the client s/w gathers the desired information directly form these sensor without layer for gaining and processing sensor data  drivers for the sensor are hardwired into the application  law extensibility, impossibility of managing multiple concurrent sensor access - Middleware based:  a layered architecture with the intention of hiding low-level sensing details  high extensibility - Context Server  middleware base architecture + distributed approach  gathering sensor data  context server

9. DB & MINING Lab., Dept. of Computer & Information Science 9 Abstract Architecture(1) application storage/management preprocessing raw data retrieval sensor Sensor - sensing hardware + every data source which provide usable context information  Virtual sensors : sourcing context data from s/w  Logical sensors : combining physical sensor + virtual sensor - Categories of Sensor  Physical sensors

10. DB & MINING Lab., Dept. of Computer & Information Science 10 Abstract Architecture(2) application storage/management preprocessing raw data retrieval sensor raw data retrieval - raw data retrieval by using appropriate drivers for physical sensor and APIs for virtual and logical sensors preprocessing - reasoning and interpreting (transforming) - aggregating of raw context data - solving of sensing conflicts

11. DB & MINING Lab., Dept. of Computer & Information Science 11 Abstract Architecture(3) application storage/management preprocessing raw data retrieval sensor storage/management - storing context history application - implementing actual reaction on different events and context-instances

12. DB & MINING Lab., Dept. of Computer & Information Science 12 Existent Context-aware Applications Categories of Context-aware Applications - Office and Meeting Tools - Tourist Guide - Memory Aids - Context-aware Home

13. DB & MINING Lab., Dept. of Computer & Information Science 13 Office and Meeting Tools(1) The Active Badge System(1992) - Group: Olivetti Research Ltd. - the first context-aware applications. - persons with badges that transmitted IR-signal - after sensing a location of a person, forwarding calls to the closest phone

14. DB & MINING Lab., Dept. of Computer & Information Science 14 Office and Meeting Tools(2) The ParcTab System - Group: Xerox Palo Alto Research Center - ParcTab: small wireless palm-sized computer - presenting information about the room the user was in - helping the user find the most convenient local resource, e.g. the nearest printer - Locating other person with ParcTab  Displaying the location information on Active map ParcTab terminal

15. DB & MINING Lab., Dept. of Computer & Information Science 15 Office and Meeting Tools Gaia - Group : Dept. of CS, University of Illinois - extending typical operating system concepts to include context, location awareness, mobile computing devices and actuators like door locks and light switches

16. DB & MINING Lab., Dept. of Computer & Information Science 16 Tourist Guides GUIDE - Group: Lancaster University(96~99) - terminal: Fujitsu Teampad 7600 portable PC - supporting Lancaster city visitor - screenshot of GUIDE

17. DB & MINING Lab., Dept. of Computer & Information Science 17 Memory Aids(1) Forget-Me-Not - Group: Rank Xerox Research Center - PDA system that records where its user is, who they are with, whom they phone, and other autobiographical information and store it in a DB for later query - terminal: ParcTab

18. DB & MINING Lab., Dept. of Computer & Information Science 18 Memory Aids(2) StartleCam - Group: MIT Media Lab. - wearable computer (with digital camera, digital modem) + skin conductivity sensor

19. DB & MINING Lab., Dept. of Computer & Information Science 19 Context-aware Homes(1) EasyLiving - Group: Vision Group at Microsoft Research - computer vision for person-tracking and visual user interaction. - multiple sensor modalities combined. - use of a geometric model of the world to provide context. - automatic or semi-automatic sensor calibration and model building. - fine-grained events and adaptation of the user interface. - device-independent communication and data protocols.

20. DB & MINING Lab., Dept. of Computer & Information Science 20 상황 인식 서비스 요소 기술 (1) 상황 인식 서비스 요소기술들 1. 상황정보 감지 기술 (context sensing) 2. 상황정보의 변화 감지 기술 (context change sensing) 3. 상황정보 모델링 기술 (context modeling) 4. 상황정보 융합 및 추론 기술 (context fusion & reasoning) 5. 상황정보 교환 기술 (context exchange) 6. 상황정보 툴킷 기술 (context toolkit) 7. 상황인식 서비스 묘사 및 발견 기술 (context-aware service description language) 8. 상항인식 서비스 구조 기술 (context-aware service infrastructure)

21. DB & MINING Lab., Dept. of Computer & Information Science 21 상황인식 서비스 요소 기술 (2) 1. 상황정보 감지 기술 - 상황정보 수집 경로 :  사용자 인터페이스, 센서, 센서 네트워크 - 위치 인식 시스템의 분류  매크로 위치인식 시스템 – 광역 위치 인식 시스템 ( 주로 GPS 사용 )  마이크로 위치인식 시스템  실내나 지하, 건물 밀집 지역의 위치 인식 제공  적외선 기반, 초음파 기반, 무선랜 기반, 영상 기반 시스템  Ad hoc 위치인식 시스템  위치인식을 위한 추가적인 h/w 도입이 어려운 환경에서 사용  위치정보를 가지는 레퍼런스 노드와의 무선 링크 연결성만으로 위치를 계산

22. DB & MINING Lab., Dept. of Computer & Information Science 22 상황인식 서비스 요소 기술 (3) 2. 상황정보의 변화 감지기술 - 상황정보의 변화 주기에 따라 주기적인 polling 이나 상황정보가 특정조건을 만족할 경우에만 reporting 3. 상황정보 모델링 기술 4. 상황정보 융합 및 추론 기술 − 상황정보 융합을 위한 계층적 모델링 요구 − 지능적인 추론이나 확률적인 메커니즘이 도입 5. 상황정보 교환기술 − application 간의 복잡한 자료구조를 교환하기 위한 통신 매커니즘 개발이 요구됨 6. 상황정보 툴킷 기술 − 코딩부담을 줄여주는 비주얼 프로그래밍 툴이 요구됨

23. DB & MINING Lab., Dept. of Computer & Information Science 23 상황인식 서비스 요소 기술 (4) 7. 상황인식 서비스 묘사 및 발견 기술 - 상황인식 서비스 묘사 언어 개발  WSDL 등이 사용 - 서비스 발견 기술  에이전트 기술이 주로 활용 8. 상황인식 서비스 구조 기술 − 센서, 네트워크, 서비스, 장치 등에 독립적인 infrastructure 개발이 요구됨

24. DB & MINING Lab., Dept. of Computer & Information Science 24 Part II Context-Aware Artifacts: Two Development Approaches http://idb.korea.ac.kr DB & Mining Lab. Korea University

25. DB & MINING Lab., Dept. of Computer & Information Science 25 Introduction Two development approaches for CA - Self-supported context awareness  designers build the ability to perceive context, reason with it, and act accordingly into the device or its dedicated hardware support. - Infrastructure-supported context awareness  designers obtain context aware capabilities by harnessing a hardware and software infrastructure external to and associated with the device ’ s space.

26. DB & MINING Lab., Dept. of Computer & Information Science 26 Context-aware mobile phones (1) SenSay - context-aware mobile phone that modifies its behavior based on its user's state and surroundings - uses a combination of sensors, including a voice microphone, an ambient-noise microphone, accelerometers, and a light sensor.

27. DB & MINING Lab., Dept. of Computer & Information Science 27  uses a finite-state machine to track the phone ’ s internal states (uninterruptible, high activity, normal, and idle) - automatically controls ringer and vibration levels, send short-message-service messages to callers, suggest calls to make  provides access to the user ’ s electronic calendar - Architecture of Sensay Context-aware mobile phones (2)

28. DB & MINING Lab., Dept. of Computer & Information Science 28 TEA-II (Technology for Enabling Awareness) - self-contained hardware device that plugs into mobile phones to add context-aware capabilities  The plug-in ’ s sensors: two light sensor, two microphones, a dual axis accelerometer, a digital temperature sensor and a touch sensor. - detects phone situations( in hand, on table, in pocket, and outdoors) - explores an exchange of context information between caller and callee. - adds a delay of up to 30 seconds.  Doesn ’ t consider internal phone state and action decision making Context-aware mobile phones (3)

29. DB & MINING Lab., Dept. of Computer & Information Science 29 Sensor-Doll - emits different sounds and music according to its situation and how users handle it Spoons and cups - Mediacup  MIT ’ s Chameleon Mug  MIT ’ s Intelligent Spoon Other context-aware objects (1)

30. DB & MINING Lab., Dept. of Computer & Information Science 30 Funiture - Chameleon Table - Smart Couch Medication dispensers - Context-aware pill bottle Other context-aware objects (2)

31. DB & MINING Lab., Dept. of Computer & Information Science 31 Cameras - Context photography:  uses sound and movement as context information and a set of custom-made computer graphics effects which affect images in real time  Example of context pictures Other context-aware objects (3)

32. DB & MINING Lab., Dept. of Computer & Information Science 32 Discussion Advantages of self-supported CA - By equipping objects with sensors, we can gain otherwise unavailable information about the object’s situation - Reduced dependency on external infrastructure Challenges of self-supported CA - embedding sensors unobtrusively - reasoning with the context efficiently and accurately given limited computational resources - enabling appropriate actions

33. DB & MINING Lab., Dept. of Computer & Information Science 33 Infrastructure-supported CA (1) Infrastructure for CA - acts as a mechanism for detecting, combining, and reasoning with different objects’ disparate context information - should not only consist of an architecture to represent objects and events, but also provide various services - a well-established, pervasive, reliable, and publicly accessible set of technologies that act as a foundation for other systems The relationship between CA artifacts and the context-awareness infrastructure

34. DB & MINING Lab., Dept. of Computer & Information Science 34 Infrastructure-supported CA (2) Weather-aware clothes hanger - hangers light up according to the day’s forecast RFID Chef prototype - Detects RFID-tagged grocery items on a kitchen counter - Display a list of recipes using these items as ingredients

35. DB & MINING Lab., Dept. of Computer & Information Science 35 Infrastructure-supported CA (3) Benefits of infrastructure support - hardware, platform, and language independence - ease of maintenance, in that administrators can change infrastructure-linked services, sensors, and artifacts while the system is running - improved context awareness economics - Users can update infrastructures and add new artifacts to an existing environment - can support complex context-awareness behavior

36. DB & MINING Lab., Dept. of Computer & Information Science 36 Conclusion Considerations for CA Developers - How can a system best acquire context? - Should we choose a self-supported or infrastructure- based approach (or some combination)? - How can the system reason with and use context for a particular application given the constraints of cost, reasoning efficiency, timeliness of action, and user intelligibility? - How can we program such entities to respond appropriately to richer contextual information? - How will users perceive automatic responses from artifacts and devices?