Composite Device Computing Environment: A Framework for Situated Interaction Using Small Screen Devices Thai-Lai Pham, Georg Schneider, Stuart Goose and Arturo Pizano A Multimedia Systems Presentation by Modupe Omueti Spring 2005

Introduction Introduction Motivation Motivation Implementation Implementation Current status Current status Challenges and future outlook Challenges and future outlook Schema

Introduction Mobile and wireless computing Mobile and wireless computing Information access and communication across a diverse range of environments through traditional computing systems such as WWW and Internet. Information access and communication across a diverse range of environments through traditional computing systems such as WWW and Internet. Ubiquitous computing Ubiquitous computing everywhere computing not a desktop-centric system rather a human-centric system. everywhere computing not a desktop-centric system rather a human-centric system. Context Context Set of environmental states and settings that determine an applications behaviour. Set of environmental states and settings that determine an applications behaviour.

Introduction Examples of context-aware systems include the Cyberguide, Active Badge System. Examples of context-aware systems include the Cyberguide, Active Badge System. Situated Computing Situated Computing Methodology imbuing applications and services with more personal and appropriate behaviour. Methodology imbuing applications and services with more personal and appropriate behaviour. Small Screen Devices Small Screen Devices These are contemporary communication devices that have become our everyday companion e.g. Personal Digital Assistants (PDAs), Symbian Mobile Phones. These are contemporary communication devices that have become our everyday companion e.g. Personal Digital Assistants (PDAs), Symbian Mobile Phones.

Motivation A dramatic growth in the small screen devices market. A dramatic growth in the small screen devices market. Desire of users to browse rich multimedia information and diverse services while mobile. Desire of users to browse rich multimedia information and diverse services while mobile. Size limitation of the small screen devices in rich content presentation. Size limitation of the small screen devices in rich content presentation. Places limit on legibility of text and image display. Places limit on legibility of text and image display. Graphics and multi-media originally intended for desktop PCs are normally not suitable for small screen devices. Graphics and multi-media originally intended for desktop PCs are normally not suitable for small screen devices.

Motivation A lot of hard disk and RAM space is required for good performance and storage of multimedia programs. A lot of hard disk and RAM space is required for good performance and storage of multimedia programs. Efforts focused on producing explicitly one service or shrink the content to match the capabilities of the small screen client. Efforts focused on producing explicitly one service or shrink the content to match the capabilities of the small screen client. Other factors determining the design of the small screen client devices can be improved on but the small physical screen size will remain for a long time. Other factors determining the design of the small screen client devices can be improved on but the small physical screen size will remain for a long time. The environment is becoming more and more equipped with several variety of computing resources. The environment is becoming more and more equipped with several variety of computing resources.

Motivation So why not make interactive use of the potential multitude of computing resources available to us in our immediate environment for diverse application and services

Composite Device Computing Environment (CDCE) This is the framework that exploits and incorporates surrounding available resources as another facet of situated computing to overcome the physical constraints of small screen devices. This is the framework that exploits and incorporates surrounding available resources as another facet of situated computing to overcome the physical constraints of small screen devices.

Composite Device Computing Environment (CDCE) Makes a pool from surrounding available computing resources. Makes a pool from surrounding available computing resources. The resources form it elements. The resources form it elements. Provides mechanisms for seamlessly exploiting and interacting with elements. Provides mechanisms for seamlessly exploiting and interacting with elements. Combines location and mobility with static computing resources. Combines location and mobility with static computing resources.

Composite Device Computing Environment (CDCE) Based on small screen client request, dynamically creates a unified composite or virtual device from pool. Based on small screen client request, dynamically creates a unified composite or virtual device from pool. This composite device is composed of an appropriate mix for the request. This composite device is composed of an appropriate mix for the request. Augments small screen device. Augments small screen device. Creates a mobile collaborative working environment. Creates a mobile collaborative working environment.

Implementation Working Scenario Working Scenario Concept Concept Design goals Design goals Framework components Framework components

Mobile Healthcare Scenario Each doctor is equipped with a PDA affording wireless access to the hospital patient information system. Doctor wishes to query a patients medical history, including symptoms, diagnoses, prescriptions and X-rays. PDA first detects the composite elements in the room, for example a TV and telephone, using the infrared interface.

PDA submits the doctors request, details about the detected devices to the CDCE gateway server. CDCE server routes the medical symptoms, diagnoses, and prescription information to the doctors PDA The X-ray image is redirected via RF to the TV for viewing. Mobile Healthcare Scenario

CDCE Concept Small screen client does not need to perform all tasks. Tasks not suitable for the PDA are outsourced to more appropriate device(s). Multi-tier communication infrastructure is a key unit of CDCE. Support of multi-modal interaction across the range of available CDCE supported computing resources

Design Goals Standard interfaces such as HTML/HTTP, WML/WAP, IR/Bluetooth. Additional functionality supported through the browser extension mechanisms. Adaptation for exploiting an ever changing number and diverse range of available elements in the surrounding environment. Optimization of information presentation and selection of output devices.

CDCE Framework Components Small screen client Small screen client CDCE Smart Gateway CDCE Smart Gateway Network communication model Network communication model The environment The environment

Fig. 1. Elements of CDCE Framework

Small screen client Detects resources available in close proximity. Detects resources available in close proximity. Reserves the resources. Reserves the resources. Sends request to Smart Gateway Server. Sends request to Smart Gateway Server. CDCE Framework Components

Smart Gateway Server Intelligently manages the pool of services available to the users. Creates and manages a composite device based on the small screen client request. Maps requests to services, and corresponding output to appropriate nodes in the virtual device. performs any dynamic conversion needed. CDCE Framework Components

Network Communication Model Manages the convergence of wireless and wire line networks. Manages the convergence of wireless and wire line networks. Manages corresponding communication protocols. Manages corresponding communication protocols. CDCE Framework Components

The environment Represents the dynamic pool of resources available at the users current location. Varies considerably from location to location. CDCE Framework Components

Current Status CDCE prototype consists of a PDA running Windows CE and multiple Windows NT workstations equipped with infrared serial interface adapters. PDA and Smart Gateway communication realized using HTTP over both Cellular Digital Packet Data (CDCP) and WLANs.

Current Status Smart Gateway and output client communication currently achieved using a LAN. Distributed Component Object Model (DCOM) used by Smart Gateway for remote invocation of processes without requirement for propietary client code Distributed Component Object Model (DCOM) used by Smart Gateway for remote invocation of processes without requirement for propietary client code

How the prototype works PDA client detects NT workstations PDA client detects NT workstations Sends request to Smart Gateway Sends request to Smart Gateway Gateway responds by initiating requested processes on the NT workstations Gateway responds by initiating requested processes on the NT workstations Current Status

Challenges and Future Outlook Intelligent information management and distribution Intelligent information management and distribution Contents adaptation Contents adaptation Splitting Splitting Conversion Conversion Filtering Filtering Smart delivery of information Smart delivery of information

Mobile user interface Mobile user interface Modes of interaction with the environment Modes of interaction with the environment Abdication Abdication Cooperative Cooperative Exclusive Exclusive Network Architecture Network Architecture PDA-Environment-Gateway-Environment Communication PDA-Environment-Gateway-Environment Communication Security Security Challenges and Future Outlook

The End