The Mobile Environment  Environment o something which exists in a space, a concept defined with respect to the kind of system in focus (boundary).  Two (2) types of mobile environments defined by periods of interaction: o Changing location but long periods of interaction. E.g. Office  Train  Home etc. o Immediate changes due to volatility of instabilities of mobile communications (physics).  The mobile environment is characterized by change, the only thing that is constant is change.

Introduction  The design of adaptive applications is driven by one main inspiration - to give the best service possible under the given set of conditions.  The challenge in heterogeneous and mobile environments is to redefine the design of applications so that they can match the device and/or environmental constraints. o functional and environmental co-determination o adaptive delivery of multimedia data uses strategies that promote secure and timely delivery of data objects

Sources of Variability  Heterogeneous networks o variable link capacities (64kb/s Mbit/s and more) for ISDN, LANs, ATM o variable processing speeds of nodes o variable protocols (TCP/IP, ATM) stacks o congestion  Mobile networks o physics gives variable performance (turbulent environment, bandwidth, delay, jitter)  Clients o differ in CPU power, display resolution, memory, usage cost e.t.c

Shortcomings of traditional approaches  Concentrates on functional decomposition mostly o the environment is treated as a secondary issue.  Certain environment is assumed o specialisation and generalisation  Designs follow the “all or nothing” law  They tend to ignore variability in the supply and demand of resources o In reality, an application support environment is not static, hence some applications die in the face of variability  Recognition of variability has resulted in: o robust systems, reconfigurable systems, evolving systems, ad-hoc adaptive systems

An adaptation problem ????  The adaptation problem is formulated as: o How to transform actual situation A to a new situation B where the system is more stable ?  The efficiency will depend on the knowledge the system has about its environment (viewed as markers to an adaptation landscape): o features of the environment which are relevant for its general survival strategy; o features which determine whether the system will survive or not; o features which have causal relationships that can be manipulated by the system;

Some Guiding Laws and Principles  The law of requisite variety o Control can only be obtained if the variety of the controller is at least as great as the situation to be controlled.  The variety-adaptability principle o Systemic variety enhances stability by increasing adaptability  The over-specialization principle o Too much of a good thing may render systems unstable in the face of environmental change  Darkness principle o No system can be completely known

 The environment modification principle o To survive, systems have to choose two main strategies: One is to adapt to the environment, the other is to change it.  The maximum power principle o Those systems that survive in competition between alternative choices are those that develop more power inflow and use it to meet the needs of survival.  The feedback principle o The result of behaviour is always scanned and its success or failure modifies future behaviour (learning from experience) Some Guiding Laws and Principles

Adaptive Multimedia Representation VIDEOIMAGETEXT AUDIO MODALITY FIDELITY

Adaptation Spaces (Landscape)  Formally, an adaptation space is a set of adaptation cases, o partially ordered by the relation “more specialised than”  Case A is more specialised than Case B if the use condition of Case A is logically implied by the use conditions in Case B o that is the use conditions of Case B are a subset of the use conditions of Case A represented as a  b.  A complete adaptation space can be computed by taking the power set r(S) of all individual predicates that form a set S.  Being also lattice structured, not all cases are of interest

Example of adaptation Space A B C AB BC AC ABC No Predicate Holds (No Resources) All Predicates Hold (All Resources) More General More Specialised

Adaptation Space – Multimedia data VIDEOIMAGETEXTAUDIO MODALITY FIDELITY TextImageVideoAudio Client capabilty: Type: PDA Resolution: 320*240 RAM: 16MB WAP: On Media Player: Enable

Generalisation and Specialisation  The “meet” operation of two cases in the lattice provides a new case that generalises the two cases, and its use conditions is the intersection of the use conditions of the two cases o Case A MEET Case B  c = a  b  generalisation  Similarly, the “join” operation of two cases results in a new case that specialises the two cases, and its use conditions is the union of the use conditions of the two cases o C = Case A JOIN Case B  c = a  b  specialisation

Mechanisms of Adaptivity  Defining two spaces: the performance space, P, and resource space, R.  P is dimensioned along user oriented parameters (e.g play-out quality, response time, etc. )  The acceptance region, AR, of P is defined as the region in which the application is considered to be working properly.  R is dimensioned by resource characteristics in the operational environment (CPU power, memory, bandwidth, jitter, loss rates, etc.)  Formally: “for a given application class without adaptation there exist a mapping M: P  R that maps AR onto region B in R. Introducing adaptation changes M so that AR now maps onto a larger region A where A normally includes B. “

Data-based adaptation  Define predicates based on the environmental conditions.  The use conditions of each data type and fidelity is defined based on the environmental predicates (defines the conditions that should hold for each data use).  Support of a combination of different media or varying media fidelity can be reasoned about based on generalization or specialization.  The strategy is to monitor the environment and determine the usable multimedia objects at each point.  An adaptation policy determines the selection criteria out of the alternatives – e.g. best possible fidelity.

Data-based Adaptation to Environment and Device capabilities

Prototypes:Real-Time Multimedia Content speech input Adaptation manager Speech to Text Engine stream switch wireless bridge Adaptation manager Text to Speech Engine speech output mobile computer equipped with wireless card Event notification bus – change of environment or client capabilities

Prototypes:Real-Time Multimedia Content  Speech-Text conversion o Based on IBM Via Voice  Java based prototype o Java speech API o Java sound API o Java Speech Markup Language

Prototypes:Real-Time Multimedia Content 44k/16/Stereo CD quality playback 44k/16/mono medium quality playback 8k/8/mono Voice grade Full Text Textual summary Lowest connectivity key words Low connectivity AUDIO TEXT Bandwidth improvement Bandwidth reduction Case Properties e.g. media player enabled

Prototypes:Real-Time Multimedia Content  Developed for audio conferencing  The system operates on audio in different modes.  Text mode under very low bandwidth o speech recognition and speech synthesis  Problems: o Constrained vocabulary size to increase accuracy and processing speed o Background noise o Vocabulary and background noise can be included as adaptation parameter

Prototypes: Real-Time Multimedia Content  Utilization of bandwidth resources for a two line sentence: o “Hello” o “How are u?” o 8000 samples per second at 8 bits per sample (64 kbits per second) o Bandwidth saving is huge. 14 helloHow are u? bits bits 64 bits 176 bits Bandwidth Utilisation Time (sec)

Prototypes: Static Multimedia Content Proxy Policies Web Server Application Network Agent WML Wireless link HTTP Request /CCPP Content Request Send for adaptation Monitor Network WAP Gateway WML Request <rdf:RDF xmlns:rdf=" xmlns:prf=" = <prf:Defaults Vendor="Casio“ Model="E115" Type="PDA“ ScreenSize="320x240" CPU="PPC“ Keyboard="No" Memory="32MB” Speaker="Yes" /> < Quality of Service Parameter 1S1 Latency = <10 seconds Parameter 1S2 Bitrate > 9600kbps Disconnect Parameter 1S3 Bandwidth = 512Kbps /> /> RDF sent to client via CC/PP

Prototypes: Static Multimedia Content Composes web document based on o network conditions o client capabilities o User preferences  Emulation of CC/PP  Altering contents of web page o Removing images o Changing fidelity of images o Changing style of presentation for target device using XML and style sheets

Prototypes: Static Multimedia Content  Problems: o Catering for device types requires a knowledge base to assist in reasoning about data handling capabilities. o Limited levels of granularity supported by multimedia objects. o Sensible prioritisation of information o Mechanisms to compose web documents without losing some information not yet defined.

Experiences so far  Multimedia application o Audio conferencing  Adaptation space along o Audio at various quantisation levels o Speech to text o Vocabulary size  Static media o XML style sheets for device encoding

Conclusion  Adaptive strategies that manipulate digital multimedia objects (in real-time) can result in large savings of bandwidth and in addition extend the lifetime of some applications.  Device capabilities are an important aspect to be factored in during adaptation.  Computational opportunity needs to be evaluated as a basis for adaptation.