1. Tele-Action Objects (TAO) Presented by Kingsley Adeoye April 3, 2008

2. Outline Introduction Introduction Motivations Motivations TAO Basics TAO Basics Hypergraph Structure Hypergraph Structure Knowledge Structure Knowledge Structure TAO Application and Management TAO Application and Management Active Management System Active Management System Current Applications Current Applications TAO for MICE TAO for MICE Conclusion Conclusion

3. Tele-Action Object What is a Tele-Action Object According to [1], According to [1], “A tele-action object is a multimedia object with an associated hypergraph structure and corresponding knowledge base structure”“A tele-action object is a multimedia object with an associated hypergraph structure and corresponding knowledge base structure” Hypergraph Structure Hypergraph Structure Knowledge Structure Knowledge Structure

4. Motivations Many multimedia applications require Many multimedia applications require Effective presentation of objectsEffective presentation of objects Efficient transfer of objectsEfficient transfer of objects TAO approach provides: TAO approach provides: Effective presentation algorithmsEffective presentation algorithms Simple but efficient transfer protocolsSimple but efficient transfer protocols Automatic response to eventsAutomatic response to events

5. TAO Basics Hypergraph describes structure Hypergraph describes structure Knowledge describes actions Knowledge describes actions Tele-Action Objects can be: Tele-Action Objects can be: Simple piece of information or multimediaSimple piece of information or multimedia Complex network of multimediaComplex network of multimedia Object-oriented design Object-oriented design

6. Hypergraph Structure Node types Node types Simple or compositeSimple or composite Link types Link types Attachment Link: hierarchyAttachment Link: hierarchy Annotation Link: commentaryAnnotation Link: commentary Location Link: spatial distanceLocation Link: spatial distance Synchronization: temporal distanceSynchronization: temporal distance Reference Link: navigationReference Link: navigation

7. Hypergraph Example

8. Hypergraph Structure Media Types Media Types Text is coded alphanumeric data.Text is coded alphanumeric data. Graphics is formatted picture data.Graphics is formatted picture data. Image is pixel formatted picture data.Image is pixel formatted picture data. Moving-graphics also called animation, is the formatted data of a graphics sequence.Moving-graphics also called animation, is the formatted data of a graphics sequence. Moving-image is a sequence of image frames.Moving-image is a sequence of image frames. Audio is formatted sound data.Audio is formatted sound data. Video is a combination of synchronized moving-image and audio.Video is a combination of synchronized moving-image and audio. Form restricts user input, possibly with additional formula to generate the content automatically.Form restricts user input, possibly with additional formula to generate the content automatically. Live-demo is a program that can be run to provide an interactive demo.Live-demo is a program that can be run to provide an interactive demo. Composition is the media type of a composite node in TAO.Composition is the media type of a composite node in TAO.

9. Hypergraph Structure Bundled Nodes Bundled Nodes Simplifies presentation and communication of multimedia objectsSimplifies presentation and communication of multimedia objects Restricts partial access to bundled members maintaining integrityRestricts partial access to bundled members maintaining integrity Any bundled node must be fully referencedAny bundled node must be fully referenced For example, referencing to a bundled node containing text and audio will reference both sub-nodes For example, referencing to a bundled node containing text and audio will reference both sub-nodes

10. Hypergraph Structure Hypergraph decoding algorithms Hypergraph decoding algorithms Must consider network trafficMust consider network traffic Hardware capabilitiesHardware capabilities Best traversing sequence algorithmBest traversing sequence algorithm Priority of multimedia application/filePriority of multimedia application/file Link typesLink types Node resourcesNode resources Memory Memory Execution time Execution time

11. Knowledge Structure Four levels of knowledge Four levels of knowledge SystemSystem General actions for required by all TAOs; pre-fetch or decoding privileges General actions for required by all TAOs; pre-fetch or decoding privileges EnvironmentEnvironment Actions relating to set of circumstances for a group of TAOs Actions relating to set of circumstances for a group of TAOs TemplateTemplate Provides predefined hypergraph/knowledge format for frequently used TAOs Provides predefined hypergraph/knowledge format for frequently used TAOs PrivatePrivate Important action information for a single TAO from user Important action information for a single TAO from user

12. Knowledge Example

13. Knowledge Structure In Object Oriented Design In Object Oriented Design Each level knowledge is implemented using inheritance and polymorphismEach level knowledge is implemented using inheritance and polymorphism For example, system knowledge is the main class therefore every class inheriting this class will also have the same knowledge structureFor example, system knowledge is the main class therefore every class inheriting this class will also have the same knowledge structure Also, single TAOs can return different private knowledge when called using polymorphismAlso, single TAOs can return different private knowledge when called using polymorphism

14. Knowledge Structure Knowledge structures transforms simple static hypergraph objects into dynamic objects Knowledge structures transforms simple static hypergraph objects into dynamic objects Actions are pre-emptive and user- configurable Actions are pre-emptive and user- configurable

15. TAO Application and Management

17. Some actions may be associated with the presentation Some actions may be associated with the presentation TAO Specific TAO Specific Delete TAO after some timeDelete TAO after some time Request responses from usersRequest responses from users Environment Level Environment Level Require permission for some sub-TAOsRequire permission for some sub-TAOs System Level System Level Enable pre-fetch of multimediaEnable pre-fetch of multimedia

18. Active Management System Translates, executes and maintains tele-action objects Translates, executes and maintains tele-action objects Application domain allows users to develop and edit TAOs Application domain allows users to develop and edit TAOs System domain translates and executes these TAOs System domain translates and executes these TAOs

19. Important AMS Components Local Knowledge Base Local Knowledge Base Collection of ICsCollection of ICs Transition between states Transition between states Output messages to other ICs Output messages to other ICs Perform some specific action sequence Perform some specific action sequence Application Handler & Knowledge Generator Application Handler & Knowledge Generator Interface for TAO application toolInterface for TAO application tool Formats Hypergraph and Knowledge dataFormats Hypergraph and Knowledge data

20. Important AMS Components Interpretor Interpretor Receives formatted Knowledge dataReceives formatted Knowledge data Translates them into corresponding ICsTranslates them into corresponding ICs Event Filter Event Filter Retrieves important events from environmentRetrieves important events from environment Smart Engine Smart Engine Maintains Active Index with current ICsMaintains Active Index with current ICs Event driven system responding to internal IC messages or external event filter messagesEvent driven system responding to internal IC messages or external event filter messages

21. Current Applications Smart Multimedia Mail Smart Multimedia Mail Interface for email systemInterface for email system Create actions for eventsCreate actions for events Alarms, Statistics or even define template mail Alarms, Statistics or even define template mail

22. CASE Tools for Multimedia Software Engineering In traditional software engineering, involves formal well-defined steps In traditional software engineering, involves formal well-defined steps Requirements, Specifications, etc.Requirements, Specifications, etc. According to [3], multimedia applications cannot follow this traditional paradigm According to [3], multimedia applications cannot follow this traditional paradigm Minimal requirements are initially availableMinimal requirements are initially available Applications are constantly evolvingApplications are constantly evolving Multimedia IC Developer’s Environment (MICE) offers promising solutions to this problem Multimedia IC Developer’s Environment (MICE) offers promising solutions to this problem

23. MICE Application Development Distributed IC Manager Distributed IC Manager IC Builder IC Builder TAOML Interpreter TAOML Interpreter TAOMLHTML Template SR Grammar Prototyping tool Visual Specification Specification Builder Working System Specification tool

24. Conclusion TAO-based multimedia systems offer many advantages in efficiency and operation TAO-based multimedia systems offer many advantages in efficiency and operation AMS allows user-friendly access to create, maintain and edit TAOs AMS allows user-friendly access to create, maintain and edit TAOs Opportunities for optimizing presentation and communication algorithms will continue to make large network TAO-based systems realizable Opportunities for optimizing presentation and communication algorithms will continue to make large network TAO-based systems realizable

25. References [1] Hui-Jung Chang*, Tai-Yuan Hou+, Arding Hsu+, Shi-Kuo Chang, “Management and Applications of Tele-Action Objects”, Department of Computer Science, University of Pittsburgh, Siemens Corporate Research, Inc. [2]H. J. Chang, T. Y. Hou, A. Hsu and S. K. Chang, "Tele-Action Objects for an Active Multimedia System", Proc. of the Second Int'l IEEE Conference on Multimedia Computing and Systems, May 1995, Washington D.C. [3]Timothy Arndt, “The evolving role of software engineering in the production of multimedia applications”, Department of Computer and Information Science

26. Questions