SMM5101 (ADVANCED MULTIMEDIA PROGRAMMING) Review of Multimedia Programming

Outline  Objects,  Environment, and  Frameworks for Multimedia Programming  Objects,  Environment, and  Frameworks for Multimedia Programming

Why Objects?  Multimedia involves multiple type of object  Nontemporal object types  Temporal object types  Multiple data type indicating the different form of data that are present  Different data types anticipating the object- oriented approach  Multimedia involves multiple type of object  Nontemporal object types  Temporal object types  Multiple data type indicating the different form of data that are present  Different data types anticipating the object- oriented approach

Why Objects? (cont…)  Classification data types based on similarity in their representations and operations so as to help understand how to model data types using object-oriented techniques.  Object-oriented technique appears to be natural fit between multimedia programming  Classification data types based on similarity in their representations and operations so as to help understand how to model data types using object-oriented techniques.  Object-oriented technique appears to be natural fit between multimedia programming

Why Objects? (cont…)  There are some reasons for this affinity, include:  Encapsulation  Modularity  Extensibilty  Portability and cross-platform development  Software legacy  There are some reasons for this affinity, include:  Encapsulation  Modularity  Extensibilty  Portability and cross-platform development  Software legacy

Why Objects? Encapsulation  MM programmers face a challenging environment 1.Concepts from many areas (including audio recording, video production, animation and music) 2.MM applications involve special hardware with idiosyncratic interfaces and operating procedures  Strengths of OOP is its ability to encapsulate information and help shield programmers from many of the details of particular media and pieces of hardware  MM programmers face a challenging environment 1.Concepts from many areas (including audio recording, video production, animation and music) 2.MM applications involve special hardware with idiosyncratic interfaces and operating procedures  Strengths of OOP is its ability to encapsulate information and help shield programmers from many of the details of particular media and pieces of hardware

Why Objects? Modularity  The equipment used in audio and video relies to a larger and larger extend on digital technology  Mixers to editing suites and special effect devices are becoming programmable and more interconnectable  Processing functionality of studios is moving available to applications running on PCs  OOP is well suited to capturing the complex interfaces of media processing services in a modular form which is easy for application developers to use  The equipment used in audio and video relies to a larger and larger extend on digital technology  Mixers to editing suites and special effect devices are becoming programmable and more interconnectable  Processing functionality of studios is moving available to applications running on PCs  OOP is well suited to capturing the complex interfaces of media processing services in a modular form which is easy for application developers to use

Why Objects? Extensibilty  The requirements and the nature of multimedia applications are evolving and far from stable  Database system multimedia, virtual environments, conferencing system etc. applications should be able to adapt changes in interface technology, media formats and hardware platform  OOP addresses the need for extensibility by offering mechanisms for enhancing and extending existing code  The requirements and the nature of multimedia applications are evolving and far from stable  Database system multimedia, virtual environments, conferencing system etc. applications should be able to adapt changes in interface technology, media formats and hardware platform  OOP addresses the need for extensibility by offering mechanisms for enhancing and extending existing code

Why Objects? Portability and cross-platform development  MM applications run on different platform and tolerate hardware variations within platforms - require rewriting application for every time to porting from platform to another  OO interfaces can make platform dependencies more explicit and so simplify both cross-platform development  MM applications run on different platform and tolerate hardware variations within platforms - require rewriting application for every time to porting from platform to another  OO interfaces can make platform dependencies more explicit and so simplify both cross-platform development

Why Objects? Software legacy  Many software developers face a legacy problem - the need to maintain compatibility with earlier applications but not for multimedia.  MM applications design is less constrained by existing software and developers are free to exploit new techniques such as OOP languages.  Many software developers face a legacy problem - the need to maintain compatibility with earlier applications but not for multimedia.  MM applications design is less constrained by existing software and developers are free to exploit new techniques such as OOP languages.

Objects: instance variables and methods  Objects encapsulate both state and behavior  Object is a programming language construct that identifies a collection of data items and a collection of operations  Data items - object’s instance variable  Operations - object’s method  Meant to be the only interface by which objects are manipulated  Objects encapsulate both state and behavior  Object is a programming language construct that identifies a collection of data items and a collection of operations  Data items - object’s instance variable  Operations - object’s method  Meant to be the only interface by which objects are manipulated

Objects: instance variables and methods play_mc.onRelease = function() { plane_mc.play(); }; ( Make movieclip plane_mc play when control movieclip play_mc is clicked ) function flyPlane() { plane_mc.play(); } play_mc.onRelease = flyPlane; ( Define a function to make plane_mc play. Assign that function to play_mc's onRelease event, so that when play_mc is clicked, plane_mc will play ) play_mc.onRelease = function() { plane_mc.play(); }; ( Make movieclip plane_mc play when control movieclip play_mc is clicked ) function flyPlane() { plane_mc.play(); } play_mc.onRelease = flyPlane; ( Define a function to make plane_mc play. Assign that function to play_mc's onRelease event, so that when play_mc is clicked, plane_mc will play )

MMP: Environment  MM systems is an independent discipline of study in computer science and engineering  After a number of commercial products and prototype systems have been built, then, it has been observed that the current programming paradigm for developing multimedia software needs some improvement  MM systems is an independent discipline of study in computer science and engineering  After a number of commercial products and prototype systems have been built, then, it has been observed that the current programming paradigm for developing multimedia software needs some improvement

MMP: Environment  MM Programming languages are best used for applications with:  a large amount of data handling  a smaller number of screen formats  non-standard hardware and software  a need to control system parts  MM Programming languages are best used for applications with:  a large amount of data handling  a smaller number of screen formats  non-standard hardware and software  a need to control system parts

 In a way of compelling the ideas of object orientation to multimedia programming –MM framework shows the encompassing of the essential object and operations that appear in MM applications  MM framework consists of interrelated abstract classes which are tailored and specialized for different MM platforms.  In a way of compelling the ideas of object orientation to multimedia programming –MM framework shows the encompassing of the essential object and operations that appear in MM applications  MM framework consists of interrelated abstract classes which are tailored and specialized for different MM platforms. Multimedia Framework

Platform #1 Abstract classes Concrete classes Platform #2 Abstract classes Concrete classes Platform #n Abstract classes Concrete classes Abstract Classes Specialization System Programming Interface Application Programming Interface (API) … Environment

Multimedia Framework  Abstract classes – a group of methods and instance variables which shared by several subclasses where their definition are partially or deferred to the subclasses  Concrete classes – the methods and instance variables are fully defined  Abstract classes – a group of methods and instance variables which shared by several subclasses where their definition are partially or deferred to the subclasses  Concrete classes – the methods and instance variables are fully defined

Multimedia Framework  API, both classes and the platform form a specific multimedia programming environment  There are factors delaying the portability of API: 1.Platform need not offer the same capabilities, or API may be supported on some platforms but not others 2.The performance of platform differs 3.Some applications bypass the API and directly access platform- dependently 4.No consensus on system-level support for MMP (notes: design of network and operating system is currently an active research area for MMP)  API, both classes and the platform form a specific multimedia programming environment  There are factors delaying the portability of API: 1.Platform need not offer the same capabilities, or API may be supported on some platforms but not others 2.The performance of platform differs 3.Some applications bypass the API and directly access platform- dependently 4.No consensus on system-level support for MMP (notes: design of network and operating system is currently an active research area for MMP)

Multimedia Framework  Framework should be robust, well-documented, and complete  Requirement for MM Programming includes:  Economy of concept  Should based on a small number of concept, otherwise it becoming a maze of media-specific detail  Proper identify general concept that apply across media types  Open  Extendable to incorporate with new media types, new data representation, and new hardware capabilities as they become available  Framework should be robust, well-documented, and complete  Requirement for MM Programming includes:  Economy of concept  Should based on a small number of concept, otherwise it becoming a maze of media-specific detail  Proper identify general concept that apply across media types  Open  Extendable to incorporate with new media types, new data representation, and new hardware capabilities as they become available

Multimedia Framework  Queryable  should specify interface for querying environments concerning their capabilities  Application produced can recognize missing functionality and adapt their behaviour  Distribution  Should help partition applications in a way that facilities distribution  The object within the framework should correspond to easy-to- distribute units or subsystem  The utility of a framework is greatly diminished if it conflicts with distribution  Queryable  should specify interface for querying environments concerning their capabilities  Application produced can recognize missing functionality and adapt their behaviour  Distribution  Should help partition applications in a way that facilities distribution  The object within the framework should correspond to easy-to- distribute units or subsystem  The utility of a framework is greatly diminished if it conflicts with distribution

Multimedia Framework  Scalable  Should support scalable media representation  if media representation are scalable, applications can increase quality as platform performance increases  High-level interface  Should provide high-level interface for media synchronization, media composition, device control, database integration, and concurrent media processing activities  These operations are central to multimedia programming  Scalable  Should support scalable media representation  if media representation are scalable, applications can increase quality as platform performance increases  High-level interface  Should provide high-level interface for media synchronization, media composition, device control, database integration, and concurrent media processing activities  These operations are central to multimedia programming

 The advantages of MM programming languages include:  producing applications that run fast  accessing system calls directly  being able to control non-standard hardware and software  being usable for a wide variety of platforms.  The advantages of MM programming languages include:  producing applications that run fast  accessing system calls directly  being able to control non-standard hardware and software  being usable for a wide variety of platforms. MMP Advantage

MMP Disadvantange  The disadvantages of programming languages include:  slower development time  need for skilled programmers  difficulty in debugging  unsuitability for prototyping  The disadvantages of programming languages include:  slower development time  need for skilled programmers  difficulty in debugging  unsuitability for prototyping

Conclusion  C++ is currently the most popular language for multimedia design and implementation:  is object-oriented that has numerous compilers for many platforms, and can access system part  C++ is currently the most popular language for multimedia design and implementation:  is object-oriented that has numerous compilers for many platforms, and can access system part